genetic algorithms in robotics - uni-hamburg.de
TRANSCRIPT
Universitat Hamburg
MIN-FakultatFachbereich Informatik
GA’s in Robotics
Genetic Algorithms in Robotics
Julius Mayer
Universitat HamburgFakultat fur Mathematik, Informatik und NaturwissenschaftenFachbereich Informatik
Technische Aspekte Multimodaler Systeme
October 31, 2016
J. Mayer 1
Universitat Hamburg
MIN-FakultatFachbereich Informatik
GA’s in Robotics
Outline
1. IntroductionMotivationClassification
2. AlgorithmOverviewPhases
3. ApplicationGA’s in RoboticsNeuroevolution
4. DiscussionEvaluationConclusion
J. Mayer 2
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Introduction - Motivation GA’s in Robotics
Motivation
Problems that are hard to solve with classical optimizationbecause of
I too many parameters (intractability),I large search space,I non-differentiable objective functions,I varying numbers of variables within the optimization process,I a lacking mathematical function specification.
[1]
J. Mayer 3
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Introduction - Classification GA’s in Robotics
Classification
Biologic: Theory of evolution
I Adaptation of population to theenvironment
I Gradual, hereditary change inthe individuals of a species
I Information storage and transferthrough genomes
I Optimizing population fitness byreproduction [3]
J. Mayer 4
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Introduction - Classification GA’s in Robotics
Classification
Biologic: Theory of evolution
I Adaptation of population to theenvironment
I Gradual, hereditary change inthe individuals of a species
I Information storage and transferthrough genomes
I Optimizing population fitness byreproduction
[5]
J. Mayer 5
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Introduction - Classification GA’s in Robotics
Classification
Algorithmic: Stochastic optimization
I Can ’solve’ NP-hard problems
I Solution approximation
I Computational complexity asprohibiting factor
I e.g. ant colony & particle swarmoptimization
[7]
J. Mayer 6
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Algorithm - Overview GA’s in Robotics
Algorithm
1. Initialization
2. Evaluation
3. Selection
4. Mutation
5. Terminate
J. Mayer 7
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Algorithm - Phases GA’s in Robotics
Phase 1. Initialization
Initialize random population
I Choose appropriaterepresentation
I Individuals as encodedproblem solutions
I e.g. real values or binarystrings
1,67 2,34 1,04 5,83 2,10
101010011011010010101 [12]
J. Mayer 8
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Algorithm - Phases GA’s in Robotics
Phase 2. Evaluation
I Choose objective function
I Decode & evaluatechromosome
I Find minima in objectivefunction
I Survival of the fittest
[9]
J. Mayer 9
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Algorithm - Phases GA’s in Robotics
Phase 3. Selection
I Probabilistic selection
I Roulette wheel selection
I Tournament Selection
I Balance exploration &exploitation dilemma
J. Mayer 10
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Algorithm - Phases GA’s in Robotics
Phase 4. Mutation
I Assign mutation rateI Mutate
I BitflipsI Normal distribution
mutation
I RecombinationI CrossoverI Permutation
I Mutation drives change
[12]
J. Mayer 11
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Algorithm - Phases GA’s in Robotics
Phase 4. Mutation
I Assign mutation rateI Mutate
I BitflipsI Normal distribution
mutation
I RecombinationI CrossoverI Permutation
I Mutation drives change
[12]
J. Mayer 12
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Algorithm - Phases GA’s in Robotics
Phase 5. Termination
I If solution is sufficiently closeI Performance is satisfyingI Change is stagnant
J. Mayer 13
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Application - GA’s in Robotics GA’s in Robotics
GA’s in Robotics
I Evolutionary robotics
I Path planing
I Multivariate parameteroptimization
I Evolving artificial networkarchitectures & optimizingconnection weights (learning)
[6]
J. Mayer 14
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Application - Neuroevolution GA’s in Robotics
Neuroevolution of augmenting topologies (NEAT)
I Uses crossover
I Uses speciation to protectstructures
I Increment growth fromminimal structures
[10]
J. Mayer 15
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Application - Neuroevolution GA’s in Robotics
NEAT: Mutation
I Connection weights
I Network structure
I Add new connection(random weight)
I Add new node (splitconnection)
[10]
J. Mayer 16
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Application - Neuroevolution GA’s in Robotics
NEAT: Crossover
I Genes are lined up accordingto innovation number
I Matching Genes are randomlychosen
I Disjoint / excess are takenfrom fitter parent
[10]
J. Mayer 17
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Application - Neuroevolution GA’s in Robotics
NEAT for RoboCup Soccer
Keepaway (robot soccersubtask)I Large state space (unable to
explore exhaustively)I Only partial state
information for each agentI Continuous action spaceI Multiple teammates need to
learn simultaneously[4]
J. Mayer 18
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Application - Neuroevolution GA’s in Robotics
NEAT vs. Temporal Difference Methods
[11]
J. Mayer 19
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Application - Neuroevolution GA’s in Robotics
NEAT vs. Temporal Difference Methods
[11]
J. Mayer 20
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Discussion - Evaluation GA’s in Robotics
Evaluation
Challenges
I Get GA’s parameters right
I Solution encodingI Exploration vs. exploitation
I Premature convergence(local minima)
I slow convergenceI big search space
I Find representative fitnessfunction
Drawbacks
I Approximation instead ofexact solution
I Computational complexity asprohibiting factor
I No guaranteed convergenceto global optimum
I Non-exhaustive coverage ofthe complete solution space
J. Mayer 21
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Discussion - Conclusion GA’s in Robotics
Conclusion
I Inspired by biological evolution
I Stochastic optimization
I Population containing encodedproblem solutions (parameters)
I Different applications in robotics
I e.g. evolving ANN (NEAT) forRoboCup Soccer
I No guarantee for convergence toglobal optimum
[2]
J. Mayer 22
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Discussion - Conclusion GA’s in Robotics
References
[1] http://rednuht.org/genetic walkers/, 2016.
[2] https://cdn.meme.am/instances/41036988.jpg.2016.
[3] https://i.ytimg.com/vi/3ZJNyScv8to/maxresdefault.jpg, 2016.
[4] https://i.ytimg.com/vi/HHlN0TDgllE/maxresdefault.jpg.2016.
[5] http://www.hanskottke.de/wordpress/wp-content/uploads/2012/09/evolutions hip hop.jpg.2016.
[6] http://www.orocos.org.2016.
[7] http://www.turingfinance.com/wp-content/uploads/2014/10/Self-Organizing-Feature-Map-3.png.2016.
[8] John H Holland.Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, andartificial intelligence.U Michigan Press, 1975.
[9] Melanie Mitchell.Complexity: A Guided Tour.In Oxford, volume 1, chapter Cellular A, pages 145–159. Oxford University Press, Inc, New York, 2009.
J. Mayer 23
Universitat Hamburg
MIN-FakultatFachbereich Informatik
Discussion - Conclusion GA’s in Robotics
References (cont.)
[10] Kenneth O Stanley and Risto Miikkulainen.Evolving neural networks through augmenting topologies.Evolutionary computation, 10(2):99–127, 2002.
[11] Matthew E Taylor.Comparing Evolutionary and Temporal Difference Methods in a Reinforcement Learning Domain.pages 1321–1328.
[12] Mattias Wahde.Biologically Inspired Optimization Methods: An introduction.WIT Press, Boston, MA, 2008.
J. Mayer 24