advanced examples and ideas
DESCRIPTION
Advanced Examples and Ideas. Evolutionary Programming. Neural Net. EXPERT SYSTEM. Fuzzy System. Neuro -Fuzzy System. UNIVERSAL SEARCH SYSTEM. Three Layer Evolutionary Approach. Local perceptions, such as “bald head” or “long beard”. Encoded behaviors or internal states. Time intervals. - PowerPoint PPT PresentationTRANSCRIPT
Advanced Advanced Examples and Examples and
IdeasIdeas
EXPERT SYSTEM
Neural Net Neural Net Evolutionary Evolutionary ProgrammingProgramming
Fuzzy SystemFuzzy System Neuro-Fuzzy Neuro-Fuzzy SystemSystem
UNIVERSAL UNIVERSAL SEARCH SEARCH SYSTEMSYSTEM
Three Layer Evolutionary Approach
Evolve Behaviors
Evolve Motions
Evolve Perceptions
Global perceptions, possibly encoded such as “narrow Corridor” or “beautiful Princess”
Local perceptions, such as “bald head” or “long beard”
Behaviors such as “go forward until you find a wall, else turn randomly right or left
Encoded behaviors or internal states
Time intervals
Motions as timed sequences of encoded actions, for instance RFRFLL
Evolve in hierarchyEvolve in hierarchy
1. Together or separately2. Feedback from model or from real world3. First evolve motions and encode them.4. Then evolve behaviors.5. Finally develop perceptions.
If you see a beautiful
princess go to her and bow low.
If you see a dragon escape
Go to the end of the corridor and
then look for food
Evolve in hierarchyEvolve in hierarchyavoid
obstacles
Execute optimal motions
Save energy
Look for energy sources in advance
Execute actions that you enjoy
What if robot likes to play soccer and sees the ball but is low on energy?
Optimizing a motion
Parking a Parking a TruckTruck
Find the control
Solving this analytically would be very difficult
Question; How to represent the chromosomes?
Here you see several snapshots of a “movie” about parking a truck, stages of the solution process.
• t is time u
Another example
Learning Learning Obstacle Obstacle AvoidingAvoiding
Similar to Braitenberg Vehicle but has 8 sensors
how• How would you
represent chromosomes?
• Design Crossovers?
Input and output data are some form of MV logic
1. Robot can move freely but has to avoid obstacles
2. This can be like the lowest level of behaviors in subsumption or other behavioral architecture for all your robots
The key to success is often in fitness function
Remember the goal when you create the fitness function
Number of collisions
• Time of learning
When you train longer you decrease the number of collisions
Applications Applications and and
ProblemsProblems
General GA SchemaInitialize the population
Compute Fitness
Selection
Crossover
Mutation
Compute Fitness
Stop Conditions?
Solution
Yes
No
Evolutionary Methods• Optimization problems:
– Single objective optimization problems– Multi-Objective optimization Problems
• Combine with other approaches.
• Search Problems (Path search)• Optimal multi-robot coordination• Multi-task optimization• Optimal motion planning of robot arms (Trajectory planning of
manipulators )• Motion optimization Motion optimization (optimization of controller parameters -
morphology in different control schemas)– PID (PI)– FuzzyFuzzy– NeuralNeural– Hybrid (neuro-fuzzy)Hybrid (neuro-fuzzy)
• Path planning and tracking (mobile robots)• Optimal motion planning of robot arms
– Trajectory planning of manipulators• Vision – computational optimization
More examples of problems in which we use evolutionary algorithms, neuro-fuzzy, and similar
methods.
• Evolutionary Algorithms - Related techniques:– Ant colony optimization (ACO) – Particle swarm optimization – Differential evolution– Memetic algorithm (MA)– Simulated annealing – Stochastic optimization – Tabu search – Reactive search optimization (RSO) – Harmony search (HS) – Non-Tree Genetic programming (NT GP) – Artificial Immune Systems (AIS)– Bacteriological Algorithms (BA)
What are these “other algorithm”?
1. You can try them in your homework 1 if GA or GP is too easy for you.
2. Using them gives you higher possibility of creating a successful superior method for a new problem
GA-operators• Selection
– Roulette– Tournament– Stochastic sampling– Rank based selection– Boltzmann selection– Nonlinnear ranking selection
• Crossover– One point– Multiple points
• MutationRead in Auxiliary Slides about
these methods. Or invent your own operators
for your problem.
Your design parameters to be decided
• Genotype length– Fixed length genotype– Variable-length genotype
• Population– Fixed population– Variable population– Species inside population– Geometrical separation
Drawbacks of GA
• time-consuming when dealing with a large population
• premature convergence • Dealing with multiple objective problems
Solutions• Niches• Islands• Pareto approach• Others
More examples of using GA in robotics
Trajectory Trajectory Planning Planning ProblemsProblems
GA and Trajectory Planning1. GA techniques for robot arm to identify the optimal
trajectory based on minimum joint torque requirements (P. Garg and M. Kumar, 2002)
2. path planning method based on a GA while adopting the direct kinematics and the inverse dynamics (Pires and Machado, 2000)
3. point-to-point trajectory planning of flexible redundant robot manipulator (FRM) in joint space (S. G. Yue et al., 2002)
4. point-to-point trajectory planning for a 3-link (redundant) robot arm, objective function is to minimizing traveling time and space (Kazem, Mahdi, 2008)
Projects last years
Optimal path generation of robot manipulators
1. Control Schema 2. Robotic arm – kinematic model3. Controller type4. Objective function - optimal path 5. Optimization algorithm (method)6. GA use smooth operators and avoids sharp
jumps in the parameter values.
• Adaptive Control Schema Adaptive Control Schema – Track Control error function between outputs of a real system and mathematical model
• What we optimize?• Which parameters must be optimized?• How many objectives (single –objective or multiobjective)?• Collision free? (How to model collision in GA?)
EXAMPLE: EXAMPLE: Combining kinematics, Combining kinematics, control, evolutionary control, evolutionary
and neuraland neural
• Three join ManipulatorThree join Manipulator
• A three-joint robotic manipulator system has three inputs and three outputs.
• The inputs are the torques applied to the joints and the outputs are the velocities of the joints
• For n-DOF we will have n inputs ui, i=1…n, (ui ↔ i)• Controller
– PID (PI)– Neural network (multilayer perceptron, recurrent NN, RBF
based NN)– Fuzzy – Neuro-Fuzzy (hybrid)
Design of robotic Design of robotic controllerscontrollers
• NN: We must to adapt the weights and eventually the bias The chromosome:
• Adapt the weights kijw
Use of Neural NetworksUse of Neural Networks
FUZZY LOGICFUZZY LOGIC• Fuzzy Logic
• Aggregation of rules• defuzzification• free-of-obstacles workspace (Mucientes, et. al, 2007)• wall-following behavior in a mobile robot
1 1 1 1:i i i i i i i i iN N M MR if x is A and x is A then y is B and y is B
• Learning of fuzzy rule-based controllers• Find a rule for the system
Step 1: evaluate population;Step 2: eliminate bad rules and fill up population;Step 3: scale the fitness values;Step 4: repeat NI iterations for Step 4 to Step 9
Step 5: select the individuals of the population; Step 6: crossover and mutate the individuals; Step 7: evaluate population; Step 8: eliminate bad rules and fill up population;
Step 9: scale the fitness values. Step 10: Add the best rule to the final rule set.
Step 11: Penalize the selected rule. Step 12: If the stop conditions are not fulfilled go to Step 1
Learning FUZZY LOGIC ControllersLearning FUZZY LOGIC Controllers
Encoding fuzzy controls
• The chromosome encode the rules:
• Sn is constant in this application but it can be also variable to be optimized
• wall-following behavior of the robot– the robot is exploring an unknown area– moving between two points in a map
• Requirements– maintain a suitable distance from the wall that is being followed– to move at a high velocity whenever the layout of the environment is permitting– avoid sharp movements (progressive turns and changes in velocity)
1 1 2 2( , , , , , )i i i i i i iNV NVC b c b c b c
Path-based robot behaviors
• The requirements are “encoded” in Universes of discourse and precisions of the variables– right-hand distance (RD)– the distances quotient (DQ), based on left-hand
distance– Orientation– linear velocity of the robot (LV)– Linear acceleration– Angular velocity
• Path of the robot (simulated environments)
Fast, reliable, no harm to robot or to environment
• This is useful for out PSU Guide Robot1. Do not harm humans2. Do not harm robot
• Fixed points: the desired Cartesian path Pt is given the problem is to find the set of joint paths P in order to minimize the cumulative error between desire and real path during trajectory
Pk is the kinematic model
• Free end points case
1 1
2 1 2
( , ) ( , ) ( )j jN NN
t ri j i
E P i j P i j E i
( )r kP F P
Minimize the cumulative error
Find the set of joint paths, next smooth it
Weighted Global Fitness• fitness function (minimization)
• Global fitness: Linear function of individual objectives
Fot – excessive driving (sum of all maximum torques), fq – the total joint
traveling distance of the manipulator, fc - total Cartesian trajectory length, tT
- total consumed time for robot motion
• Penalty function• Population initialization (probability distribution)
– Random uniform– Gaussian
1 2 3 4total ot q c TF f f f t
1
1F
E
exampleexample
Drug Drug Delivery Delivery ProblemProblem
Drug delivery using microrobots (Tao, et. al, 2005)
1. (GA)–based area coverage approach for robot path planning.
2. Drawbacks of most currently available drug delivery methods are that the drug target area, delivery amount, and– release speed are hard to be precisely controlled.
3. It is very difficult or impossible to eliminate side effects.4. Open issues
1. actively control the delivery process2. Access to appropriate areas that cannot be reached using traditional
devices5. Current Issues
1. On-line path planning (solve unexpected obstacles problem)2. Optimal path planning (efficiency, path planning)
• microcontroller is used to guide the robot movement• GA-based approach uses fine grid cell decomposition
for area coverage• Because the robot will move cell by cell, the start point
of chromosomes has to be changed dynamically whenever the robot reaches the center of a cell
• The end point of a chromosome is not fixed and needs to be determined by applying GA operators.
• The robots may move from the center of a cell to its 8 adjacent cells along 8 directions.
• some obstacles are unknown before drug delivery (the robot discover these obstacles during the motion)
• Expandable chromosomes• Deleting the path• Crossover operator
• New mutation operators– Travel further– Delete– Reverse delete– Stretch– Shortcut
• The algorithm keep mind the visited nodes• Extension to operational research?
Other applications using evolutionary algorithms
• Autonomous mobile robot navigation - Path planning using ant colony optimization and fuzzy cost function evaluation (Garcia, et. al, 2009).
• Legged Robots and Evolutionary Design• Optimal path and gait generations (Pratihar, Debb,
and Gosh, 2002) – 0/1 absence or presence of rule
• six-legged robot• collision-free coordination of multiple robots
(Peng and Akela, 2005)
What if you want to optimize two parameters at the same time?
Pareto Pareto OptimizationOptimization
Definitions
ophelimity
noun economic, Economic satisfaction. The ability to please another.
Pareto Evolutionary
Methods
• We want to optimize both functions f1 and f2
What is better this or this?
Biobjective means two
objectives to reach
• We have x and y, two objectives here
Pareto solutions for different algorithms
Pareto Pareto FrontFront
Pareto frontPareto front• The single objective optimisation problem (SOP)
conduct to a minimization (or maximization) of one cost function, less or more complex, that is a single objective is taken into account.
• Conversely, the multi-objective optimization problem takes into account two or more objective that has to be minimized (or maximized) simultaneously.
• Some objectives can be in competition, so a simultaneous minimization is not possible, but only a trade-off among them. • Some time, the number of objectives can be high, like 16
objectives or more that make the multi-objective optimization problem (MOP) and interesting and challenging area of research
Example of Pareto Optimization of two parameters
Optimization Optimization of Airplane of Airplane
WingsWings
• Two objectives: Two objectives: Maximize lift, and minimize drag
• Two objectives: Two objectives: Maximize lift, and minimize drag
* In most of the design space the red method is better than the blue method* It is good to use many Pareto methods and modify parameters
Multi-Pareto1. We optimize many
parameters, 2. We may switch between
subsets of them.3. Subsets can have two
elements each.
Three-dimensionalThree-dimensional Minimization Minimization
ProblemProblem
Pareto Pareto FrontFront
General multiobjective General multiobjective optimization problemoptimization problem
• The multiobjective optimization problem could be generally formulated as minimization of vector objectives Jt(x) subject to a number of constraints and bounds:
Pareto-optimal set Pareto-optimal set 1. In the case of competing objectives a trade-off is involved
such a problem usually has no unique solution. – Instead, we can admit a set of solutions, equally valid non-
dominated as a set of alternative solutions known as Pareto-optimal set
2. In what follows we assume without loss of generality that all the function objectives must be minimized.– If we have a maximization case fi we simply minimize the
function -fi.
3. For any two points that are usually named candidate solutions V1,V2, V1 dominates V2 in the Pareto sense (P-dominance) if and only if the following condition hold
},...,{),()(
},...,{),()(
121
121
1
1
mjoneleastatforVfVf
miVfVf
jj
ii
The Pareto setThe Pareto set
1. The Pareto set is the set of PO (Pareto-Optimal) solution in design domain and the Pareto Front (PF) is the set of PO solutions in the objective domain.
2. The most popular way to solving the MOP (Multi Objective Optimization Problem) is to reduce the minimization problem to a scalar form by aggregating the objectives in weighted sum, with the sum of weights constant:
3. The weighted sum method has a serious drawback, the method usually fail in the case of nonconvex PF.
n
ii
n
iiit wJwJ
11
1,min
Example of a clear picture of Pareto points
Nice properties
1. GA can provide an elegant solution for tradeoff among different minimization of cost function for each variable versus total cost or other variable.
2. Non-convex solutions
3. “Immigrants”, possible solution for jump from local minima.
4. Dealing with many variables (e.g. 16 variables)
Multi-Robots
• Pareto optimal multi-robot coordination with acceleration constraints (Jung and Ghrist, 2008)1. collection of robots sharing a common environment 2. each robot constrained to move on a roadmap in its
configuration space3. each robot wishes to travel to a goal while
optimizing elapsed time considering vector-valued (Pareto) optima
4. all illegal or collision sets are removed.
ConclusionsConclusions1. GA is not a universal panacea to optimization problems.
2. Coding the problem into a genotype is the most important challenge!
3. The best selection schema of individuals for crossover operator is difficult to be chosen apriori (tournament selection seems to be more promising)
4. A number of parameters are determined empirically:1. Size of population2. pc and pm even often values inspired from biology are given3. Other parameters in hybrid or more sophisticated GA
Good properties1. One of the most important element in the design of a decoder-based
evolutionary algorithm is its genotypic representation.
2. The genotype-decoder pair must exhibit efficiency, locality, and heritability to enable effective evolutionary search
3. locality, and heritability: 1. small changes in genotypes should correspond to small changes in
the solutions they represent, and 1. solutions generated by crossover should combine features of their
parents
SourcesSources
Dragos Arotaritei
exampleexample