algoritmos genéticos aplicados em machine learning
TRANSCRIPT
![Page 1: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/1.jpg)
1
Algoritmos Genéticos aplicados em Machine Learning
Controle de um Robo(em inglês)
![Page 2: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/2.jpg)
2
Robot Control using Genetic Algorithms
![Page 3: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/3.jpg)
3
Summary
• Introduction– Robot Control– Khepera Simulator
• Genetic Model for Path Planning– Chromosome Representation– Evaluation Function– Case Studies
• Conclusions
![Page 4: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/4.jpg)
4
The Robot Controller Problem
• Given a robot and a description of an environment, provide commands (motor speeds) to the robot, in order to achieve a path between two specified locations, which is collision-free and satisfies certain optimisation criteria.
(xi, yi)
(xf, yf)
![Page 5: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/5.jpg)
5
Optimisation Criteria
• Robot should:– attempt near-optimal paths– avoid obstacles– perform straight motion
• Controller should be independent of:– the robot’s environment– target location
![Page 6: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/6.jpg)
6
The Khepera Simulator
• Freeware mobile robot simulator (designed by Olivier Michel, University of Nice Sophia-Antipolis)
• User designed worlds
• Control algorithms can be written in C/C++• Robot’s position and angle reading
• 8 sensors (S0-S7): [0, 1023]• 2 motors (M1, M2): [-10, +10]
S0
S1S2 S3
S4
S5
S7 S6
M1 M2
Front identification
![Page 7: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/7.jpg)
7
Simulator Readings:sensors, position and angle
S0
S1S2 S3
S4
S5
S7 S6
M1 M2
Front identification
S0-S7: [0, 1023]S0-S7: [0, 1023]
1000
1000
X
Y
Robot’s World
S0
S1S2 S3
S4
S5
S7 S6
M1 M2
Front identification
angle of the robot with the world
αα : [-: [-ππ , , ππ ]]
xx
yy
0
obstacle not obstacle very
detected closed
![Page 8: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/8.jpg)
8
Control Mode
• To evolve the robot’s attitudes as it interacts with the environment
• Each robot action determines:– how well the controller performs with
respect a given task;– the next input stimuli to the controller.
• The controller should learnlearn as the robot interacts with the environment
![Page 9: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/9.jpg)
9
Controller Model
Genetic Genetic AlgorithmAlgorithm
evolves robot’s evolves robot’s attitudesattitudes
Sensors
Position
Robot’s Angle
Goal Location
Motor 2 Motor 1
KheperaKheperaSimulatorSimulator
![Page 10: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/10.jpg)
10
Proposed Modelbased on human behavior
IF Obstacle detectedObstacle detected
THEN
Avoid collision, forget targetAvoid collision, forget target
ELSE
SStraight to the target according to the traight to the target according to the
target directiontarget direction
END
![Page 11: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/11.jpg)
11
Sensors Reading Simplification
Sleft Sright
Sback
S0
S1S2 S3
S4S5
S7 S6
Sleft = ( S0 + S1 + S2 ) / 3
Sright = ( S3 + S4 + S5 ) / 3
Sback = ( S6 + S7 ) / 2
![Page 12: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/12.jpg)
12
Determining the Target Direction
(x,y)α = π/2
arrival position: (xf ,yf)
β
β = tan-1
[(yf - y)/(xf - x)]
β - απ/2
π
π/4
0
−π/4
−π/2
3π/4
−3π/4
Goal point toright side
Goal point infront
Goal point toleft side
Goal pointbehind
Direction =Direction =
![Page 13: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/13.jpg)
13
Model
IF ((((SSleftleft > L > L) or () or (SSrightright > L > L) or () or (SSbackback > L > L))))
THENObstacle detected, avoid collision, forget target
Proximity-sensor = highest value (SProximity-sensor = highest value (Sleftleft, S, Srightright, S, Sbackback))
ELSEObstacle not detected (collision-free), straight to the target
Target direction = Target direction = ββ - - αα
END
L=collision threshold=900
![Page 14: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/14.jpg)
14
Genetic Algorithm Modelling
• Problem
• Chromosome Representation
• Evaluation Function• Genetic Operators
• Techniques
• Parameters
![Page 15: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/15.jpg)
15
Chromosome Representation
M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2
TargetDirection(collision free) Obstacle detected
Left Right BackFront Left Right Back
Attitudes (Genes)
1 2 3 4 5 6 7
Which speed should be imposed to each motor in Which speed should be imposed to each motor in each situation the robot is?each situation the robot is?
![Page 16: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/16.jpg)
16
Evaluation Function
( )
10;10;10
**7
1
≤≤≤≤≤≤
= ∑=
AiDiVi
AiDiViFi
• Main objectives:– (V) speed: as high as possible– (D) straight motion: same motor speed for M1 e M2– (A) action: reach a target and avoid obstacles
• Calculated based on the contribution for each gene [1,7], at each step.
![Page 17: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/17.jpg)
17
Speed
max)*2/(|)2||1(| MMMVi +=
• Normalised sum of the absolute value of the motors speeds;
• Vi increases as both speeds increase• Whatever the robot does, it does quickly.
![Page 18: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/18.jpg)
18
Straight Motion
=++=
=]7,6,5,1[,2))max)*2()21((1(
]4,3,2[,1
iMMM
iDi
• It favours high positive speeds to both motors
• When the robot is not oriented to the target (2,3,4), D=1 avoids contradictory learning
![Page 19: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/19.jpg)
19
Action
• It considers the benefit of each gene regarding to:– obstacle avoidance– target closeness
=−=≤∆=∆=≤∆=∆
=]7,6,5[,max1
0,)0(])4,3,2[(,max
0,)0()1(,max
iifSS
AAielsegiandiifggi
AAielsediandiifddi
AAi
t
t
t
t
≠
== ∑
=
0,)(
0,0
1
TPiTPiAAi
TPiAi TPi
tt
TPiTPi = total of steps executed by attitude i = total of steps executed by attitude i
AAiAAi=action’s fitness at stept of attitude i=action’s fitness at stept of attitude i
![Page 20: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/20.jpg)
20
Action
• It considers the benefit of each gene regarding to:– obstacle avoidance– target closeness
=−=≤∆=∆=≤∆=∆
=]7,6,5[,max1
0,)0(])4,3,2[(,max
0,)0()1(,max
iifSS
AAielsegiandiifggi
AAielsediandiifddi
AAi
t
t
t
t
≠
== ∑
=
0,)(
0,0
1
TPiTPiAAi
TPiAi TPi
tt
TPiTPi = total of steps executed by attitude i = total of steps executed by attitude i
AAiAAi=action’s fitness at stept of attitude i=action’s fitness at stept of attitude i
Rates the distance variation to the target between
two consecutive steps, and the maximum distance in one step,
for collision free/front
![Page 21: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/21.jpg)
21
Action
• It considers the benefit of each gene regarding to:– obstacle avoidance– target closeness
=−=≤∆=∆=≤∆=∆
=]7,6,5[,max1
0,)0(])4,3,2[(,max
0,)0()1(,max
iifSS
AAielsegiandiifggi
AAielsediandiifddi
AAi
t
t
t
t
≠
== ∑
=
0,)(
0,0
1
TPiTPiAAi
TPiAi TPi
tt
TPiTPi = total of steps executed by attitude i = total of steps executed by attitude i
AAiAAi=action’s fitness at stept of attitude i=action’s fitness at stept of attitude i
Rates the angle variation between two consecutive steps, and
the maximum angle in one step, for collision freeleft, right, back
![Page 22: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/22.jpg)
22
Action
• It considers the benefit of each gene regarding to:– obstacle avoidance– target closeness
=−=≤∆=∆=≤∆=∆
=]7,6,5[,max1
0,)0(])4,3,2[(,max
0,)0()1(,max
iifSS
AAielsegiandiifggi
AAielsediandiifddi
AAi
t
t
t
t
≠
== ∑
=
0,)(
0,0
1
TPiTPiAAi
TPiAi TPi
tt
TPiTPi = total of steps executed by attitude i = total of steps executed by attitude i
AAiAAi=action’s fitness at stept of attitude i=action’s fitness at stept of attitude i Increases as the distance
to the proximity-sensor increasesin the step
![Page 23: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/23.jpg)
23
Improving the Target Direction Model
Â2 - Â1 3π/4 π/4
Target at LEFT
Target Behind Target in FRONT
Target at RIGHT
-3π/4 - π/4
First Target Direction Model
4 possible target directions
0
π/4
π/2
3π/4
π/2
-3π/4 -π/4
-π/2
8 possible target directions
![Page 24: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/24.jpg)
24
Chromosome Representations
Cada estado corresponde a uma única atitude (par de velocidadesM1 e M2), e cada atitude corresponde a um gene do cromossoma. Target Direction (collision free) Obstacle detected
M1, M2 M1, M2 M1, M2 M1, M2 M1, M2 M1, M2 M1, M2 Front Left Right Back Left Right Back
Chromosome with 7 attitude Genes
Target Direction (collision free) Obstacle detected
M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 M1,M2 Front Front Left Left Right Right Back Back Left Right Back left right front back front back left right
Chromosome with 11 attitude Genes
![Page 25: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/25.jpg)
25
Genetic Algorithm
• Integer chromosome• Population Size =100• Generations = 50• Crossover Rate = 80 %
• Mutation Rate = 4%• Roulette Wheel Reproduction• Elitism• Linear scaling of fitness• 300 Evaluation Steps for each chromosome
• Average of 25 Experiments
![Page 26: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/26.jpg)
26
Genetic Algorithm Performance7 Genes Chromosome
Best Chromosomesin 1 experiment
0
0,5
1
1 ,5
2
2,5
3
3,5
4
4,5
Number of Generations
FIT
NES
S
![Page 27: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/27.jpg)
27
Genetic Algorithm Performance7 Genes Chromosome
Average of Best Chromosomes in 25 experiments
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
1 3 5 7 9
11 13 15 17
19
21
23 25 27 29
31
33
35 37 39
41
43
45 47 49
Number of Generations
FIT
NE
SS
![Page 28: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/28.jpg)
28
Genetic Algorithm Performance11 Genes Chromosome
Best Chromosomesin 1 experiment
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
1 3 5 7 9
11
13
15
17
19
21
23
25
27
29
31
33
35
37
39
41
43
45
47
49
Number of Generations
FIT
NE
SS
![Page 29: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/29.jpg)
29
Genetic Algorithm Performance 11 Genes Chromosome
Average of Best Chromosomesin 25 experiments
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
5
1 3 5 7 9
11
13
15
17
19
21
23
25
27
29
31
33
35
37
39
41
43
45
47
49
Number of Generations
FIT
NE
SS
![Page 30: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/30.jpg)
30
Paths Achieved in World 1Case Study 1SITUAÇÃO 1
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
![Page 31: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/31.jpg)
31
Paths Achieved in World 1Case Study 2SITUAÇÃO 4
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
![Page 32: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/32.jpg)
32
Speed Comparison
0
1000
2000
3000
4000
5000
6000
1 2 3 4 5
Case Studies
Nu
mb
er
of
Ste
ps
11 Genes Chromosome
7 Genes Chromosome
![Page 33: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/33.jpg)
33
Paths Achieved in World 2Case Study 1SITUAÇÃO 1
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
![Page 34: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/34.jpg)
34
Paths Achieved in World 2Case Study 2SITUAÇÃO 2
CROMOSSOMA DE 7 GENES CROMOSSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
![Page 35: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/35.jpg)
350
500
1000
1500
2000
2500
3000
3500
4000
4500
1 2 3
Case Studies
Nu
mb
er
of
Ste
ps
Speed Comparison
11 Genes Chromosome
7 Genes Chromosome
![Page 36: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/36.jpg)
36
84,62%
40,00%
214,29%
Speed Comparison (%)
Case Study 1Case Study 1
Case Study 2Case Study 2
Case Study 3Case Study 3
![Page 37: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/37.jpg)
37
Paths Achieved in World 3Case Study 1SITUAÇÃO 1
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
![Page 38: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/38.jpg)
38
Paths Achieved in World 3Case Study 2SITUAÇÃO 2
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
![Page 39: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/39.jpg)
39
Paths Achieved in World 3Case Study 3SITUAÇÃO 3
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
![Page 40: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/40.jpg)
40
Paths Achieved in World 3Case Study 4SITUAÇÃO 4
CROMOSSOMA DE 7 GENES CROMOSSOMA DE 11 GENES7 Genes Chromosome 11 Genes Chromosome
![Page 41: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/41.jpg)
41
Speed Comparison
Speed Comparison – World 37 Genes Cromosome 11 Genes Cromosome
Case Study 1 target not reached 729Case Study 2 1124 618Case Study 3 target not reached target not reachedCase Study 4 target not reached 1655
![Page 42: Algoritmos Genéticos aplicados em Machine Learning](https://reader033.vdocuments.site/reader033/viewer/2022060203/559eeed51a28ab43438b4711/html5/thumbnails/42.jpg)
42
Conclusions
• A simple GA was able to gradually evolve the robot control
• The robot achieved near optimal path towards the goal,
avoiding obstacles
• Retraining is not necessary when the environment changes
• Controller improved performance with the 11 genes model
• The robot has no memory about previous unsuccessful paths
and may get lost
• Other tasks can be included in the model (e.g. energy supply)
• Chromosome codification is limited for few robot’s situations