![Page 1: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/1.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 1
Lecture XIIIDynamical Systems as Movement Policies
Contents:• Differential Equation• Force Fields, Velocity Fields• Dynamical systems for Trajectory Plans
• Generating plans dynamically • Fitting (or modifying) plans• Imitation based learning
Thanks to my collaborator Auke Ijspeert (EPFL) for many of the contents on the slides for this lecture.
![Page 2: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/2.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 2
Movement policies as Dynamical Systems
• Represent complex movements in globally stableattractor landscapes of nonlinear autonomous differential equations
• Choose kinematic representation for easy re-use in different workspace location
• Ensure easy temporal and spatial scaling (topological equivalence)
• Use local learning to modify the attractors according to demonstration of teacher and self-learning
Discreet & Rhythmic Movement Primitives
( , , )des desf goalτ =x x x&
![Page 3: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/3.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 3
Discreet & Rhythmic Movement superposition
Open loop with oscillators Closed loop control in horizontal plane
Open (vertical) + Closed (horizontal) loop control
![Page 4: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/4.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 4
What is a Differential Equation?
• Differential equation: an equation that describes how state variables evolve over time, for instance:
)( ycy −= α&
![Page 5: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/5.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 5
Some Definitions
• Ordinary differential equation: differential equation thatinvolves only ordinary derivatives (as opposed to partial derivatives)
• Autonomous equation: differential equation that does not(explicitely) depend on time
• Linear differential equation: differential equation in which thestate variables only appear in linear combinations
• Nonlinear differential equation: differential equation in whichsome state variables appear in nonlinear combinations (e.g. products, cosine,…)
• Fixed point: point at which all derivatives are zero (can be an attractor, a repeller, or a saddle point, cf later)
• Limit cycle: periodic isolated closed trajectory (can only occur in nonlinear systems)
![Page 6: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/6.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 6
Interesting Regimes of Differential Equations
From Strogatz 1994
Limit cycles ChaosAttractors
Attractor
saddles
Unstable node
![Page 7: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/7.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 7
First Order Linear Systems
• First order linear system:
• How to solve this equation, for a given y(t=0), c, and α ?
• Two methods: analytical solution or numerical integration
• Analytical solution:
• Numerical integration: Euler method, Runge-Kutta,…
)( ycy −= α&
ctcyty +−−= )exp()()( 0 α
![Page 8: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/8.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 8
First Order Linear Systems
)( ycy −= α&
ctcyty +−−= )exp()()( 0 α
![Page 9: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/9.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 9
Second Order Linear Systems
yxyxcy
=−−=
&
& ))((βα
![Page 10: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/10.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 10
Second Order Linear Systems
yxyxcy
=−−=
&
& ))((βα
88
==
βα
18
==
βα
28
==
βα
![Page 11: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/11.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 11
Second Order Non-linear System
xyxyxy
coscos2coscos2
−−=−−=
&
&
Attractor
saddles
Repeller
![Page 12: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/12.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 12
Learning a movement by demonstration
Movement recognition
Movement execution
(Inv. Dyn.)
Trajectory formation
system
Joint angles
Task of the trajectory formation system: • To encode demonstrated trajectories with high accuracy, • To be able to modulate the learned trajectory when:
• Perceptual variables are varied (e.g. timing, amplitude)• Perturbations occur
![Page 13: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/13.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 13
Encoding a trajectory
Traditionally, the problem of replaying a trajectory has been decomposedinto two different issues:• One of encoding the trajectory, and• One of modifying the trajectory, for instance, in case the movement is
perturbed, or when it requires to be modulated.
Our approach: combine both abilities in a nonlinear dynamical system
Aim: to encode the trajectory in a nonlinear dynamical system with welldefined attractor landscape
![Page 14: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/14.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 14
Nonlinear Dynamical Systems Approach
y
dy/dt
Single point attractor
Discrete movements
y
dy/dt
Limit cycle attractor
Rhythmic movements
![Page 15: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/15.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 15
Two types of movement recordings
« Kinesthetic » demonstrationSensuit
![Page 16: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/16.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 16
Pointing Demonstration
position velocity
Demo forOne DOF
![Page 17: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/17.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 17
Shaping Attractor Landscapes
Goal: g
( )( )( )( )?
z zz g y zy f z
α β= − −
= +
&
&
( )( )z zz g y zy z
α β= − −=
&
&
Can one create more complex dynamics by non-linearly modifying the dynamic system:
![Page 18: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/18.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 18
Discreet Control Policy
Goal: g
Amplitude and phase system
Output System
![Page 19: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/19.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 19
Shaping attractor landscapes
( )( )z zz g y zy z
α β= − −=
&
&
y
dy/d
tg = goal Can one create more
complex dynamics by non-linearly modifying the above dynamic system:
( )( )( )( )?
z zz g y zy f z
α β= − −
= +
&
&
g = goal
![Page 20: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/20.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 20
Shaping Attractor Landscapes
A globally stable learnable nonlinear point attractor:
( )( )( )( )
( )( )
( )
( )
1
1
2 0
0
,where
,
1exp and 2
z z
y
v v
xk
i ii
k
ii
i i i
z g y zy f x v z
v g x vx v
w b vf x v
w
x xw d x c xg x
α βα
α βα
=
=
= − −
= +
= − −=
=
−⎛ ⎞= − − =⎜ ⎟ −⎝ ⎠
∑
∑
&
&
&
&
Trajectory PlanDynamics
Canonical Dynamics
Local LinearModel Approx.
![Page 21: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/21.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 21
Learning the Attractor
Given a demonstrated trajectory y(t)demo and a goal g
Extract movement durationAdjust time constants of canonical dynamics to movement durationUse LWL to learn supervised problem
( )target ,demo
y
yy z f x vα
= − =&
&
[ Stefan Schaal, Sethu Vijayakumar et al, Proc. of Intl. Symp. Rob. Res.(ISRR) (2001) ] Also extended to rhythmic primitives :
![Page 22: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/22.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 22
Trajectory following & Generalization
Backhand Demonstration Backhand Reproduction
![Page 23: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/23.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 23
Modulation of Goal: Anchor
![Page 24: Lecture XIII Dynamical Systems as Movement Policies€¦ · Lecture XIII Dynamical Systems as Movement Policies ... • Linear differential equation: differential equation in which](https://reader034.vdocuments.site/reader034/viewer/2022051602/5af1af957f8b9abc788ec317/html5/thumbnails/24.jpg)
Lecture XIII: MLSC - Dr. Sethu Vijayakumar 24
Drumming: Modulating Frequency
Drumming: Kinesthetic Demo
Imitation and Modulation