optimal control and dynamical systems2020/07/15 · optimal control and dynamical systems si yi...
TRANSCRIPT
![Page 1: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/1.jpg)
Optimal Control and Dynamical Systems
Si Yi (Cathy) MengJuly 15, 2020
UBC MLRG
![Page 2: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/2.jpg)
Introduction
![Page 3: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/3.jpg)
Introduction
Control theory is the study and practice of manipulating dynamical systems.
• Inseparable from data science - sensor measurements (data)• Characteristics of this data is different from a statistical learning setting.
1
![Page 4: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/4.jpg)
Example - PID temperature controller
Figure 1: https://bit.ly/2Zk2JKE
• A Proportional-Integral-Derivativecontroller is a feedback controlmechanism.
• A temperature controller takesmeasurements from a temperature sensor.
• Its output is connected to a controlelement such as a heater or a fan.
2
![Page 5: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/5.jpg)
Example - PID temperature controller
Figure 1: https://bit.ly/2Zk2JKE
• A Proportional-Integral-Derivativecontroller is a feedback controlmechanism.
• A temperature controller takesmeasurements from a temperature sensor.
• Its output is connected to a controlelement such as a heater or a fan.
2
![Page 7: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/7.jpg)
Types of control
passive active
no sensor
s
open-lo
op
sensor based
disturban
ce
feedfor
wardclosed-loop
feedback
• Passive control does not require input energy.
• Cheap, simple, reliable.• May not be sufficient.• Example: stop signs at traffic
intersections.
• Active control requires input energy.
• Further categorized based on whethersensors are used.
4
![Page 8: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/8.jpg)
Types of control
passive active
no sensor
s
open-lo
op
sensor based
disturban
ce
feedfor
wardclosed-loop
feedback
• Passive control does not require input energy.
• Cheap, simple, reliable.• May not be sufficient.• Example: stop signs at traffic
intersections.
• Active control requires input energy.
• Further categorized based on whethersensors are used.
4
![Page 9: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/9.jpg)
Types of control
passive active
no sensor
s
open-lo
op
sensor based
disturban
ce
feedfor
wardclosed-loop
feedback
• Open-loop control relies on a pre-programmedcontrol sequence.
• Example: traffic lights.
• Sensor-based control uses sensor measurementsto inform the control law.
4
![Page 10: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/10.jpg)
Types of control
passive active
no sensor
s
open-lo
op
sensor based
disturban
ce
feedfor
wardclosed-loop
feedback
• Open-loop control relies on a pre-programmedcontrol sequence.
• Example: traffic lights.• Sensor-based control uses sensor measurements
to inform the control law.
4
![Page 11: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/11.jpg)
Types of control
passive active
no sensor
s
open-lo
op
sensor based
disturban
ce
feedfor
wardclosed-loop
feedback
• Disturbance feedforward control measuresexternal disturbances to the system, then feedsthis into an open-loop control law.
• Example: Preemptive road closure near astadium before a concert.
• Closed-loop control measures the system directly,then feeds the sensor measurements back.
• Example: Sensors in the roadbed.
• This will be our main focus.
4
![Page 12: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/12.jpg)
Types of control
passive active
no sensor
s
open-lo
op
sensor based
disturban
ce
feedfor
wardclosed-loop
feedback
• Disturbance feedforward control measuresexternal disturbances to the system, then feedsthis into an open-loop control law.
• Example: Preemptive road closure near astadium before a concert.
• Closed-loop control measures the system directly,then feeds the sensor measurements back.
• Example: Sensors in the roadbed.
• This will be our main focus.
4
![Page 13: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/13.jpg)
Types of control
passive active
no sensor
s
open-lo
op
sensor based
disturban
ce
feedfor
wardclosed-loop
feedback
• Disturbance feedforward control measuresexternal disturbances to the system, then feedsthis into an open-loop control law.
• Example: Preemptive road closure near astadium before a concert.
• Closed-loop control measures the system directly,then feeds the sensor measurements back.
• Example: Sensors in the roadbed.• This will be our main focus.
4
![Page 14: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/14.jpg)
Outline
We will follow Chapter 8 in Brunton and Kutz [2019],
• Closed-loop feedback control (Section 8.1)• Stability and eigenvalues (Section 8.2)• Controllability (Section 8.3)• Reachability (Section 8.3)• Optimal full-state control: LQR (Section 8.4)
5
![Page 15: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/15.jpg)
Closed-loop feedback control
![Page 16: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/16.jpg)
Closed-loop feedback control
System
Controller
Sensorsy(t)
Actuatorsu(t)
Disturbances
w =[wT
d wTn wT
r
]T
CostJ(x,u,wr )
6
![Page 17: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/17.jpg)
Closed-loop feedback control
System
Controller
Sensorsy(t)
Actuatorsu(t)
Disturbances
w =[wT
d wTn wT
r
]T
CostJ(x,u,wr )
• y(t) sensor measurements
6
![Page 18: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/18.jpg)
Closed-loop feedback control
System
Controller
Sensorsy(t)
Actuatorsu(t)
Disturbances
w =[wT
d wTn wT
r
]T
CostJ(x,u,wr )
• y(t) sensor measurements• u(t) actuation signal
6
![Page 19: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/19.jpg)
Closed-loop feedback control
System
Controller
Sensorsy(t)
Actuatorsu(t)
Disturbances
w =[wT
d wTn wT
r
]T
CostJ(x,u,wr )
• wd disturbances to the system
6
![Page 20: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/20.jpg)
Closed-loop feedback control
System
Controller
Sensorsy(t)
Actuatorsu(t)
Disturbances
w =[wT
d wTn wT
r
]T
CostJ(x,u,wr )
• wd disturbances to the system• wn measurement noise
6
![Page 21: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/21.jpg)
Closed-loop feedback control
System
Controller
Sensorsy(t)
Actuatorsu(t)
Disturbances
w =[wT
d wTn wT
r
]T
CostJ(x,u,wr )
• wd disturbances to the system• wn measurement noise• wr reference trajectory
6
![Page 22: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/22.jpg)
Closed-loop feedback control
System
Controller
Sensorsy(t)
Actuatorsu(t)
Disturbances
w =[wT
d wTn wT
r
]T
CostJ(x,u,wr )
Together, this forms a dynamical system given by
x := ddt x = f(x,u,wd ), y = g(x,u,wn),
and the goal is to construct a control law
u = k(y,wr ) such that the cost J is minimized. 6
![Page 23: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/23.jpg)
Example: Inverted pendulum
7
![Page 24: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/24.jpg)
Benefits of feedback control
Compared to open-loop control, closed-loop feedback makes it possible to
• Stabilize an unstable system.
• Compensate for external disturbances.• Correct for unmodeled dynamics.
8
![Page 25: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/25.jpg)
Benefits of feedback control
Compared to open-loop control, closed-loop feedback makes it possible to
• Stabilize an unstable system.• Compensate for external disturbances.
• Correct for unmodeled dynamics.
8
![Page 26: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/26.jpg)
Benefits of feedback control
Compared to open-loop control, closed-loop feedback makes it possible to
• Stabilize an unstable system.• Compensate for external disturbances.• Correct for unmodeled dynamics.
8
![Page 27: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/27.jpg)
Stability and eigenvalues
![Page 28: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/28.jpg)
Linearization of nonlinear dynamics
Our nonlinear dynamical system is given by
x = f(x,u,wd ), y = g(x,u,wn),
and the goal is to construct a control law
u = k(y,wr ) such that the cost J(x,u,wr ) is minimized.
9
![Page 29: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/29.jpg)
Linearization of nonlinear dynamics
For simplicity, let’s ignore the external disturbances w, which gives
x = f(x,u), y = g(x,u).
Near a fixed point (x, u) where f(x, u) = 0, we can use a Taylor expansion to obtain thefollowing linearization
x = Ax + Bu, y = Cx + Du,
where A = ∇fx(x , u), B = ∇fu(x , u), C = ∇gx(x , u), and D = ∇gu(x , u).
10
![Page 30: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/30.jpg)
Linearization of nonlinear dynamics
For simplicity, let’s ignore the external disturbances w, which gives
x = f(x,u), y = g(x,u).
Near a fixed point (x, u) where f(x, u) = 0, we can use a Taylor expansion to obtain thefollowing linearization
x = Ax + Bu, y = Cx + Du,
where A = ∇fx(x , u), B = ∇fu(x , u), C = ∇gx(x , u), and D = ∇gu(x , u).
10
![Page 31: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/31.jpg)
Unforced linear system - without control
Linear system
x = Ax + Bu, y = Cx + Du
Now suppose
• In the absence of control: u = 0• and with measurements of the full state: y = x,
our dynamical system becomesx = Ax,
and the solution x(t) is given byx(t) = eAtx(0).
11
![Page 32: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/32.jpg)
Unforced linear system - without control
Linear system
x = Ax + Bu, y = Cx + Du
Now suppose
• In the absence of control: u = 0• and with measurements of the full state: y = x,
our dynamical system becomesx = Ax,
and the solution x(t) is given byx(t) = eAtx(0).
11
![Page 33: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/33.jpg)
Unforced linear system - without control
Linear system
x = Ax, y = x
and the solution x(t) is given byx(t) = eAtx(0),
where the matrix exponential is given by the infinite power series
eAt = I + At + 12!A2t2 + 1
3!A2t3 + · · · =∞∑
k=0
1k!Aktk .
• When A is diagonalizable, eAt can be computed by leveraging A’s eigendecomposition:• A = QΛQ−1 =⇒ eAt = QeΛtQ−1
• When A is not diagonalizable, write Λ in Jordan form and compute the matrix exponentialwith simple extensions.
12
![Page 34: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/34.jpg)
Unforced linear system - without control
Linear system
x = Ax, y = x
and the solution x(t) is given byx(t) = eAtx(0),
where the matrix exponential is given by the infinite power series
eAt = I + At + 12!A2t2 + 1
3!A2t3 + · · · =∞∑
k=0
1k!Aktk .
• When A is diagonalizable, eAt can be computed by leveraging A’s eigendecomposition:• A = QΛQ−1 =⇒ eAt = QeΛtQ−1
• When A is not diagonalizable, write Λ in Jordan form and compute the matrix exponentialwith simple extensions.
12
![Page 35: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/35.jpg)
Unforced linear system - without control
If we write the states as x = Qz, then
z = Q−1x= Q−1Ax= Q−1AQz= Λz.
Our dynamical system simplifies from x = Ax to z = Λz, with solution
13
![Page 36: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/36.jpg)
Unforced linear system - without control
If we write the states as x = Qz, then
z = Q−1x= Q−1Ax= Q−1AQz= Λz.
Our dynamical system simplifies from x = Ax to z = Λz, with solution
x(t) = QeΛtQ−1x(0).
13
![Page 37: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/37.jpg)
Unforced linear system - without control
If we write the states as x = Qz, then
z = Q−1x= Q−1Ax= Q−1AQz= Λz.
Our dynamical system simplifies from x = Ax to z = Λz, with solution
x(t) = Q eΛt Q−1x(0)︸ ︷︷ ︸z(0)︸ ︷︷ ︸
z(t)
.
The eigenvalues in Λ also tell us about the stability of the system.
13
![Page 38: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/38.jpg)
Unforced linear system - without control
If we write the states as x = Qz, then
z = Q−1x= Q−1Ax= Q−1AQz= Λz.
Our dynamical system simplifies from x = Ax to z = Λz, with solution
x(t) = Q eΛt Q−1x(0)︸ ︷︷ ︸z(0)︸ ︷︷ ︸
z(t)
.
The eigenvalues in Λ also tell us about the stability of the system.
13
![Page 39: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/39.jpg)
Unforced linear system - stability
x(t) = QeΛtQ−1x(0).
• In general, the eigenvalues may be complex numbers: λ = a + ib.• Using Euler’s formula: eλt = eat(cos(bt) + i sin(bt)).
• Therefore, if all the eigenvalues λk have negative real part, i.e. a < 0, then thesystem is stable and x = 0 as t →∞.
• If for any λk we have a > 0, then the system will diverge in this direction, which is verylikely for a random initial condition.
14
![Page 40: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/40.jpg)
Unforced linear system - stability
x(t) = QeΛtQ−1x(0).
• In general, the eigenvalues may be complex numbers: λ = a + ib.• Using Euler’s formula: eλt = eat(cos(bt) + i sin(bt)).• Therefore, if all the eigenvalues λk have negative real part, i.e. a < 0, then the
system is stable and x = 0 as t →∞.
• If for any λk we have a > 0, then the system will diverge in this direction, which is verylikely for a random initial condition.
14
![Page 41: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/41.jpg)
Unforced linear system - stability
x(t) = QeΛtQ−1x(0).
• In general, the eigenvalues may be complex numbers: λ = a + ib.• Using Euler’s formula: eλt = eat(cos(bt) + i sin(bt)).• Therefore, if all the eigenvalues λk have negative real part, i.e. a < 0, then the
system is stable and x = 0 as t →∞.• If for any λk we have a > 0, then the system will diverge in this direction, which is very
likely for a random initial condition.
14
![Page 42: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/42.jpg)
Example: Stability of the inverted pendulum
From physics, we have θ = − gL sin(θ) + u.
Writing the system as a first-order differential equation,
x =[
x1
x2
]=[θ
θ
]=⇒ d
dt
[x1
x2
]=[
x2
− gL sin(x1) + u
].
Taking the Jacobian of x = f(x,u) yields
dfdx =
[0 1
− gL cos(x1) 0
],
dfdu =
[01
].
15
![Page 43: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/43.jpg)
Example: Stability of the inverted pendulum
From physics, we have θ = − gL sin(θ) + u.
Writing the system as a first-order differential equation,
x =[
x1
x2
]=[θ
θ
]=⇒ d
dt
[x1
x2
]=[
x2
− gL sin(x1) + u
].
Taking the Jacobian of x = f(x,u) yields
dfdx =
[0 1
− gL cos(x1) 0
],
dfdu =
[01
].
15
![Page 44: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/44.jpg)
Example: Stability of the inverted pendulum
From physics, we have θ = − gL sin(θ) + u.
Writing the system as a first-order differential equation,
x =[
x1
x2
]=[θ
θ
]=⇒ d
dt
[x1
x2
]=[
x2
− gL sin(x1) + u
].
Taking the Jacobian of x = f(x,u) yields
dfdx =
[0 1
− gL cos(x1) 0
],
dfdu =
[01
].
15
![Page 45: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/45.jpg)
Example: Stability of the inverted pendulum
From physics, we have θ = − gL sin(θ) + u.
Writing the system as a first-order differential equation,
x =[
x1
x2
]=[θ
θ
]=⇒ d
dt
[x1
x2
]=[
x2
− gL sin(x1) + u
].
Taking the Jacobian of x = f(x,u) yields
dfdx =
[0 1
− gL cos(x1) 0
],
dfdu =
[01
].
15
![Page 46: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/46.jpg)
Stability of the inverted pendulum
dfdx
=[
0 1− g
L cos(x1) 0
],
dfdu
=[
01
].
Linearizing at the pendulum up (x1 = π, x2 = 0) fixed point,
x =[
0 1gL 0
][x1x2
]+[
01
]u
and down (x1 = 0, x2 = 0) fixed point,
x =[
0 1− g
L 0
][x1x2
]+[
01
]u
• Pendulum up (“inverted”): λ = ±√
g/L, positive real part =⇒ instability.
• Pendulum down: λ = 0± i√
g/L, stable.• Good news: if we use closed-loop feedback control u = −Kx, we may be able to stabilize it!
16
![Page 47: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/47.jpg)
Stability of the inverted pendulum
dfdx
=[
0 1− g
L cos(x1) 0
],
dfdu
=[
01
].
Linearizing at the pendulum up (x1 = π, x2 = 0) fixed point,
x =[
0 1gL 0
][x1x2
]+[
01
]u
and down (x1 = 0, x2 = 0) fixed point,
x =[
0 1− g
L 0
][x1x2
]+[
01
]u
• Pendulum up (“inverted”): λ = ±√
g/L, positive real part =⇒ instability.
• Pendulum down: λ = 0± i√
g/L, stable.• Good news: if we use closed-loop feedback control u = −Kx, we may be able to stabilize it!
16
![Page 48: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/48.jpg)
Stability of the inverted pendulum
dfdx
=[
0 1− g
L cos(x1) 0
],
dfdu
=[
01
].
Linearizing at the pendulum up (x1 = π, x2 = 0) fixed point,
x =[
0 1gL 0
][x1x2
]+[
01
]u
and down (x1 = 0, x2 = 0) fixed point,
x =[
0 1− g
L 0
][x1x2
]+[
01
]u
• Pendulum up (“inverted”): λ = ±√
g/L, positive real part =⇒ instability.
• Pendulum down: λ = 0± i√
g/L, stable.
• Good news: if we use closed-loop feedback control u = −Kx, we may be able to stabilize it!
16
![Page 49: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/49.jpg)
Stability of the inverted pendulum
dfdx
=[
0 1− g
L cos(x1) 0
],
dfdu
=[
01
].
Linearizing at the pendulum up (x1 = π, x2 = 0) fixed point,
x =[
0 1gL 0
][x1x2
]+[
01
]u
and down (x1 = 0, x2 = 0) fixed point,
x =[
0 1− g
L 0
][x1x2
]+[
01
]u
• Pendulum up (“inverted”): λ = ±√
g/L, positive real part =⇒ instability.
• Pendulum down: λ = 0± i√
g/L, stable.• Good news: if we use closed-loop feedback control u = −Kx, we may be able to stabilize it!
16
![Page 50: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/50.jpg)
Controllability
![Page 51: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/51.jpg)
Controllability
Linear system
x = Ax + Bu, y = x
where x ∈ Rn, u ∈ Rq, A ∈ Rn×m, and B ∈ Rn×q.
Controllability:
• When can we use feedback control to manipulate the system into what we want?
• If we can control the system, how do we design the control law u = −Kx to drive thesystem to the desired behaviour?
With feedback control, we can write the dynamical system as
x = (A− BK)x
and hopefully we can use K such that we can place the eigenvalues wherever we want.
17
![Page 52: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/52.jpg)
Controllability
Linear system
x = Ax + Bu, y = x
where x ∈ Rn, u ∈ Rq, A ∈ Rn×m, and B ∈ Rn×q.
Controllability:
• When can we use feedback control to manipulate the system into what we want?• If we can control the system, how do we design the control law u = −Kx to drive the
system to the desired behaviour?
With feedback control, we can write the dynamical system as
x = (A− BK)x
and hopefully we can use K such that we can place the eigenvalues wherever we want.
17
![Page 53: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/53.jpg)
Controllability
Linear system
x = Ax + Bu, y = x
where x ∈ Rn, u ∈ Rq, A ∈ Rn×m, and B ∈ Rn×q.
Controllability:
• When can we use feedback control to manipulate the system into what we want?• If we can control the system, how do we design the control law u = −Kx to drive the
system to the desired behaviour?
With feedback control, we can write the dynamical system as
x = (A− BK)x
and hopefully we can use K such that we can place the eigenvalues wherever we want.17
![Page 54: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/54.jpg)
Controllability matrix
The controllability of a linear system in the form x = (A− BK)x is determined entirely by the column space ofthe controllability matrix:
Controllability matrix
C =[
B AB A2B . . . An−1B]
The following conditions are equivalent:
• Controllability:
• Columns of C span all of Rn.• Arbitrary eigenvalue placement:
• It’s possible to choose K such that the eigenvalues of (A− BK) can be wherever we want.• Reachability of Rn:
• It’s possible to steer the system to any arbitrary state x(t) = ξ ∈ Rn in finite time with someactuation signal u(t).
18
![Page 55: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/55.jpg)
Controllability matrix
The controllability of a linear system in the form x = (A− BK)x is determined entirely by the column space ofthe controllability matrix:
Controllability matrix
C =[
B AB A2B . . . An−1B]
The following conditions are equivalent:
• Controllability:
• Columns of C span all of Rn.
• Arbitrary eigenvalue placement:
• It’s possible to choose K such that the eigenvalues of (A− BK) can be wherever we want.• Reachability of Rn:
• It’s possible to steer the system to any arbitrary state x(t) = ξ ∈ Rn in finite time with someactuation signal u(t).
18
![Page 56: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/56.jpg)
Controllability matrix
The controllability of a linear system in the form x = (A− BK)x is determined entirely by the column space ofthe controllability matrix:
Controllability matrix
C =[
B AB A2B . . . An−1B]
The following conditions are equivalent:
• Controllability:
• Columns of C span all of Rn.• Arbitrary eigenvalue placement:
• It’s possible to choose K such that the eigenvalues of (A− BK) can be wherever we want.
• Reachability of Rn:
• It’s possible to steer the system to any arbitrary state x(t) = ξ ∈ Rn in finite time with someactuation signal u(t).
18
![Page 57: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/57.jpg)
Controllability matrix
The controllability of a linear system in the form x = (A− BK)x is determined entirely by the column space ofthe controllability matrix:
Controllability matrix
C =[
B AB A2B . . . An−1B]
The following conditions are equivalent:
• Controllability:
• Columns of C span all of Rn.• Arbitrary eigenvalue placement:
• It’s possible to choose K such that the eigenvalues of (A− BK) can be wherever we want.• Reachability of Rn:
• It’s possible to steer the system to any arbitrary state x(t) = ξ ∈ Rn in finite time with someactuation signal u(t).
18
![Page 58: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/58.jpg)
Controllability - Example I
Consider the following system:
x =[
1 00 2
][x1
x2
]+[
01
]u
Is this system controllable?
No. The eigenvalues are real and greater than 0, the states x1 and x2 are completely decoupledbut u only affects x2.We can also check the controllability matrix, which is in this case
C =[
0 01 2
]
and the two columns are linearly dependent.
19
![Page 59: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/59.jpg)
Controllability - Example I
Consider the following system:
x =[
1 00 2
][x1
x2
]+[
01
]u
Is this system controllable?No. The eigenvalues are real and greater than 0, the states x1 and x2 are completely decoupledbut u only affects x2.
We can also check the controllability matrix, which is in this case
C =[
0 01 2
]
and the two columns are linearly dependent.
19
![Page 60: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/60.jpg)
Controllability - Example I
Consider the following system:
x =[
1 00 2
][x1
x2
]+[
01
]u
Is this system controllable?No. The eigenvalues are real and greater than 0, the states x1 and x2 are completely decoupledbut u only affects x2.We can also check the controllability matrix, which is in this case
C =[
0 01 2
]
and the two columns are linearly dependent.
19
![Page 61: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/61.jpg)
Controllability - Example II
What about allowing two knobs? Consider the following system:
x =[
1 00 2
][x1
x2
]+[
1 00 1
][u1
u2
]
Is this system controllable?
Yes. Both states can be independently controlled by u1 and u2.The controllability matrix is
C =[
1 0 1 00 1 0 2
]which spans all of R2.
20
![Page 62: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/62.jpg)
Controllability - Example II
What about allowing two knobs? Consider the following system:
x =[
1 00 2
][x1
x2
]+[
1 00 1
][u1
u2
]
Is this system controllable?Yes. Both states can be independently controlled by u1 and u2.
The controllability matrix is
C =[
1 0 1 00 1 0 2
]which spans all of R2.
20
![Page 63: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/63.jpg)
Controllability - Example II
What about allowing two knobs? Consider the following system:
x =[
1 00 2
][x1
x2
]+[
1 00 1
][u1
u2
]
Is this system controllable?Yes. Both states can be independently controlled by u1 and u2.The controllability matrix is
C =[
1 0 1 00 1 0 2
]which spans all of R2.
20
![Page 64: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/64.jpg)
Controllability - Example III
What about when the states are coupled? Consider the following system:
x =[
1 10 2
][x1
x2
]+[
01
]u
Is this system controllable?
Maybe not obvious, but Yes. Even though we only have a single actuation, we can actuallycontrol x1 through controlling x2 since the states are coupled.In this case, the controllability matrix is
C =[
0 11 2
]
which again spans all of R2.
21
![Page 65: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/65.jpg)
Controllability - Example III
What about when the states are coupled? Consider the following system:
x =[
1 10 2
][x1
x2
]+[
01
]u
Is this system controllable?Maybe not obvious, but Yes. Even though we only have a single actuation, we can actuallycontrol x1 through controlling x2 since the states are coupled.
In this case, the controllability matrix is
C =[
0 11 2
]
which again spans all of R2.
21
![Page 66: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/66.jpg)
Controllability - Example III
What about when the states are coupled? Consider the following system:
x =[
1 10 2
][x1
x2
]+[
01
]u
Is this system controllable?Maybe not obvious, but Yes. Even though we only have a single actuation, we can actuallycontrol x1 through controlling x2 since the states are coupled.In this case, the controllability matrix is
C =[
0 11 2
]
which again spans all of R2.
21
![Page 67: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/67.jpg)
The PBH test for controllability
The Popov-Belevitch-Hautus test
The system x = Ax + Bu is controllable if and only if the column rank of[(A− λI) B
]is
equal to n for all λ ∈ C.
• If λ is not an eigenvalue of A, then rank(A− λI) = n is guaranteed,.• Only need to test for the eigenvalues of A!
• If λ is an eigenvalue of A, then N (A− λI) is the span of the eigenvector.• To make up for this rank deficiency, columns of B must have components in the eigenvector
direction corresponding to λ.
• If A has n distinct eigenvalues, then B only needs to account for one direction per eigenvalue.• Take B to be the sum of all n linearly-independent eigenvectors, and we only need a single
actuation to control ths system!• Or just take a random vector...
22
![Page 68: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/68.jpg)
The PBH test for controllability
The Popov-Belevitch-Hautus test
The system x = Ax + Bu is controllable if and only if the column rank of[(A− λI) B
]is
equal to n for all λ ∈ C.
• If λ is not an eigenvalue of A, then rank(A− λI) = n is guaranteed,.
• Only need to test for the eigenvalues of A!
• If λ is an eigenvalue of A, then N (A− λI) is the span of the eigenvector.• To make up for this rank deficiency, columns of B must have components in the eigenvector
direction corresponding to λ.
• If A has n distinct eigenvalues, then B only needs to account for one direction per eigenvalue.• Take B to be the sum of all n linearly-independent eigenvectors, and we only need a single
actuation to control ths system!• Or just take a random vector...
22
![Page 69: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/69.jpg)
The PBH test for controllability
The Popov-Belevitch-Hautus test
The system x = Ax + Bu is controllable if and only if the column rank of[(A− λI) B
]is
equal to n for all λ ∈ C.
• If λ is not an eigenvalue of A, then rank(A− λI) = n is guaranteed,.• Only need to test for the eigenvalues of A!
• If λ is an eigenvalue of A, then N (A− λI) is the span of the eigenvector.• To make up for this rank deficiency, columns of B must have components in the eigenvector
direction corresponding to λ.
• If A has n distinct eigenvalues, then B only needs to account for one direction per eigenvalue.• Take B to be the sum of all n linearly-independent eigenvectors, and we only need a single
actuation to control ths system!• Or just take a random vector...
22
![Page 70: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/70.jpg)
The PBH test for controllability
The Popov-Belevitch-Hautus test
The system x = Ax + Bu is controllable if and only if the column rank of[(A− λI) B
]is
equal to n for all λ ∈ C.
• If λ is not an eigenvalue of A, then rank(A− λI) = n is guaranteed,.• Only need to test for the eigenvalues of A!
• If λ is an eigenvalue of A, then N (A− λI) is the span of the eigenvector.
• To make up for this rank deficiency, columns of B must have components in the eigenvectordirection corresponding to λ.
• If A has n distinct eigenvalues, then B only needs to account for one direction per eigenvalue.• Take B to be the sum of all n linearly-independent eigenvectors, and we only need a single
actuation to control ths system!• Or just take a random vector...
22
![Page 71: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/71.jpg)
The PBH test for controllability
The Popov-Belevitch-Hautus test
The system x = Ax + Bu is controllable if and only if the column rank of[(A− λI) B
]is
equal to n for all λ ∈ C.
• If λ is not an eigenvalue of A, then rank(A− λI) = n is guaranteed,.• Only need to test for the eigenvalues of A!
• If λ is an eigenvalue of A, then N (A− λI) is the span of the eigenvector.• To make up for this rank deficiency, columns of B must have components in the eigenvector
direction corresponding to λ.
• If A has n distinct eigenvalues, then B only needs to account for one direction per eigenvalue.• Take B to be the sum of all n linearly-independent eigenvectors, and we only need a single
actuation to control ths system!• Or just take a random vector...
22
![Page 72: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/72.jpg)
The PBH test for controllability
The Popov-Belevitch-Hautus test
The system x = Ax + Bu is controllable if and only if the column rank of[(A− λI) B
]is
equal to n for all λ ∈ C.
• If λ is not an eigenvalue of A, then rank(A− λI) = n is guaranteed,.• Only need to test for the eigenvalues of A!
• If λ is an eigenvalue of A, then N (A− λI) is the span of the eigenvector.• To make up for this rank deficiency, columns of B must have components in the eigenvector
direction corresponding to λ.
• If A has n distinct eigenvalues, then B only needs to account for one direction per eigenvalue.• Take B to be the sum of all n linearly-independent eigenvectors, and we only need a single
actuation to control ths system!
• Or just take a random vector...
22
![Page 73: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/73.jpg)
The PBH test for controllability
The Popov-Belevitch-Hautus test
The system x = Ax + Bu is controllable if and only if the column rank of[(A− λI) B
]is
equal to n for all λ ∈ C.
• If λ is not an eigenvalue of A, then rank(A− λI) = n is guaranteed,.• Only need to test for the eigenvalues of A!
• If λ is an eigenvalue of A, then N (A− λI) is the span of the eigenvector.• To make up for this rank deficiency, columns of B must have components in the eigenvector
direction corresponding to λ.
• If A has n distinct eigenvalues, then B only needs to account for one direction per eigenvalue.• Take B to be the sum of all n linearly-independent eigenvectors, and we only need a single
actuation to control ths system!• Or just take a random vector...
22
![Page 74: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/74.jpg)
The Gramian - degrees of controllability
• The rank tests only give yes or no answers.• But some states can be easier to control than others.
The controllability Gramian
W(t) =∫ t
0eAτ BBT eAT τ dτ ∈ Rn×n,
which is often evaluated at infinite time,
W = limt→∞
W(t).
• The controllability of a state is measured by xT Wx, the larger the more controllable.• The eigendecomposition of W also tells us how much we can steer the system in the
direction of the eigenvectors.
23
![Page 75: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/75.jpg)
The Gramian - degrees of controllability
• The rank tests only give yes or no answers.• But some states can be easier to control than others.
The controllability Gramian
W(t) =∫ t
0eAτ BBT eAT τ dτ ∈ Rn×n,
which is often evaluated at infinite time,
W = limt→∞
W(t).
• The controllability of a state is measured by xT Wx, the larger the more controllable.• The eigendecomposition of W also tells us how much we can steer the system in the
direction of the eigenvectors.
23
![Page 76: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/76.jpg)
The Gramian - degrees of controllability
• The rank tests only give yes or no answers.• But some states can be easier to control than others.
The controllability Gramian
W(t) =∫ t
0eAτ BBT eAT τ dτ ∈ Rn×n,
which is often evaluated at infinite time,
W = limt→∞
W(t).
• The controllability of a state is measured by xT Wx, the larger the more controllable.
• The eigendecomposition of W also tells us how much we can steer the system in thedirection of the eigenvectors.
23
![Page 77: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/77.jpg)
The Gramian - degrees of controllability
• The rank tests only give yes or no answers.• But some states can be easier to control than others.
The controllability Gramian
W(t) =∫ t
0eAτ BBT eAT τ dτ ∈ Rn×n,
which is often evaluated at infinite time,
W = limt→∞
W(t).
• The controllability of a state is measured by xT Wx, the larger the more controllable.• The eigendecomposition of W also tells us how much we can steer the system in the
direction of the eigenvectors.
23
![Page 78: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/78.jpg)
Reachability
![Page 79: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/79.jpg)
The Cayley-Hamilton theorem and reachability
Reachability: it’s possible to steer the system to any arbitrary state x(t) = ξ ∈ Rn in finite time with someactuation signal u(t).
The Cayley-Hamilton theoremEvery square matrix A satisfies its own characteristic equation:
det(A− λI) = λn + an−1λn−1 + · · ·+ a2λ
2 + a1λ+ a0 = 0
=⇒ An + an−1An−1 + · · ·+ a2A2 + a1A + a0I = 0.
This allows us to express An as a linear combination of the lower-order powers:
An = −an−1An−1 − · · · − a2A2 − a1A− a0I.
More importantly, we can do this for any power greater than n:
Ak≥n =n−1∑j=0
αj Aj .
24
![Page 80: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/80.jpg)
The Cayley-Hamilton theorem and reachability
Reachability: it’s possible to steer the system to any arbitrary state x(t) = ξ ∈ Rn in finite time with someactuation signal u(t).
The Cayley-Hamilton theoremEvery square matrix A satisfies its own characteristic equation:
det(A− λI) = λn + an−1λn−1 + · · ·+ a2λ
2 + a1λ+ a0 = 0
=⇒ An + an−1An−1 + · · ·+ a2A2 + a1A + a0I = 0.
This allows us to express An as a linear combination of the lower-order powers:
An = −an−1An−1 − · · · − a2A2 − a1A− a0I.
More importantly, we can do this for any power greater than n:
Ak≥n =n−1∑j=0
αj Aj .
24
![Page 81: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/81.jpg)
The Cayley-Hamilton theorem and reachability
Reachability: it’s possible to steer the system to any arbitrary state x(t) = ξ ∈ Rn in finite time with someactuation signal u(t).
The Cayley-Hamilton theoremEvery square matrix A satisfies its own characteristic equation:
det(A− λI) = λn + an−1λn−1 + · · ·+ a2λ
2 + a1λ+ a0 = 0
=⇒ An + an−1An−1 + · · ·+ a2A2 + a1A + a0I = 0.
This allows us to express An as a linear combination of the lower-order powers:
An = −an−1An−1 − · · · − a2A2 − a1A− a0I.
More importantly, we can do this for any power greater than n:
Ak≥n =n−1∑j=0
αj Aj .
24
![Page 82: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/82.jpg)
The Cayley-Hamilton theorem and reachability
Reachability: it’s possible to steer the system to any arbitrary state x(t) = ξ ∈ Rn in finite time with someactuation signal u(t).
The Cayley-Hamilton theoremEvery square matrix A satisfies its own characteristic equation:
det(A− λI) = λn + an−1λn−1 + · · ·+ a2λ
2 + a1λ+ a0 = 0
=⇒ An + an−1An−1 + · · ·+ a2A2 + a1A + a0I = 0.
This allows us to express An as a linear combination of the lower-order powers:
An = −an−1An−1 − · · · − a2A2 − a1A− a0I.
More importantly, we can do this for any power greater than n:
Ak≥n =n−1∑j=0
αj Aj .
24
![Page 83: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/83.jpg)
The Cayley-Hamilton theorem and reachability
The Cayley-Hamilton theorem allows us to express the infinite power series eAt as a finite sum:
eAt = I + At +12!
A2t2 +13!
A2t3 + . . .
= α0(t)I + α1(t)A + α2(t)A2 + · · ·+ αn−1(t)An−1.
What does this have to do with reachability?
With control and zero initial condition x(0) = 0, the solution to the system x = Ax + Bu is
x(t) =∫ t
0eA(t−τ)Bu(τ)dτ.
So a state ξ ∈ Rn being reachable just means there exists u(t) such that
ξ =∫ t
0eA(t−τ)Bu(τ)dτ.
25
![Page 84: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/84.jpg)
The Cayley-Hamilton theorem and reachability
The Cayley-Hamilton theorem allows us to express the infinite power series eAt as a finite sum:
eAt = I + At +12!
A2t2 +13!
A2t3 + . . .
= α0(t)I + α1(t)A + α2(t)A2 + · · ·+ αn−1(t)An−1.
What does this have to do with reachability?With control and zero initial condition x(0) = 0, the solution to the system x = Ax + Bu is
x(t) =∫ t
0eA(t−τ)Bu(τ)dτ.
So a state ξ ∈ Rn being reachable just means there exists u(t) such that
ξ =∫ t
0eA(t−τ)Bu(τ)dτ.
25
![Page 85: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/85.jpg)
The Cayley-Hamilton theorem and reachability
The Cayley-Hamilton theorem allows us to express the infinite power series eAt as a finite sum:
eAt = I + At +12!
A2t2 +13!
A2t3 + . . .
= α0(t)I + α1(t)A + α2(t)A2 + · · ·+ αn−1(t)An−1.
What does this have to do with reachability?With control and zero initial condition x(0) = 0, the solution to the system x = Ax + Bu is
x(t) =∫ t
0eA(t−τ)Bu(τ)dτ.
So a state ξ ∈ Rn being reachable just means there exists u(t) such that
ξ =∫ t
0eA(t−τ)Bu(τ)dτ.
25
![Page 86: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/86.jpg)
The Cayley-Hamilton theorem and reachability
A state ξ ∈ Rn is reachable if there exists u(t) such that
ξ =∫ t
0eA(t−τ)Bu(τ)dτ
=∫ t
0[α0(t − τ)I + α1(t − τ)A + α2(t − τ)A2 + · · ·+ αn−1(t − τ)An−1]Bu(τ)dτ
= B∫ t
0α0(t − τ)u(τ)dτ + AB
∫ t
0α1(t − τ)u(τ)dτ + · · ·+ An−1B
∫ t
0αn−1(t − τ)u(τ)dτ
=[
B AB . . . An−1B]∫ t
0 α0(t − τ)u(τ)dτ∫ t0 α1(t − τ)u(τ)dτ
...∫ t0 αn−1(t − τ)u(τ)dτ
.
26
![Page 87: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/87.jpg)
The Cayley-Hamilton theorem and reachability
A state ξ ∈ Rn is reachable if there exists u(t) such that
ξ =∫ t
0eA(t−τ)Bu(τ)dτ
=∫ t
0[α0(t − τ)I + α1(t − τ)A + α2(t − τ)A2 + · · ·+ αn−1(t − τ)An−1]Bu(τ)dτ
= B∫ t
0α0(t − τ)u(τ)dτ + AB
∫ t
0α1(t − τ)u(τ)dτ + · · ·+ An−1B
∫ t
0αn−1(t − τ)u(τ)dτ
=[
B AB . . . An−1B]∫ t
0 α0(t − τ)u(τ)dτ∫ t0 α1(t − τ)u(τ)dτ
...∫ t0 αn−1(t − τ)u(τ)dτ
.
26
![Page 88: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/88.jpg)
The Cayley-Hamilton theorem and reachability
A state ξ ∈ Rn is reachable if there exists u(t) such that
ξ =∫ t
0eA(t−τ)Bu(τ)dτ
=∫ t
0[α0(t − τ)I + α1(t − τ)A + α2(t − τ)A2 + · · ·+ αn−1(t − τ)An−1]Bu(τ)dτ
= B∫ t
0α0(t − τ)u(τ)dτ + AB
∫ t
0α1(t − τ)u(τ)dτ + · · ·+ An−1B
∫ t
0αn−1(t − τ)u(τ)dτ
=[
B AB . . . An−1B]∫ t
0 α0(t − τ)u(τ)dτ∫ t0 α1(t − τ)u(τ)dτ
...∫ t0 αn−1(t − τ)u(τ)dτ
.
26
![Page 89: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/89.jpg)
The Cayley-Hamilton theorem and reachability
A state ξ ∈ Rn is reachable if there exists u(t) such that
ξ =∫ t
0eA(t−τ)Bu(τ)dτ
=∫ t
0[α0(t − τ)I + α1(t − τ)A + α2(t − τ)A2 + · · ·+ αn−1(t − τ)An−1]Bu(τ)dτ
= B∫ t
0α0(t − τ)u(τ)dτ + AB
∫ t
0α1(t − τ)u(τ)dτ + · · ·+ An−1B
∫ t
0αn−1(t − τ)u(τ)dτ
=[
B AB . . . An−1B]∫ t
0 α0(t − τ)u(τ)dτ∫ t0 α1(t − τ)u(τ)dτ
...∫ t0 αn−1(t − τ)u(τ)dτ
.
26
![Page 90: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/90.jpg)
The Cayley-Hamilton theorem and reachability
A state ξ ∈ Rn is reachable if there exists u(t) such that
ξ =[B AB . . . An−1B
]∫ t
0 α0(t − τ)u(τ)dτ∫ t0 α1(t − τ)u(τ)dτ
...∫ t0 αn−1(t − τ)u(τ)dτ
.
• Therefore, the only way for all of Rn to be reachable is when the columns of C spans Rn.
• If C has rank n, then we can design u(t) to reach any state ξ ∈ Rn.
27
![Page 91: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/91.jpg)
The Cayley-Hamilton theorem and reachability
A state ξ ∈ Rn is reachable if there exists u(t) such that
ξ =[B AB . . . An−1B
]︸ ︷︷ ︸
Controllability matrix C
∫ t
0 α0(t − τ)u(τ)dτ∫ t0 α1(t − τ)u(τ)dτ
...∫ t0 αn−1(t − τ)u(τ)dτ
.
• Therefore, the only way for all of Rn to be reachable is when the columns of C spans Rn.• If C has rank n, then we can design u(t) to reach any state ξ ∈ Rn.
27
![Page 92: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/92.jpg)
The Cayley-Hamilton theorem and reachability
A state ξ ∈ Rn is reachable if there exists u(t) such that
ξ =[B AB . . . An−1B
]︸ ︷︷ ︸
Controllability matrix C
∫ t
0 α0(t − τ)u(τ)dτ∫ t0 α1(t − τ)u(τ)dτ
...∫ t0 αn−1(t − τ)u(τ)dτ
.
• Therefore, the only way for all of Rn to be reachable is when the columns of C spans Rn.• If C has rank n, then we can design u(t) to reach any state ξ ∈ Rn.
27
![Page 93: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/93.jpg)
Optimal full-state control: LQR
![Page 94: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/94.jpg)
Optimal control
System
Controller
Sensorsy(t)
Actuatorsu(t)
Disturbancesw
CostJ(x,u)
• Recall that if the system x = Ax + Bu is controllable, then it’s possible to arbitrarily manipulate theeigenvalues through a full-state feedback control law u = −Kx.
• If we choose u to make the system arbitrarily stable, this can lead to• Expensive control expenditure J(x, u).• Over-react to noise and disturbances.
28
![Page 95: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/95.jpg)
Optimal control
System
Controller
Sensorsy(t)
Actuatorsu(t)
Disturbancesw
CostJ(x,u)
• Recall that if the system x = Ax + Bu is controllable, then it’s possible to arbitrarily manipulate theeigenvalues through a full-state feedback control law u = −Kx.
• If we choose u to make the system arbitrarily stable, this can lead to• Expensive control expenditure J(x, u).• Over-react to noise and disturbances.
28
![Page 96: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/96.jpg)
Optimal control: LQR
• Optimal control: choosing the best gain matrix K to stabilize the system with minimum effort.
• Seek balance between stability and aggressiveness of control.
Consider the cost function
J(t) =∫ t
0x(τ)T Qx(τ)︸ ︷︷ ︸
cost of deviations of x
+ u(τ)T Ru(τ)︸ ︷︷ ︸cost of control
dτ
• Q � 0 - can achieve zero deviation.
• R � 0 - but control effort is always needed.
• Often diagonal, tuned to weigh the relative importance of the states/control knobs.
• We now have an optimization problem!!!!!
29
![Page 97: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/97.jpg)
Optimal control: LQR
• Optimal control: choosing the best gain matrix K to stabilize the system with minimum effort.
• Seek balance between stability and aggressiveness of control.
Consider the cost function
J(t) =∫ t
0x(τ)T Qx(τ)︸ ︷︷ ︸
cost of deviations of x
+ u(τ)T Ru(τ)︸ ︷︷ ︸cost of control
dτ
• Q � 0 - can achieve zero deviation.
• R � 0 - but control effort is always needed.
• Often diagonal, tuned to weigh the relative importance of the states/control knobs.
• We now have an optimization problem!!!!!
29
![Page 98: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/98.jpg)
Optimal control: LQR
• Optimal control: choosing the best gain matrix K to stabilize the system with minimum effort.
• Seek balance between stability and aggressiveness of control.
Consider the cost function
J(t) =∫ t
0x(τ)T Qx(τ)︸ ︷︷ ︸
cost of deviations of x
+ u(τ)T Ru(τ)︸ ︷︷ ︸cost of control
dτ
• Q � 0 - can achieve zero deviation.
• R � 0 - but control effort is always needed.
• Often diagonal, tuned to weigh the relative importance of the states/control knobs.
• We now have an optimization problem!!!!!
29
![Page 99: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/99.jpg)
Optimal control: LQR
• Optimal control: choosing the best gain matrix K to stabilize the system with minimum effort.
• Seek balance between stability and aggressiveness of control.
Consider the cost function
J(t) =∫ t
0x(τ)T Qx(τ)︸ ︷︷ ︸
cost of deviations of x
+ u(τ)T Ru(τ)︸ ︷︷ ︸cost of control
dτ
• Q � 0 - can achieve zero deviation.
• R � 0 - but control effort is always needed.
• Often diagonal, tuned to weigh the relative importance of the states/control knobs.
• We now have an optimization problem!!!!!
29
![Page 100: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/100.jpg)
Optimal control: LQR
• Optimal control: choosing the best gain matrix K to stabilize the system with minimum effort.
• Seek balance between stability and aggressiveness of control.
Consider the cost function
J(t) =∫ t
0x(τ)T Qx(τ)︸ ︷︷ ︸
cost of deviations of x
+ u(τ)T Ru(τ)︸ ︷︷ ︸cost of control
dτ
• Q � 0 - can achieve zero deviation.
• R � 0 - but control effort is always needed.
• Often diagonal, tuned to weigh the relative importance of the states/control knobs.
• We now have an optimization problem!!!!!
29
![Page 101: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/101.jpg)
Optimal control: LQR
J(t) =∫ t
0x(τ)T Qx(τ)︸ ︷︷ ︸
cost of deviations of x
+ u(τ)T Ru(τ)︸ ︷︷ ︸cost of control
dτ
The linear-quadratic-regulator (LQR) control law u = −Kr x is designed to minimize J = limt→∞ J(t).
• Linear control law u = −Kr x
• Quadratic cost function J
• Regulates the state of the system to limt→inf x(t) = 0.
30
![Page 102: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/102.jpg)
Optimal control: LQR
J(t) =∫ t
0x(τ)T Qx(τ)︸ ︷︷ ︸
cost of deviations of x
+ u(τ)T Ru(τ)︸ ︷︷ ︸cost of control
dτ
The linear-quadratic-regulator (LQR) control law u = −Kr x is designed to minimize J = limt→∞ J(t).
• Linear control law u = −Kr x
• Quadratic cost function J
• Regulates the state of the system to limt→inf x(t) = 0.
30
![Page 103: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/103.jpg)
Optimal control: LQR
J(t) =∫ t
0x(τ)T Qx(τ)︸ ︷︷ ︸
cost of deviations of x
+ u(τ)T Ru(τ)︸ ︷︷ ︸cost of control
dτ
The linear-quadratic-regulator (LQR) control law u = −Kr x is designed to minimize J = limt→∞ J(t).
• Linear control law u = −Kr x
• Quadratic cost function J
• Regulates the state of the system to limt→inf x(t) = 0.
30
![Page 104: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/104.jpg)
Optimal control: LQR
J(t) =∫ t
0x(τ)T Qx(τ)︸ ︷︷ ︸
cost of deviations of x
+ u(τ)T Ru(τ)︸ ︷︷ ︸cost of control
dτ
The linear-quadratic-regulator (LQR) control law u = −Kr x is designed to minimize J = limt→∞ J(t).
• Linear control law u = −Kr x
• Quadratic cost function J
• Regulates the state of the system to limt→inf x(t) = 0.
30
![Page 105: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/105.jpg)
Optimal control: LQR
J(t) =∫ t
0x(τ)T Qx(τ)︸ ︷︷ ︸
cost of deviations of x
+ u(τ)T Ru(τ)︸ ︷︷ ︸cost of control
dτ
The linear-quadratic-regulator (LQR) control law u = −Kr x is designed to minimize J = limt→∞ J(t).
• Linear control law u = −Kr x
• Quadratic cost function J
• Regulates the state of the system to limt→inf x(t) = 0.
30
![Page 106: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/106.jpg)
Optimal control: LQR
Since J(t) is quadratic, there is an analytical solution given by
Kr = R−1BT X,
where X is the solution to an algebraic Riccati equation:
AT X + XA− XBR−1BT X + Q = 0.
• There exists numerically robust implementations to solve this.
• Very expensive for high-dimensional systems - O(n3).
• Reduced-order models: use fewer states.
31
![Page 107: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/107.jpg)
Optimal control: LQR
Since J(t) is quadratic, there is an analytical solution given by
Kr = R−1BT X,
where X is the solution to an algebraic Riccati equation:
AT X + XA− XBR−1BT X + Q = 0.
• There exists numerically robust implementations to solve this.
• Very expensive for high-dimensional systems - O(n3).
• Reduced-order models: use fewer states.
31
![Page 108: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/108.jpg)
Optimal control: LQR
Since J(t) is quadratic, there is an analytical solution given by
Kr = R−1BT X,
where X is the solution to an algebraic Riccati equation:
AT X + XA− XBR−1BT X + Q = 0.
• There exists numerically robust implementations to solve this.
• Very expensive for high-dimensional systems - O(n3).
• Reduced-order models: use fewer states.
31
![Page 109: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/109.jpg)
Summary
What we covered:
• Closed-loop feedback control.
• Stability and eigenvalues of a linear dynamical system.• Controllability and Reachability.• Optimal full-state control: LQR.
What we didn’t cover:
• How to derive the Riccati equations for LQR. (End of Section 8.4 in [Brunton and Kutz,2019])
• Full-state estimation and the Kalman filter. (Section 8.5 in [Brunton and Kutz, 2019])
32
![Page 110: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/110.jpg)
Summary
What we covered:
• Closed-loop feedback control.• Stability and eigenvalues of a linear dynamical system.
• Controllability and Reachability.• Optimal full-state control: LQR.
What we didn’t cover:
• How to derive the Riccati equations for LQR. (End of Section 8.4 in [Brunton and Kutz,2019])
• Full-state estimation and the Kalman filter. (Section 8.5 in [Brunton and Kutz, 2019])
32
![Page 111: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/111.jpg)
Summary
What we covered:
• Closed-loop feedback control.• Stability and eigenvalues of a linear dynamical system.• Controllability and Reachability.
• Optimal full-state control: LQR.
What we didn’t cover:
• How to derive the Riccati equations for LQR. (End of Section 8.4 in [Brunton and Kutz,2019])
• Full-state estimation and the Kalman filter. (Section 8.5 in [Brunton and Kutz, 2019])
32
![Page 112: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/112.jpg)
Summary
What we covered:
• Closed-loop feedback control.• Stability and eigenvalues of a linear dynamical system.• Controllability and Reachability.• Optimal full-state control: LQR.
What we didn’t cover:
• How to derive the Riccati equations for LQR. (End of Section 8.4 in [Brunton and Kutz,2019])
• Full-state estimation and the Kalman filter. (Section 8.5 in [Brunton and Kutz, 2019])
32
![Page 113: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/113.jpg)
Summary
What we covered:
• Closed-loop feedback control.• Stability and eigenvalues of a linear dynamical system.• Controllability and Reachability.• Optimal full-state control: LQR.
What we didn’t cover:
• How to derive the Riccati equations for LQR. (End of Section 8.4 in [Brunton and Kutz,2019])
• Full-state estimation and the Kalman filter. (Section 8.5 in [Brunton and Kutz, 2019])
32
![Page 114: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/114.jpg)
Thank you
33
![Page 115: Optimal Control and Dynamical Systems2020/07/15 · Optimal Control and Dynamical Systems Si Yi (Cathy) Meng July 15, 2020 UBC MLRG Introduction Introduction Control theory is the](https://reader035.vdocuments.site/reader035/viewer/2022071421/611a0c8b40610b3f6b5c45f8/html5/thumbnails/115.jpg)
References i
Steven L. Brunton. Control Bootcamp.https://www.youtube.com/playlist?list=PLMrJAkhIeNNR20Mz-VpzgfQs5zrYi085m, 2020.
Steven L. Brunton and J. Nathan Kutz. Data-Driven Science and Engineering: Machine Learning, DynamicalSystems, and Control. Cambridge University Press, 2019.
34