short notes on linear systems theory 1 introductionsanand/linearsystems.pdf · 2019-11-04 ·...
TRANSCRIPT
Short notes on Linear systems theory
by Sanand D“Sometimes the truth isn’t good enough, sometimes people need more...” -Batman
————————————————————————————————————————-
1 Introduction
The various types of systems are
1. Linear/Non-linear
2. Continuous/Discrete
3. Time invariant/Time varying
4. Deterministic/Stochastic
5. Lumped/Distributed
6. Finite/Infinite dimensional
7. Autonomous/Non-autonomous
8. Dissipative/Lossless.
In here, we study linear systems which one must understand before studying more general non-linear systems and complex systems occuring in many real life applications. Some examples oflinear systems are electrical circuits with linear elements (e.g., RLC circuits), spring-mass-dampersystems, linear filters and so on. One can study non-linear systems locally around an equilibriumpoint by local linearization. A system is said to be linear if the superposition and the homogeneityprinciples hold.
Study of systems mainly involve: modelling, analysis and synthesis and design. Modelling isdone using first principles i.e., using Euler-Lagrange equations, Kirchoff/Newton’s laws, conserva-tion laws (e.g., Maxwell’s equation, heat equation, wave equation) etc. We consider state-spacemodels which can be obtained from n−th order odes by introducing auxiliary variables. The numberof states is given by the number of independent energy storing elements in the system. Sometimes,linear systems can also be represented in the input-output form using a convolution operator e.g.,signal processing systems in communication engineering. The other form of modeling is known assystem identification where one tries to estimate the system equations by probing the system withknown input signals and observing the corresponding output. Modeling allows us to represent aphysical system using mathematical expression and allows us to determine the system propertiesusing mathematical concepts. It allows us to predict the system response to various input signals.
Analysis and synthesis involves studying intrinsic system properties such as controllability,observability, stabilizability, controllable/unobservable subspaces, controllability indices, poles andzeros etc. Designing control systems involve: control, estimation and optimization. We considercontinuous time systems of the form
x = Ax+Bu
y = Cx+Du (1)
and discrete time systems of the form
x(t+ 1) = Ax(t) +Bu(t)
y(t) = Cx(t) +Du(t) (2)
1
where x ∈ Rn, u ∈ Rm, y ∈ Rp, A ∈ Rn×n, B ∈ Rn×m, C ∈ Rp×n, D ∈ Rp×m. In other words, theseare the models for systems under consideration, if A,B,C,D are time dependent, then the systemis called linear time varying system (LTV) whereas, if they are time independent, then it is calledlinear time invariant (LTI) system. These are called non-autonomous systems i.e. systems withinputs. The simplest example of LTI systems is given by an integrator system where the outputis the integration of the input i.e., y =
´ t0 udt. Let x = y, then x = u. This is a scalar LTI
system where A = 0, B = 1, C = 1 and D = 0. One can also consider a double integrator wherey =´ t
0
´ s0 udsdt. Let x1 = y and x2 = x1, then x1 = x2 and x2 = u. (Find A,B,C,D.) Autonomous
systems are of the for x = Ax where there is no input (e.g., LC circuit with no current or voltagesources). The advantages of state space models is that both LTI and LTV can be studied. ForLTI systems and signal processing system, transfer function approach is also applicable (using theLaplace and Fourier transforms). State space models also allows us to study the internal dynamicsof a system.
In general, one can consider the following questions for linear systems
1. How to drive the state from an initial condition to the desired state, how to choose a controlinput?
2. What is controllability and observability? How to check if a system is controllable/observable?
3. What is stabilizability? How to check stabilizability and how to make a system stable?
4. What is the optimal control input for a state transfer, e.g., least energy input?
5. How to compute the energy required for a state transfer?
6. How to estimate states from the measured outputs?
7. What is state feedback/output feedback?
8. How to build a state/output feedback controller?
9. How to control a system in the presence of disturbance and noise?
10. How to study non-linear systems locally using linear models?
We need linear algebra (refer short notes on linear algebra) and linear odes to address the abovequestions.
2 Linear ODEs
Consider a linear ode p(D)y = u where p(D) is a polynomial involving differential operator D.Consider a homogeneous ode p(D)y = 0 and let yh be a solution of this homogeneous ode. Letyp be a particular solution of the ode p(D)yp = u (where yp depends on u). A general solutioncan be written as y = yh + yp. This is true for both time varying and time invariant odes as longas they are linear. This not only holds for scalar odes above but also holds for vector odes. Byusing additional variables, scalar odes can be converted to vector odes which are in a first orderform (i.e. only first order derivatives are involved). Roots of the scalar ode and eigenvalues of thecorresponding matrix in the first order form are the same. (This can be shown using ideas fromQuotient spaces from linear algebra and the proof is given in the linear algebra notes. Moreover, thematrix obtained in the first order form is a companion matrix. Thus, its characteristic polynomialis same as the characteristic equation for ode again from linear algebra.)
For linear time invariant odes, one can also apply Laplace transform methods to find a solution.One can write down a solution of linear odes explicitly. This is called as a closed form solution
2
which is the sum of the two terms as explained above. By the existence and uniqueness theoremfor odes, a unique solution always exists. The terms such as a solution of an ode, a trajectory, anintegral curve are all synonymous. We assume that the reader is familiar with definitions of linearode, impulse response, causality, LTI systems and transfer functions. Let u(t) = δ(t) be an inputto a system x = Ax + Bu. The homogeneous solution is xh = eAtx(0) and a particular solutionis xp(t) =
´ t0 e
A(t−τ)Bδ(τ)dτ = eAtB. Thus x(t) = eAtx(0) + eAtB. Suppose u(t) = u0ejωt. Then,
xp(t) = xmejωt. Substituting xp(t) in the system equation, xm = (jωI − A)−1Bu0 and x(t) =
eAtx(0) + (jωI − A)−1Bu0ejωt. The first term represents the transient response and the second
term represents the steady state response. Let y(t) = Cx(t) be the output of the system. Hence,the impulse response is CeAtB and the frequency response is C(jωI − A)−1Bu0e
jωt (Kwakernaakand Sivan).
As far as the existence and uniqueness of solutions of an ode is concerned we refer the readerto any standard textbook on ode. It turns out that for the linear odes under consideration, thereexists a unique solution which can be written in a closed form using the state transition matrix asshown in the next section.
3 State transition matrix
We show in this section that for linear ODEs, the state transition matrix determines the evolutionof a solution trajectory depending upon the initial conditions.
Definition 3.1. The state transition matrix for a linear homogeneous system (or autonomoussystem i.e. a system without any inputs) x = A(t)x is defined as
Φ(t, t0) := I +
ˆ t
t0
A(s1)ds1 +
ˆ t
t0
A(s1)
ˆ s1
t0
A(s2)ds2ds1 + . . . . (3)
The state transition matrix has the following properties:
1. ddtΦ(t, t0) = A(t)Φ(t, t0), Φ(t0, t0) = I.Proof: From the definition, and the fundamental theorem of calculus,
d
dtΦ(t, t0) = A(t) +A(t)
ˆ t
t0
A(s2)ds2 +A(t)
ˆ t
t0
A(s2)
ˆ s2
t0
A(s3)ds3ds2 + . . .
= A(t)Φ(t, t0)
2. x(t) = Φ(t, t0)x(t0) is a solution trajectory.Proof: x(t) = Φ(t, t0)x(t0) = A(t)Φ(t, t0)x(t0) = A(t)x(t).
3. Φ(t, s)Φ(s, τ) = Φ(t, τ).Proof: x(t) = Φ(t, s)x(s) and x(s) = Φ(s, τ)x(τ). Thus x(t) = Φ(t, s)Φ(s, τ)x(τ). Moreover,x(t) = Φ(t, τ)x(τ). Then by existence and uniqueness of solutions of ODEs, Φ(t, s)Φ(s, τ) =Φ(t, τ).
4. Φ(t, τ)−1 = Φ(τ, t).Proof: From the previous property, Φ(t, τ)Φ(τ, t) = Φ(t, t) = I. Therefore, Φ(t, τ)−1 =Φ(τ, t).
For LTI systems, A(t) = A and therefore from the definition of the state transition matrix (STM),Φ(t, t0) = eA(t−t0). Thus, by Cayley-Hamilton theorem, we can express eA(t−t0) as a linear combi-nation of I, A, . . . , An−1 with time dependent coefficients. Note that AeAt = eAtA.
3
For non-homogeneous linear ODEs,
x(t) := Φ(t, t0)x(t0) +
ˆ t
t0
Φ(t, τ)B(τ)u(τ)dτ. (4)
⇒ x(t) = A(t)Φ(t, t0)x(t0) +A(t)
ˆ t
t0
Φ(t, τ)B(τ)u(τ)dτ + Φ(t, t)B(t)u(t)
= A(t)x(t) +B(t)u(t).
Thus, the closed form expression for x(t) satisfies the ODE hence, it is a solution. Observe that a so-lution is a sum of a homogeneous response Φ(t, t0)x(t0) and a forced response
´ tt0
Φ(t, τ)B(τ)u(τ)dτ .
4 Change of basis and system representation
Let (A,B,C,D) be representation of a system w.r.t. a given basis for a state space.
x = Ax+Bu, y = Cx+Du
Consider a new basis for the state space formed by columns of matrix T . (Basis for the input spaceand the output space is kept the same.) Let x be the state vector w.r.t. the new basis. Thusx = T−1x (refer notes on Linear algebra).
˙x = Ax+ Bu, y = Cx+ Du
We need to find (A, B, C, D) in terms of (A,B,C,D).
˙x = T−1x = T−1(Ax+Bu) = T−1(ATx+Bu) = T−1ATx+ T−1Bu
y = Cx+Du = CTx+Du
Thus, A = T−1AT, B = T−1B, C = CT, D = D.
4.1 Transfer matrix and realization
The transfer matrix for the above model is given by G(s) = C(sI − A)−1B + D. When D = 0,then transfer matrix is strictly proper. Constructing (A,B,C,D) from G(s) is called realization.We now see how to construct a realization for a given transfer function. Write G(s) = Gsp(s) +Di.e., write G(s) as a sum of a constant matrix and a transfer matrix in strictly proper form, D :=lims→∞G(s). Note that Gsp(s) = C(sI − A)−1B where (sI − A)−1 = 1
det(sI−A)Adj.(sI − A). Let
d(s) = sn +α1sn−1 + . . .+αn be the monic least common denominator of all entries of Gsp(s). Let
Gsp(s) =1
d(s)[N1s
n−1 + . . .+Nn−1s+Nn]. (5)
where Gsp(s) ∈ Rp×k(s). Let
A =
−α1Ik×k −α2Ik×k · · · −αn−1Ik×k −αnIk×kIk×k 0 · · · 0 0. . · · · . .. . · · · . .0 0 · · · Ik×k 0
, B =
Ik×k
0..0
, C =[N1 N2 · · · Nn−1 Nn
](6)
We now show that this (A,B,C) realizes Gsp(s). Let Z(s) = (sI−A)−1B. Thus, (sI−A)Z(s) = Bi.e.,
s+ α1Ik×k α2Ik×k · · · αn−1Ik×k αnIk×k−Ik×k sIk×k · · · 0 0. . · · · . .. . · · · . .0 0 · · · −Ik×k sIk×k
Z1
Z2
.
.Zn
=
Ik×k
0..0
. (7)
4
Therefore, Zn = 1sZn−1, Zn−1 = 1
sZn−2, . . . , Z2 = 1sZ1 ⇒ Zk = 1
sk−1Z1. Hence,
(s+ α1 +α2
s+ . . .+
αn−1
sn−2+
αnsn−1
)Z1 = Ik×k (8)
i.e., d(s)sn−1Z1 = Ik×k ⇒ Z1 = sn−1
d(s) Ik×k. Thus,Z1
Z2
.
.Zn
=1
d(s)
sn−1Ik×ksn−2Ik×k
.
.Ik×k
.Now, CZ(s) = C(sI −A)−1B = Gsp(s). It can be verified that CZ(s) gives (5) giving the desiredtransfer matrix. This realization is called controller canonical realization of G(s).
Now suppose
A =
−α1Ik×k Ik×k 0 · · · 0−α2Ik×k 0 0 · · · 0
. . . · · · .
. . . · · · .−αn−1Ik×k 0 0 · · · Ik×k−αnIk×k 0 0 · · · 0
, B =
N1
N2
.
.Nn
, C =[Ik×k 0 · · · 0 0
]. (9)
One can verify that this also realizes Gsp(s). This is called observer canonical realization.
5 Local linearization and feedback linearization
5.1 Local linearization
Consider a nonlinear systemx = f(x, u), y = g(x, u). (10)
where there may or may not be an explicit time dependence.
Definition 5.1. An equilibrium point of a dynamical system of the form (10) is given by (x∗, u∗)such that x = f(x∗, u∗) = 0.
Let u(t) = u∗ + δu(t), y∗ = g(x∗, u∗) and x(0) = x∗ + δx. Define δx(t) := x(t) − x∗ andδy(t) := y(t)− y∗. Notice that
δx(t) = x(t) = f(x∗ + δx(t), u∗ + δu(t)) = f(x∗, u∗) +∂f
∂xδx+
∂f
∂uδu+O(‖δx‖2) +O(‖δu‖2) (11)
δy(t) = g(x, u)− y∗ = g(x∗ + δx(t), u∗ + δu(t))− y∗ =∂g
∂xδx+
∂g
∂uδu+O(‖δx‖2) +O(‖δu‖2). (12)
Thus, one obtains a linearized model around an equilibrium point. Note that linearization is validif there are no eigenvalues of the Jacobian matrix ∂f
∂x having zero real parts.Now let’s linearize around a trajectory. Let xsol, usol, ysol be a non constant solution trajectory
of (10). Let u(t) = usol(t)+δu(t), x(0) = xsol(0)+δx, δx(t) := x(t)−xsol(t), δy(t) := y(t)−ysol(t).Therefore,
δx(t) = x(t)− ˙xsol(t) = f(xsol(t) + δx(t), usol(t) + δu(t))− f(xsol(t), usol(t))
=∂f(xsol(t), usol(t))
∂xδx+
∂f(xsol(t), usol(t))
∂uδu+O(‖δx‖2) +O(‖δu‖2) (13)
δy(t) = y(t)− ysol(t) = g(xsol(t) + δx(t), usol(t) + δu(t))− g(xsol(t), usol(t))
=∂g(xsol(t), usol(t))
∂xδx+
∂g(xsol(t), usol(t))
∂uδu+O(‖δx‖2) +O(‖δu‖2) (14)
Thus, one obtains time varying matrices while linearizing along a trajectory.
5
5.2 Feedback linearization
The equations of motion of a general mechanical system can be written as
M(q)q +B(q, q)q +G(q) = F (15)
where q is a generalized position vector, M(q) > 0 is the mass matrix, B(q, q) represents cen-trifugal/Coriolis/friction matrix, and G(q) is a vector due to conservative forces. Using F =
B(q, q)q +G(q) +M(q)v, one obtains M(q)q = M(q)v. Now using x =
], one gets
x =
[0 I0 0
]x+
[0I
]v, y =
[0 I
]x.
Using v = −KP q −KD q = −[KP KD
]x, one obtains
x =
[0 I−KP −KD
]x, y =
[0 I
]x.
Systems in strict feedback form: Consider a system in the following form called strict-feedbackform
x1 = f1(x1) + x2
x2 = f2(x1, x2) + u. (16)
In this case, one cannot cancel the nonlinearity f1(x1) using u. Consider a change of variablez2 := f1(x1) + x2. Thus,
x1 = z2
z2 =∂f
∂x1x1 + x2 =
∂f(x1)
∂x1(f1(x1) + x2) + f2(x1, x2) + u. (17)
Now, we can cancel the nonlinearity using u and linearize the system.
6 Stability
We now study stability properties of linear systems in absence and presence of inputs. First westudy the internal stability of a system in the absence of inputs. Then, we consider bounded inputbounded output stability in the presence of inputs. This discussion is borrowed from Hespanha([1]).
6.1 Internal stability
Definition 6.1. • An autonomous system x = A(t)x is stable (marginally) if x(t) = Φ(t, t0)x0
is uniformly bounded.
• It is asymptotically stable if it is stable and x(t)→ 0 as t→∞.
• Exponentially stable if in addition to stability, there exists c, λ > 0 such that ‖x(t)‖ ≤ce−λ(t−t0)‖x(t0)‖.
• Unstable if it is not marginally stable.
Theorem 6.2. Let x = Ax. Then, the following statements are equivalent.
6
1. All eigenvalues of A lie strictly in the LHP.
2. The system is asymptotically stable.
3. The system is exponentially stable.
4. For every positive definite matrix Q, there exists a unique positive definite matrix P such thatATP + PA = −Q (Lyapunov equation).
5. There exists a positive definite matrix P such that ATP + PA < 0 (LMI).
Proof. Suppose 1 is satisfied. Therefore, eAt → 0 as t→∞. Hence x(t) = eAtx(0)→ 0 as t→∞.Thus 1⇒ 2. Suppose 1 is not satisfied. Then, eAt does not go to zero as t→∞. Hence the systemis not asymptotically stable. Hence 2 ⇒ 1. Since exponential stable is stronger than asymptoticstability, 3⇒ 2. We now show that 1⇒ 3.We want to show that the system is exponentially stable i.e. there exists c > 0, γ > 0 ∈ R suchthat ‖x(t)‖ ≤ ce−γ(t−t0)‖x(t0)‖.Since x(t) = eA(t−t0)x(t0),
‖x(t)‖ ≤ ‖eA(t−t0)‖‖x(t0)‖. (18)
Note that by definition, ‖eA(t−t0)‖ = max{y(t),‖y(t)‖=1}‖eA(t−t0)y(t)‖. Let y(t) be such that
‖eA(t−t0)‖ = ‖eA(t−t0)y(t)‖. For simplicity, assume that A has only one eigenvalue λ lying in theleft half plane. Therefore,
eA(t−t0)y(t) =
eλ(t−t0) (t− t0)eλ(t−t0) (t−t0)22! eλ(t−t0) . . . (t−t0)n−2
(n−2)! eλ(t−t0) (t−t0)n−1
(n−1)! eλ(t−t0)
0 eλ(t−t0) (t− t0)eλ(t−t0) . . . 0 (t−t0)n−2
(n−2)! eλ(t−t0)
. . . . . . . .
. . . . . . . .0 0 0 . . . eλ(t−t0) (t− t0)eλ(t−t0)
0 0 0 . . . 0 eλ(t−t0)
y1(t)y2(t)...
yn−1(t)yn(t)
Let {v1, . . . , vn} be the columns of eA(t−t0). Hence,
‖eA(t−t0)y(t)‖ = ‖y1(t)v1 + . . .+ yn(t)vn‖ ≤ |y1(t)|‖v1(t)‖+ . . .+ |yn(t)|‖vn(t)‖ (19)
Note that ‖y(t)‖ = 1, therefore |yi(t)| ≤ 1 for 1 ≤ i ≤ n. Hence the previous equation reduces tothe following
‖eA(t−t0)‖ = ‖eA(t−t0)y(t)‖ ≤ ‖v1(t)‖+ . . .+ ‖vn(t)‖
= eλ(t−t0) +√
(12 + (t− t0)2)eλ(t−t0) + . . .+
√(12 + (t− t0)2 + . . .+ (
(t− t0)n−1
(n− 1)!)2)eλ(t−t0)
= g(t− t0)eλ(t−t0) (20)
Observe that |g(t − t0)eλ(t−t0)| is bounded and goes to zero as t → ∞. Since g(t − t0)eλ(t−t0)
is decaying to zero, we can choose γ > 0 such that |g(t − t0)eλ(t−t0)| ≤ e−γ(t−t0) for t ≥ t1for some t1 ∈ R. Let c1 = supt≥0|g(t − t0)eλ(t−t0)|. Thus by choosing an appropriate c ∈ R,|g(t− t0)eλ(t−t0)| ≤ ce−γ(t−t0) for all t ≥ 0.
Hence, ‖x(t)‖ ≤ ce−γ(t−t0)‖x(t0)‖. The general case for multiple Jordan blocks follows similarly.
Suppose the system has all eigenvalues strictly in the LHP. Let P =´∞
0 eATtQeAtdt. Observe
that
ATP + PA =
ˆ ∞0
ATeATtQeAtdt+
ˆ ∞0
eATtQeAtAdt.
7
But ddt(e
ATtQeAt) = ATeATtQeAt + eA
TtQeAtA. Therefore,
ATP + PA =
ˆ ∞0
d
dt(eA
TtQeAt)dt = limt→∞(eATtQeAt)−Q = −Q.
Thus, P satisfies the Lyapunov equation. Observe that P = PT and P > 0 since Q > 0. Toshow that P is unique, let P be another solution. Therefore, ATP + PA = ATP + PA ⇒ AT(P −P ) + (P − P )A = 0 ⇒ eA
TtAT(P − P )eAt + eATt(P − P )AeAt = 0 ⇒ d
dt(eATt(P − P )eAt) = 0.
Thus, eATt(P − P )eAt remains constant with time, but it goes to zero as t → ∞. Therefore,
eATt(P − P )eAt = 0 ⇒ P = P . Hence, 1⇒ 4. Now 4⇒ 5 is trivial.Finally, we show that 5⇒ 3. Let Q := −(ATP+PA) and v(t) = xTPx > 0. Now v = xT(ATP+
PA)x = −xTQx < 0. Thus, v is a decreasing signal and v(t) = xT(t)Px(t) < v(0) = xT(0)Px(0).
v(t) = xT(t)Px(t) ≥ λmin(P )‖x(t)‖2 ⇒ ‖x(t)‖2 ≤ v(t)
λmin(P )(21)
v = −xT(t)Qx(t) ≤ −λmin(Q)‖x(t)‖2 ≤ −λmin(Q)
λmin(P )v(t). (22)
Lemma 6.3 (comparison lemma). Let v(t) be a scalar signal such that v ≤ µv(t), ∀t ≥ t0, forsome µ ∈ R. Then, v(t) ≤ eµ(t−t0)v(t0).
Proof. Let s(t) := e−µ(t−t0)v(t). Thus,
s = −µe−µ(t−t0)v(t) + e−µ(t−t0)v(t) ≤ −µe−µ(t−t0)v(t) + µe−µ(t−t0)v(t) = 0.
Therefore, s is nonincreasing and s(t) = e−µ(t−t0)v(t) ≤ s(t0) = v(t0) which proves the lemma.
Now applying the comparison lemma to (22), v(t) ≤ e−λmin(Q)
λmin(P )(t−t0)
v(t0). Since v(t) convergesto 0 exponentially, from (21), x(t) also converges to 0 exponentially.
Theorem 6.4. For the system x = Ax, following are equivalent.
1. The system is stable.
2. Every eigenvalue of A has a real part strictly less than zero or equal to zero and eigenvalueswith zero real part have trivial Jordan structure (i.e., they are semi-simple).
Proof. Note that eAt is bounded iff the condition on the eigenvalues of A mentioned above issatisfied. Therefore, 1⇔ 2.
Thus, to check stability, it is enough to find eigenvalues of the system matrix. From thecharacteristic equation of the underlying system, one can find a matrix in companion form havingthe same characteristic polynomial as the characteristic equation.
Remark 6.5. For stability/asymptotic stability of time varying systems, we must have Φ(t, t0)bounded for all t ≥ t0 or Φ(t, t0)→ 0 as t→∞. For exponential stability, Φ(t, t0) must go to zeroexponentially and so on.
Geometry of Lyapunov functions Any positive definite matrix P defines an energy func-tions on the state space where the energy v(t) = xTPx. This defines ellipsoids which are energysurfaces on the state space. Consider a set {x ∈ Rn |xTPx = c} where c ∈ R. This is an ellipsoidwhich contains all states whose energy v(t) = xTPx = c. Thus, the entire state space is partitionedas a union of these ellipsoids (energy surfaces). As x(t) evolves as a function of time (say x = f(x)is the underlying dynamical system), its energy varies accordingly since x(t) is varying as a functionof time. If v = 0, then the rate of change of energy is zero. Thus, the energy of x(t) is constant.
8
Hence, it does not leave the energy surface (ellipsoid) on which it is already lying. If v > 0, thenthe rate of change is positive and energy of the state trajectory is increasing hence, it goes to ahigher energy surface. If v < 0, then the trajectory moves to a lower energy surface. If v ≤ 0 for allpoints in the set {x ∈ Rn |xTPx ≤ c}, then once a trajectory enters this set, it can not leave thisset. This is the idea to prove invariance of a set or stability/asymptotic stability related arguments.
Consider a nonlinear autonomous system x = f(x) and its linearization around the equilibriumpoint x∗.
Theorem 6.6 (local stability via linearization). Consider x = f(x) where f is at least twicedifferentiable on a closed ball B ⊂ Rn around the equilibrium point x∗. If the linearized systemδx = Aδx, A := ∂f(x∗)
∂x is exponentially stable, then there is exists a ball B ⊂ Rn around x∗ andconstants c, λ > 0 such that for every solution x(t) of the nonlinear system that starts at x(t0) ∈ B,
‖x(t)− x∗‖ ≤ ce−λ(t−t0)‖x(t0)− x∗‖, ∀t ≥ t0. (23)
Proof. Let r(x) := f(x)− f(x∗)−Aδx. By Taylor’s formula, there exists d > 0 such that
‖r(x)‖ ≤ d‖δx‖2, ∀x ∈ B. (24)
Since, the linearized system is exponentially stable, there exists P > 0 such that ATP +PA = −I.Let v(t) := δxTPδx.
v = f(x)TPδx+ δxTPf(x)
= (Aδx+ r(x))TPδx+ δxTP (Aδx+ r(x))
= δxT(ATP + PA) + 2δxTPr(x)
= −‖δx‖2 + 2δxTPr(x)
≤ −‖δx‖2 + 2‖P‖‖δx‖‖r(x)‖. (25)
We want to make the rhs negative. For example,
−‖δx‖2 + 2‖P‖‖δx‖‖r(x)‖ ≤ −1
2‖δx‖2.
To achieve this, let ε > 0 be small enough such that the ellipsoid
E := {x ∈ Rn | (x− x∗)TP (x− x∗) ≤ ε} (26)
satisfies the following properties
1. The ellipsoid E is contained in B. When x is inside this ellipsoid, (24) holds, hence,
x(t) ∈E ⇒ v ≤ −‖δx‖2 + 2d‖P‖‖δx‖3 = −(1− 2d‖P‖‖δx‖)‖δx‖2.
2. We further shrink ε such that inside the ellipsoid E, (1− 2d‖P‖‖δx‖) ≥ 12 ⇔ ‖δx‖ ≤
14d‖P‖ .
For this choice of ε, x(t) ∈E ⇒ v ≤ −12‖δx‖
2.
Once x(t) ∈E, v(t) ≤ ε, v ≤ 0. Thus, v cannot increase above ε, hence, it cannot exit E.Note that
δxTPδx ≤ ‖P‖‖δx‖2 ⇒ ‖δx‖2 ≥ δxTPδx
‖P‖⇒ −1
2‖δx‖2 ≤ −1
2
δxTPδx
‖P‖= −1
2
v(t)
‖P‖. (27)
Now if x(0) ∈ B ⊂E, by (27) v ≤ −12‖δx‖
2 ≤ −12v(t)‖P‖ . Hence, by the comparison lemma, v and
consequently δx = x− x∗ decrease to zero exponentially.
Theorem 6.7 (unstability via linearization). Consider x = f(x) where f is twice differentiable
and f(x∗) = 0. If the linearized system δx = Aδx, A := ∂f(x∗)∂x is unstable, then there are solutions
that start arbitrarily close to x∗ , but do not converge to this point as t→∞.
9
6.2 Input-output stability
Note that the forced output of a linear system due to the input u(t) is given by
yf (t) =
ˆ t
0C(t)Φ(t, τ)B(τ)u(τ)dτ +D(t)u(t).
Definition 6.8. A system is said to be uniformly BIBO stable if there exists a finite constant gsuch that
supt∈[0,∞]‖yf (t)‖ ≤ gsupt∈[0,∞]‖u(t)‖ (28)
Theorem 6.9 (time domain characterization). Following statements are equivalent.
1. The system is uniformly BIBO stable.
2. Every entry of D(.) is uniformly bounded and
supt≥0
ˆ t
0|gij(t, τ)|dτ <∞ (29)
for every entry gij(t, τ) of g(t, τ) = C(t)Φ(t, τ)B(τ).
Proof. Sketch: Clearly, 2 ⇒ 1 since, taking sup over the rhs, we can bound the forced responseyf (t). On the other hand, if 2 is not satisfied, then we cannot bound yf (t) over all bounded inputsas there will be bounded inputs corresponding to unbounded entries in either D(.) or g(t, τ) whichmake the forced response yf (t) unbounded.
Theorem 6.10 (Frequency domain characterization). Following statements are equivalent for LTIsystems.
1. The system is uniformly BIBO stable.
2. Every pole of every entry of the transfer function of the system has a strictly negative realpart.
Proof. Sketch: The second statement in the time-domain characterization is equivalent to thestatement that every entry Gij(s) of the transfer matrix G(s) corresponds to either an exponentiallydecaying term in the time domain.
Note that exponential/asymptotic or internal stability implies BIBO stability but the converseneed not be true.
Example 6.11. Consider x = Ax+Bu, y = Cx where
A =
[1 00 −2
], B =
[01
], C =
[1 1
].
In this case eAt becomes unbounded, hence the system is not internally stable. But CeAtB = e−2t
which is bounded, hence the system is BIBO stable.
7 Controllability
We now formalize one of the central properties of control system which is controllability. Theother one being observability. We also study an equally important notion of stabilizability and poleplacement. We will see that control problems are basically least norm problems.
10
Definition 7.1. A system x = A(t)x(t) + B(t)u(t) is said to be controllable if for any initialcondition x(t0) ∈ Rn (or Cn) and for any final condition xf ∈ Rn (or Cn), there exists an inputu(t) which drives the state from x(t0) to xf in finite time t ∈ R. If t0 = 0 and x(0) = 0 in theabove case and xf is arbitrary, then we say that the system is reachable. Moreover, if xf = 0 andx(t0) is arbitrary in the above case, then we say that the system is 0−controllable.
Consider the following equation for linear systems
x(t) = Φ(t, t0)x(t0) +
ˆ t
t0
Φ(t, τ)B(τ)u(τ)dτ (30)
⇒ x(t)− Φ(t, t0)x(t0) =
ˆ t
t0
Φ(t, τ)B(τ)u(τ)dτ (31)
Thus, the system if controllable ⇔ the range space of´ tt0
Φ(t, τ)B(τ)u(τ)dτ spans the entire statespace. Note that if the system is controllable, then we have a least norm problem (please refer mylinear algebra notes). There could be multiple u(t)s driving the state from the initial to the finalcondition and we would like to find the input with least norm (or least energy). It is clear that forreachability and 0−controllability, we have the same equivalent condition. Hence all these notionsare equivalent for continuous time systems. For discrete time systems, state transition matrix isnot invertible if the system matrix A is non-invertible. For discrete time systems, these notions areequivalent when A is invertible. If A is not invertible for discrete time systems, then the systemcannot be controllable or reachable. But it can be 0−controllable.
Theorem 7.2. Following statements are equivalent for LTI systems:
1. (A,B) is controllable.
2. The Controllability Gramian Wt(A,B) :=´ tt0eA(t−τ)BBTeA
T(t−τ)dτ > 0 for some t > t0.
3. The Controllability matrix C(A,B) =[B AB . . . An−1B
]has rank n (rank test).
4. [λI −A B] has rank n for all λ ∈ C (PBH test).
Proof. (1 ⇔ 2) Suppose the Controllability Gramian´ tt0eA(t−τ)BBTeA
T(t−τ)dτ > 0. Since x(t) =
eA(t−t0)x(t0) +´ tt0eA(t−τ)Bu(τ)dτ , choose u(τ) = BTeA
T(t−τ)η. Thus, x(t) = eA(t−t0)x(t0) +
Wt(A,B)η and η = Wt(A,B)−1(x(t)− eA(t−t0)x(t0)). This shows that 2⇒ 1.Suppose (A,B) is controllable, hence for all x(t) ∈ Rn, there exists u(t) such that x(t) =
eA(t−t0)x(t0) +´ tt0eA(t−τ)Bu(τ)dτ . Thus, x(t) − eA(t−t0)x(t0) =
´ tt0eA(t−τ)Bu(τ)dτ . Since, x(t) −
eA(t−t0)x(t0) lies in the range space of´ tt0eA(t−τ)Bdτ , the rank of
´ tt0eA(t−τ)Bdτ must be n for some
t > t0. Therefore, the rank of´ tt0eA(t−τ)BBTeA
T(t−τ)dτ must also be n for t > t0 and 1⇒ 2. (One
can also show ∼ 2⇒∼ 1 as follows. Suppose the Controllability Gramian´ tt0eA(t−τ)BBTeA
T(t−τ)dτ
is singular i.e. there exists v such that vTWt(A,B)v = 0 i.e. vTeA(t−τ)BBTeAT(t−τ)v = 0. Hence,
vTeA(t−τ)B = 0. Thus, the range space of´ tt0eA(t−τ)Bu(τ)dτ is not equal to Rn since it has a
non-trivial kernel. Hence, (A,B) is uncontrollable.)
Suppose the Controllability Gramian´ tt0eA(t−τ)BBTeA
T(t−τ)dτ is singular i.e., there exists v
such that vTWt(A,B)v = 0 i.e. vTeA(t−τ)BBTeAT(t−τ)v = 0. Hence, vTeA(t−τ)B = 0. Let ρ = t−τ .
Thus, Wt(A,B) =´ t−t0
0 eAρBBTeATρdρ and f(ρ) = vTeAρB = 0 for all ρ ∈ [0, t − t0]. At ρ = 0,
f(0) = vTB = 0. Since this is a differentiable function of ρ, differentiating f(ρ) n−1 times at ρ = 0,we obtain vTAB = . . . = vTAn−1B = 0. Thus, the rank of C(A,B) =
[B AB . . . An−1B
]is less than n since there exists a non-trivial kernel. Conversely, if C(A,B) is rank deficient, thenWt(A,B) singular since by Cayley-Hamilton theorem, eAρ can be expressed as a linear combinationof I, A, . . . , An−1. Hence, (2⇔ 3).
11
Suppose [λI − A B] loses rank at λ ∈ C. Therefore, v∗[λI − A B] = 0. Thus, v∗ belongs tothe left kernel of C(A,B) hence C(A,B) is not full row rank. Conversely, suppose v∗C(A,B) = 0.Thus, v∗B = 0 and v∗Ai ∈ ker(B) for i = 1, . . . , n − 1. Consider < v∗, v∗A, . . . , v∗An−1 >, anA−invariant subspace contained in ker(B). Since the subspace is A−invariant, it must contain aneigenvector of A. Therefore, there exists w∗ ∈< v∗, v∗A, . . . , v∗An−1 > such that w∗A = λA andw∗B = 0. Hence, [λI −A B] is rank deficient. Therefore, (3⇔ 4).
Corollary 7.3. For LTV systems, (A(t), B(t)) is controllable⇔´ tt0
Φ(t, τ)B(t)BT(t)ΦT(t, τ)dτ > 0for some t > t0.
Theorem 7.4. Controllability of (A,B) remains invariant under a change of basis.
Proof. Follows from the rank test of the Controllability matrix. Suppose the Controllability matrixC(A,B) =
[B AB . . . An−1B
]has rank n. Let A = T−1AT and B = T−1B. Consider
T−1C(A,B) =[T−1B T−1ATT−1B . . . T−1An−1TT
−1B]
= C(A, B). Thus, C(A, B) hasrank n. Reversing the arguments, one can show that if C(A, B) has rank n, then C(A,B) has rankn.
Theorem 7.5. (Invariance under feedback) (A,B) is controllable ⇔ (A−BF,B) is controllable.
Proof. Follows by applying PBH test to (A,B) and (A − BF,B). Assuming that the matrix forPBH test is rank deficient for one system, one can show that the corresponding matrix for othersystem is also rank deficient.
Now we show that for controllable systems, one can do an arbitrary pole placement to havethe desired system poles. Controllability is unaffected by the previous theorem. The followinglemma and the pole placement theorem is taken from Wonham (Linear Multi-variable control: Ageometric approach).
Lemma 7.6. Let 0 6= b ∈ column span of B. If (A,B) is controllable, then there exists a feedbackmatrix F such that (A+BF, b) is controllable.
Proof. Let b1 = b. Let X1 be a cyclic subspace generated by b1 under the action of A i.e.X1 =< b1, Ab1, . . . , A
n1−1b1 > where {b1, Ab1, . . . , An1−1b1} forms a linearly independent set. Letx1 = b1, x2 = Ax1 + b1, . . . , xn1 = Axn1−1 + b1. Clearly, xis are linearly independent since theset {b1, Ab1, . . . , An1−1b1} is linearly independent. Choose b2 /∈ X1 and generate a cyclic subspaceX2 =< b2, Ab2, . . . , A
n2−1b2 > where {b2, Ab2, . . . , An2−1b2} forms a linearly independent set. De-fine xn1+1 = Axn1 + b2, xn1+2 = Axn1+1 + b2 and so on. Thus, {x1, . . . , xn1+n2} is a basis forX1 + X2. Continuing this way we obtain a basis {x1, . . . , xn} for the entire state space. (Since(A,B) is controllable, rank of C(A,B) is n. Therefore, the above procedure generates the entirestate space.)
Let xi+1 = Axi + bi (1 ≤ i ≤ n − 1). Let bi = Bηi. Define a linear map F on the state spacesuch that F (xi) = ηi (1 ≤ i ≤ n). Thus, xi+1 = Axi+ bi = Axi+Bηi = Axi+BFxi = (A+BF )xi.Therefore, {x1, (A+BF )x1, . . . , (A+BF )n−1x1} generate the entire state space where x1 = b1.
Theorem 7.7 (Pole placement). (A,B) is controllable ⇔ for every symmetric set Λ of complexnumbers, there exists a feedback matrix F such that eigenvalues of (A+BF ) are given by Λ.
Proof. (⇒) Suppose we have a single input system and (A,B) is controllable. Therefore, B = bis a cyclic vector. Suppose pA(s) = sn − ans
n−1 − . . . − a2s − a1. Let Λ = {λ1, . . . , λn} and(s− λ1).(s− λ2) . . . (s− λn) = sn − ansn−1 − . . .− a2s− a1 the desired characteristic polynomial.By change of basis (since (A, b) is controllable, span{b, Ab, . . . , An−1b} = Rn, choose v1 = An−1b−
12
anAn−2b − . . . − a2b, v2 = An−2b − anAn−3b − . . . − a3b, . . . , vn−1 = Ab − anb, vn = b as a new
basis), we may assume that (A, b) is in the following companion form
A =
0 1 . . . 00 0 . . . 0. . . . . .. . . . . 1a1 a2 . . . an
, b =
0..01
.
Choosing f =[a1 − a1 a2 − a2 . . . an − an
], A+ bf has the desired eigenvalues.
For multi-input systems, from the previous lemma, there exists F1 and b ∈ column span of Bi.e. b = Bη such that (A + BF1, b) is cyclic. Thus by the single input case above, there exists fsuch that A+BF1 + bf = A+BF1 +Bηf = A+B(F1 +ηf) = A+BF has the desired eigenvalues.
(⇐) Choose Λ to be a set of n real and distinct numbers {λ1, . . . , λn} such that λi is not aneigenvalue of A (1 ≤ i ≤ n). Choose F such that {λ1, . . . , λn} are eigenvalues of (A + BF ) and{x1, . . . , xn} are the corresponding eigenvectors.
(A+BF )xi = λixi ⇒ (λiI −A)xi = BFxi ⇒ xi = (λiI −A)−1BFxi
((λiI − A)) is invertible for (1 ≤ i ≤ n) since λi is not an eigenvalue of A. Note that (λiI − A)−1
is an analytic function since λi is not an eigenvalue of A and hence it is a power series expansion.By Cayley-Hamilton theorem, this power series can be expressed as follows:
(λiI −A)−1 =n∑j=1
pj(λi)Aj−1 (32)
Thus, xi = (λiI − A)−1BFxi =∑n
j=1 pj(λi)Aj−1BFxi. Therefore, the basis vectors xi ∈ column
span of C(A,B) =[B AB . . . An−1B
]which implies that rank of C(A,B) is n. Hence,
(A,B) is controllable.
Example 7.8 (MIMO pole placement). Let A =
[λ 00 λ
], B =
[1 00 1
]. Following the proof
of Lemma 7.6, X1 =< b1 > and X2 =< b2 >. X = X1 ⊕ X2. Moreover, a basis for X isx1 = b1, x2 = Ax1 + b2. Note that in the notation of Lemma 7.6, x2 = Ax1 + b1 = Ax1 +Bη1 whereb1 = b2 and b2 = Be2. Hence, η1 = e2 and Fx1 = e2. Let Fx2 = 0. Thus,
A+BF =
[λ 00 λ
]+
[1 00 1
] [0 01 0
]=
[λ 01 λ
]. (33)
Clearly, (A+BF, b1) is cyclic. Now we can do an pole placement using f1 such that A+BF + b1f1
has the desired characteristic polynomial. (A+BF + b1f1 = A+BF +Be1f1 = A+B(F + e1f1) =A+BK).
Example 7.9 (MIMO pole placement). Let A =
λ 1 0 00 λ 0 00 0 λ 10 0 0 λ
and B =
0 01 00 00 1
. Suppose
we want to place poles at arbitrary locations. Following the proof of Lemma 7.6, X1 =< b1, Ab1 >and X2 =< b2, Ab2 >. X = X1 ⊕X2. Moreover, a basis for X is x1 = b1, x2 = Ax1 + b1, x3 =Ax2 + b2, x4 = Ax3 + b2. Note that xi+1 = Axi + bi 1 ≤ i ≤ 3 where b1 = b1, b2 = b2, b3 = b2.Furthermore, b1 = Be1 and b2 = Be2. Hence, η1 = e1, η2 = e2, η3 = e2. Recall that F was definedby its action of xis i.e.,
Fx1 = η1 = e1, Fx2 = η2 = e2, Fx3 = η3 = e2, Fx4 = η (34)
13
where η is some vector whose value will be specified later. Now we find F by its action on old basisvectors e1, . . . , e4 of X since we have (A,B) in the standard basis representation.
x1 = b1 = e2, x2 = Ax1 + b1 = Ae2 + e2 = e1 + (λ+ 1)e2, (35)
x3 = Ax2 + b2 = A(e1 + (λ+ 1)e2) + e4 = (2λ+ 1)e1 + λ(λ+ 1)e2 + e4 (36)
x4 = Ax3 + b2 = A((2λ+ 1)e1 + λ(λ+ 1)e2 + e4) + e4 = (2λ+ 1)λe1 + λ(λ+ 1)Ae2 +Ae4 + e4
= (λ(2λ+ 1) + λ(λ+ 1))e1 + λ2(λ+ 1)e2 + e3 + (λ+ 1)e4
= λ(3λ+ 2)e1 + λ2(λ+ 1)e2 + e3 + (λ+ 1)e4 (37)
Therefore, from previous four equations,
Fx1 = Fe2 = e1, Fx2 = F (e1 + (λ+ 1)e2) = e2 ⇒ Fe1 = −(λ+ 1)e1 + e2 (38)
F (x3) = F ((2λ+ 1)e1 + λ(λ+ 1)e2 + e4) = e2 ⇒−(2λ+ 1)(λ+ 1)e1 + (2λ+ 1)e2 + λ(λ+ 1)e1 + Fe4 = e2 ⇒Fe4 = (λ+ 1)2e1 − 2λe2 (39)
F (x4) = F (λ(3λ+ 2)e1 + λ2(λ+ 1)e2 + e3 + (λ+ 1)e4) = η ⇒−λ2(3λ+ 2)e1 + λ2(λ+ 1)e1 + Fe3 + (λ+ 1)(λ2e1 + e2) = η (40)
Choose η such that Fe3 = 0. Thus F =
[−(λ+ 1) 1 0 (λ+ 1)2
1 0 0 −2λ
]and
A+BF =
λ 1 0 00 λ 0 00 0 λ 10 0 0 λ
+
0 01 00 00 1
[ −(λ+ 1) 1 0 (λ+ 1)2
1 0 0 −2λ
]
=
λ 1 0 00 λ 0 00 0 λ 10 0 0 λ
+
0 0 0 0
−(λ+ 1) 1 0 (λ+ 1)2
0 0 0 01 0 0 −2λ
=
λ 1 0 0
−(λ+ 1) λ+ 1 0 (λ+ 1)2
0 0 λ 11 0 0 −λ
. (41)
One can check that (A + BF, b1) = (A + BF, e2) is cyclic. Now we can use SISO trick for poleplacement.
We give the following algorithm for MIMO pole placement based on Lemma 7.6.
1. Select b = b1 i.e. the first column of B and generate a maximal cyclic subspace X1. Thusx1 = b1, . . . , xn1 = Axn1−1 + b1.
2. Select b2 i.e. the second column of B and generate X2. xn1+1 = Axn1 + b2, . . . , xn1+n2 =Axn1+n2−1 + b2 and so on. Thus, we obtain a basis {x1, . . . , xn} for X.
3. b1 = Be1, b2 = Be2, . . . , bk = Bek, xi+1 = Axi + bi. For i = 1, . . . , n1 − 1 bi = b1 = Be1. Fori = n1, . . . , n1 + n2 − 1, bi = b2 = Be2 and so on.
4. Define F : X → U (where U is an input space) as follows. For i = 1, . . . , n1 − 1, Fxi = e1.For i = n1, . . . , n1 + n2 − 1, Fxi = e2 and so on and finally Fxn = η where η can be definedto ease computations.
14
5. Find representation of F in standard basis (assuming that A,B were given in a standardbasis). Then (A+BF ) is cyclic w.r.t. b1 and we are in SISO case.
Remark 7.10. Suppose one wants to drive the state from x(t0) to x(tf ) = xf and hold it therefor all t ≥ tf . For a controllable system, one needs to choose u|[t0,tf ] such that x(tf ) = xf . Then,for all t ≥ tf , u(t) must be chosen such that x = Ax + Bu = 0. Since x(t) = xf for all t ≥ tf ,0 = Axf +Bu(t) i.e., choose u(t) such that Bu(t) = −Axf . This implies that Axf must lie in thecolumn span of B if one wants to hold the state at xf for all t ≥ tf .
Some canonical forms:
Lemma 7.11 (Wonham). Let B be a subspace with minimal polynomial β w.r.t. A. Then, thereexists b ∈B with minimal polynomial β.
Proof. Let B1 = B + AB + . . . + An−1B be of dimension m. Restrict A to the subspace B1
and denote this restriction by AB1 (an m ×m matrix). Now the minimal polynomial of B w.r.t.A is the same as the minimal polynomial of the A−invariant subspace B1 which is the minimalpolynomial of AB1 .
Suppose b1 ∈ B generates the maximal cyclic subspace Bb1 among all vectors in B. Let Bb2
be the second maximal cyclic subspace for some b2 ∈ B and so on. Using similar arguments usedin linear algebra in the derivation of Rational canonical form, one can show that the minimalpolynomial of b2 divides the minimal polynomial of b1 and so on. Let B1 = Bb1 ⊕ . . . ⊕ Bbk ,bi ∈ B. This gives a cyclic decomposition of B1 and consequently that of AB1 . Let mb1 be theminimal polynomial of b1. Thus, mb1(AB1) = 0. We need to show that B1 = Bb1 ⊕ . . . ⊕Bbk
gives the Rational canonical decomposition of AB1 . Let B1 = Bv1 ⊕ . . .⊕Bvl be a decompositionwhich gives the Rational canonical form where vi ∈ B1 and l ≤ k. Clearly, dim(Bv1) ≥ dim(Bb1)and the minimal polynomial of v1 is same as the minimal polynomial of AB1 which is β(s). Butmb1(AB1) = 0 ⇒ β(s)|mb1(s) which implies that dim(Bv1) = dim(Bb1). Thus, b1 ∈ B generatesthe maximal cyclic subspace of B1. Therefore, b = b1 gives the required vector.
Theorem 7.12 (Wonham). Suppose (A,B) is controllable. Let α1, . . . , αk be the invariant factorsof A. Then, there exists A−invariant subspaces Xi ⊆ X and vectors bi ∈ column span of B suchthat
• X = X1 ⊕ . . .⊕Xk.
• A restricted to Xi is cyclic with minimal polynomial αi.
• If Bi =< b1 > + . . .+ < bi > (1 ≤ i ≤ k), then <Bi, ABi, . . . , An−1Bi >= X1 ⊕ . . .⊕Xi.
Proof. Since (A,B) is controllable, the minimal polynomial of B is same as the minimal polynomialof A. By the previous lemma, there exists b1 ∈B such that the minimal polynomial of b1 is sameas the minimal polynomial of B which is mA. Let X1 =< b1, Ab1, . . . , A
n−1b1 >. Since X1 ismaximaly cyclic, X = X1 ⊕Y1 where AY1 ⊂ Y1. Let Q be a projection on Y1 along X1. Sinceboth X1 and Y1 are A−invariant, QA = AQ and (I −Q)A = A(I −Q). Therefore,
Y1 = QX = Q <B, AB, . . . , An−1B >=< QB, AQB, . . . , An−1QB >
By the previous lemma and controllability, there exists b2 ∈ B such that Qb2 under the action ofA forms a maximal cyclic subspace X2 ⊂ Y1 = X2 ⊕Y2. Note that b2 ∈ X1 ⊕X2. Furthermore,B2 =< b1 > + < b2 > and <B2, AB2, . . . , A
n−1B2 >= X1⊕X2. Continuing this way, one obtainsthat if Bi =< b1 > + . . .+ < bi > (1 ≤ i ≤ k), then <Bi, ABi, . . . , A
n−1Bi >= X1⊕ . . .⊕Xi.
15
It is clear that the basis used in the proof above to convert A into Rational canonical formis {b1, Ab1, . . . , b2, Ab2, . . . , bk, Abk, . . . , Ank−1bk}. Let B =
[b1 b2 . . . bm
]. Recall that un-
der a change of basis, B is mapped to T−1B. Choose B1 =[b1 b2 . . . bm
]and define
T−1 such that T−1bi = bi for 1 ≤ i ≤ k and extend it to an n × n invertible map. Thus,under this change of basis, we may assume that A gets transformed to T−1AT and T−1B =[b1 b2 . . . bm
]. Denote the transformed matrices by (A,B) pair itself by abusing notation.
Now choose {b1, Ab1, . . . , b2, Ab2, . . . , bk, Abk, . . . , Ank−1bk} as a basis. With respect to this basis,
A =
A1 0 . . . 00 A2 . . . 0. . . . . .. . . . . .0 0 . . . Ak
, B =
b11 b12 . . . b1n ∗0 b22 . . . b2n ∗. . . . . . .. . . . . . .0 0 . . . bkk ∗
. (42)
Note that in general, one can obtain various canonical forms for controllable (A,B) pairs. LetB =
[b1 b2 . . . bm
]. We may consider {b1, Ab1, . . . , b2, Ab2, . . . , } as a basis for X. X1 =<
b1, Ab1, . . . > is a cyclic subspace generated by b1. If b2 /∈ X1 adjoin b2 to previous linearlyindependent set. Keep adjoining Aib2 until one gets a linearly independent set and so on. Thisbasis gives a canonical form for (A,B).
One can choose another basis as follows. Consider {b1, . . . , bm}, the linearly independentcolumns of B. Now act A on this set and adjoin linearly independent vectors. Act A againon the set and adjoin newly generated linearly independent elements and so on. This also gener-ates a basis for X and w.r.t. this basis, one gets a different canonical form for (A,B). We willrevisit the significance of subspaces mentioned above in Section 13.2.
Remark 7.13. Suppose the basis obtained in the previous method to generate a basis for the statespace is {b1, Ab1, . . . , Ak1−1b1, b2, Ab2, . . . , A
k2−1b2, . . . , Akm−1bm}. Then {k1, . . . , km} are called
controllability indices of an (A,B) pair. They are invariants of a control system.
Open loop control using the Controllability Gramian and the least input energy:We now obtain an expression for the minimum input energy required for a state transfer from x(0) to
xf in time t. Let ul(τ) = BTeAT(t−τ)η. One can check that choosing η = Wt(A,B)−1(xf−eAtx(0)),
ul(τ) = BTeAT(t−τ)Wt(A,B)−1(xf−eAtx(0)) does the state transfer from x(0) to xf in time t. Thus,
the energy required for this transfer isˆ t
0‖ul(τ)‖2dτ = (xf − eAtx(0))TWt(A,B)−1Wt(A,B)Wt(A,B)−1(xf − eAtx(0))
= (xf − eAtx(0))TWt(A,B)−1(xf − eAtx(0)). (43)
Let u(τ) be any other input doing the same state transfer. We show that (u−ul) ⊥ ul. Since bothinputs does the same state transfer,ˆ t
0eA(t−τ)B(u(τ)− ul(τ))dτ = 0
⇒ ηTˆ t
0eA(t−τ)B(u(τ)− ul(τ))dτ = 0
⇒ˆ t
0(BTeA
T(t−τ)η)T(u(τ)− ul(τ))dτ = 0
⇒ˆ t
0uTl (τ)(u(τ)− ul(τ))dτ = 0.
Now using above relation and using
‖u‖2 =
ˆ t
0u(τ)Tu(τ)dτ = ‖u− ul + ul‖2 = ‖ul‖2 + ‖u− ul‖2( using the orthogonality relation above)
16
Thus, ‖u‖2 ≥ ‖ul‖2 and ul is the least energy input.
Now we study some properties of the Controllability Gramian when (A,B) is controllable.
Wt(A,B) =
ˆ t
0eA(t−τ)BBTeA
T(t−τ)dτ = eAt(
ˆ t
0e−AτBBTe−A
Tτdτ)eATt
⇒ d
dtWt(A,B) = AWt(A,B) +BBT +Wt(A,B)AT( differential Lyapunov equation).
Suppose A is stable, i.e. all eigenvalues of A lie strictly in the LHP. Then,
W∞(A,B) =
ˆ ∞0
eA(t−τ)BBTeAT(t−τ)dτ =
ˆ ∞0
eAρBBTeATρdρ
⇒ AW∞(A,B) +W∞(A,B)AT =
ˆ ∞0
d
dρ(eAρBBTeA
Tρ)dρ = −BBT
Thus, W∞(A,B) satisfies a Lyapunov equation. Moreover, the solution to this Lyapunov equationis unique. Suppose W is some other solution of the Lyapunov equation.
A(W∞(A,B)− W ) + (W∞(A,B)− W )AT = 0⇒ eAt(A(W∞(A,B)− W ) + (W∞(A,B)− W )AT)eATt
= 0
⇒ d
dt(eAt(W∞(A,B)− W )eA
Tt) = 0⇒ˆ ∞
0
d
dt(eAt(W∞(A,B)− W )eA
Tt)dt = 0
Thus, evaluating the integral at boundary points, eAt vanishes at ∞, hence W∞(A,B) = W .
The following result is from Hespanha (Theorem 12.4).
Theorem 7.14. Suppose all eigenvalues of A lie strictly in the LHP. Then, (A,B) is controllable⇔there exists a unique positive definite solution W to the Lyapunov equation AW +WAT = −BBT.
Proof. If (A,B) is controllable and A stable, we saw above that the Controllability Gramian satisfiesthe Lyapunov equation and the solution is unique. Conversely, assume that a unique positivedefinite solution exists for the Lyapunov equation. We need to show that (A,B) is controllable.Suppose (A,B) is not controllable. Therefore, by PBH test, there exists w such that w∗A = λw∗
and w∗B = 0. This implies that
w∗(AW +WAT)w = (λ+ λ∗)w∗Ww = −w∗BBTw = 0
Since A is stable, Re(λ) < 0. Thus, w∗Ww = 0 contradicting positive definiteness of W . Hence,(A,B) is controllable.
Observe that eigenvectors of A and −µI − A are the same. One can choose µ > 0 such thateigenvalues of −µI − A are strictly in the LHP. By the PBH test, (−µI − A,B) is controllable ⇔(A,B) is controllable. Thus, from the above theorem, there exists W > 0 such that
(−µI −A)W +W (−µI −A)T = −BBT ⇒ AW +WAT −BBT = −2µW (44)
Let P = W−1 > 0. Therefore, from the previous equation,
PA+ATP − PBBTP = −2µP ⇒ P (A−BK) + (A−BK)TP = −2µP (K :=1
2BTP ). (45)
Thus, from Lyapunov stability test, A−BK is stable and a state feedback control u = −Kx makesthe system asymptotically stable.Controllable subspace and stabilizability: Suppose rank(C(A,B)) < n. Then, the columnspan of C(A,B) forms a controllable subspace of the state space X. If x(0) and xf both lie in the
17
controllable subspace, it is possible to find an input which does the required state transfer in finitetime. But if either one of them lies outside the controllable subspace, it is impossible to do therequired state transfer.
Observe that by construction, the controllable subspace if A−invariant. Thus, extending a basisfor controllable subspace to the entire state space, we get the following canonical form for (A,B)
A =
[A1 A12
0 A2
], B =
[B1
0
].
where (A1, B1) is controllable and eigenvalues of A1 are controllable. Observe that A1 above isobtained by restricting A to the A−invariant controllable subspace and B1 is the representation ofB on this subspace. Note that if the rank of the Controllability matrix is n1 < n, then n1 is thedimension of this subspace. Therefore, by construction (A1, B1) is controllable. Note that A2 formsan uncontrollable part of the system and eigenvalues of A2 are uncontrollable. Observe that theleft eigenvectors corresponding to uncontrollable eigenvalues belong to the left kernel of B. Since(A1, B1) is controllable, one can change eigenvalues of A1 using appropriate feedback. Eigenvaluesof A2 can not be changed under any feedback. Note that if eigenvalues of A2 lie in the LHP, thenusing a proper feedback, eigenvalues of (A + BF ) can be made to lie in the LHP. Observe thatuncontrollable states need not form an A−invariant subspace.
Definition 7.15. A system (A,B) is said to be stabilizable, if for every initial condition x(0), thereexists an input u(.) such that limt→∞x(t) = 0.
Theorem 7.16. Following are equivalent:
1. (A,B) is stabilizable.
2. Uncontrollable eigenvalues of A are stable.
3. rank([λI −A B]) = n for all λ ∈ C such that Re(λ) ≥ 0.
Proof. 1 ⇔ 2 is clear. 2 ⇒ 3 and 3 ⇒ uncontrollable eigenvalues are in the LHP, hence, stable.Thus, 3⇒ 2.
Note that by pole placement theorem, controllability implies stabilizability. The following resultis from Hespanha (Theorem 14.4).
Theorem 7.17. (A,B) is stabilizable ⇔ there exists a positive definite solution P to the Lyapunovmatrix inequality AP + PAT −BBT < 0.
Proof. (⇒) Suppose (A,B) is in the following form after a change of basis
A =
[A1 A12
0 A2
], B =
[B1
0
].
Since, (A1, B1) is controllable, by Equation (44), there exists positive definite P1 such that A1P1 +P1A
T1−B1B
T1 = −Q1 < 0. Moreover, sinceA2 is stable, there exists P2 > 0 such thatA2P2+P2A
T2 =
−Q2 < 0. Let P =
[P1 00 ρP2
], ρ > 0. Thus, AP + PAT −BBT =
[A1 A12
0 A2
] [P1 00 ρP2
]+
[P1 00 ρP2
] [AT
1 0AT
12 AT2
]−[B1B
T1 0
0 0
]= −
[Q1 −ρA12P2
−ρP2AT12 ρQ2
].
By making ρ sufficiently small, the RHS can be made negative definite.(⇐) Let v∗ be a left eigenvector of A associated with an unstable eigenvalue λ. Thus, Re(λ) > 0
v∗(AP + PAT −BBT)v < 0⇒ 2Re(λ)v∗Pv − ‖v∗B‖2 < 0⇒ ‖v∗B‖2 > 0. (46)
Therefore, v∗ /∈ kerB. Hence, (A,B) is stabilizable by PBH test for stabilizability.
18
Let K = 12B
TP−1 where P > satisfies the Lyapunov matrix inequality above.
AP + PAT −BBT = (A− 1
2BBTP−1)P + P (A− 1
2BBTP−1)T = (A−BK)P + P (A−BK)T < 0.
Multiplying on the left and right by Q = P−1, we get a Lyapunov inequality hence, (A−BK) mustbe stable. Thus, for stabilizable systems, there exists a feedback law u = −Kx which asymptoticallystabilizes the system.
8 Observability
We show that the observability problem is least squares problem. We also give a canonical Kalmandecomposition of a linear system into controllable/uncontrollable and observable/unobservableparts.
Definition 8.1. Consider a linear system x = Ax+Bu and y = Cx+Du. We say that a systemis observable if it is possible to uniquely determine the state x(t) from the knowledge of the outputy(t) and the input u(t).
For simplicity, assume that D = 0 and y = Cx. Thus, taking derivatives on both sides of theoutput equation,
y = Cx
d
dty = C
d
dtx = CAx+ CBu
.
.
(d
dt)n−1y = C(
d
dt)n−1x = CAn−1x+ CAn−2Bu+ . . .+ CB(
d
dt)n−2u
yddty..
( ddt)n−1y
=
CCA..
CAn−1
x+
0 . . . 0 0CB . . . 0 0. . . . . .. . . . . .
CAn−2B . . . CB 0
uddtu..
( ddt)n−1u
(47)
Let O(C,A) :=
CCA..
CAn−1
be the Observability matrix and let
z :=
yddty..
( ddt)n−1y
−
0 . . . 0 0CB . . . 0 0. . . . . .. . . . . .
CAn−2B . . . CB 0
uddtu..
( ddt)n−1u
Thus, we have a linear equation O(C,A)x = z. Observe that this is a Least squares problem.Observe that by construction, z ∈ Im(O(C,A)). Thus, this system of linear equations has asolution. The solution is unique ⇔ O(C,A) has full column rank. Thus, a system is observable ⇔O(C,A) has full column rank.
Assuming that O(C,A) has full column rank, O(C,A)TO(C,A)x = O(C,A)Tz andx = (O(C,A)TO(C,A))−1O(C,A)Tz.
19
Lemma 8.2. O(C,A) is observable ⇔ (AT, CT) is controllable
Proof. Follows from the rank test above for observability and the corresponding rank test forcontrollability.
Observe that if O(C,A) is not full column rank, then it is not possible to uniquely determinex which implies unobservability. Ker(O(C,A)) = ker(C)∩ker(CA) ∩ . . .∩ker(CAn−1) forms unob-servable states. If a state x is unobservable, then Cx = CAx = . . . = CAn−1x = 0. Hence, byCayley-Hamilton theorem, CAix = 0 for all i ∈ N∪0. Thus, if O = ker(O(C,A)) is the unobservablesubspace, then AO ⊆ O. Therefore, the unobservable subspace is A−invariant.
Consider a following equation for y with D = 0 (for simplicity)
y(t) = Cx(t) = CeAtx(0) +
ˆ t
0CeA(t−τ)Bu(τ)dτ
⇒ˆ T
0eA
TtCTy(t)dt =
ˆ T
0eA
TtCTCeAtdtx(0) +
ˆ T
0eA
TtCT
ˆ t
0CeA(t−τ)Bu(τ)dτdt. (48)
Define WT (C,A) =´ T
0 eATtCTCeAtdt as the Observability Gramian. Thus, from the previous
equation, one can uniquely determine x(0) from the knowledge of outputs and inputs ⇔ WT (C,A)is positive definite (hence invertible). From x(0), one can determine x(t) using the closed formexpression for x(t).
Theorem 8.3. Following statements are equivalent.
1. (C,A) is observable.
2. O(C,A) has full column rank equal to n (rank test).
3. The Observability Gramian is positive definite.
4.
[C
λI −A
]has full column rank for all λ ∈ C (PBH).
Proof. 1⇔ 2 and 1⇔ 3 is already proved above. 2⇔ 4 follows using the same arguments used toprove the corresponding equivalence for controllability.
Note that in presence of noise, taking derivatives of the output would increase the effect ofnoise. Determining the state using the Observability Gramian would work better in the presenceof noise. Moreover, it works for LTV too.
Corollary 8.4. For LTV systems, (C(t), A(t)) is observable ⇔´ tt0
ΦT(t, τ)CT(t)C(t)Φ(t, τ)dτ > 0for some t > t0.
Theorem 8.5. Observability is invariant under a change of basis.
Theorem 8.6. (C,A) is observable ⇔ (C,A+KC) is observable.
Proof. Follows from the PBH test.
Theorem 8.7. (C,A) is observable ⇔ for every symmetric set Λ of complex numbers, there existsa feedback matrix K such that eigenvalues of (A+KC) are given by Λ.
Proof. Follows from (C,A), (AT, CT) duality (Lemma 8.2) and the corresponding controllabilitytheorem for pole placement.
20
Thus, one use a feedback matrix such that all eigenvalues of the system matrix lie in the LHPi.e. (A−KC) has eigenvalues in the LHP.
Recall that WT (C,A) =´ T
0 eATtCTCeAtdt.
y(t) = CeAtx(0) +
ˆ t
0CeA(t−τ)Bu(τ)dτ = CeAtx(0) + CeAt
ˆ t
0e−AτBu(τ)dτ
therefore, yT(t) = xT(0)eATtCT + (
ˆ t
0uT(τ)BTe−A
Tτdτ)eATtCT
Let z(t) =´ t
0 e−AτBu(τ)dτ . Thus, y(t) = CeAtx(0) + CeAtz(t) and yT(t) = xT(0)eA
TtCT +
zT(t)eATtCT.
⇒ yT(t)y(t) = xT(0)eATtCTCeAtx(0) + zT(t)eA
TtCTCeAtx(0)
+xT(0)eATtCTCeAtz(t) + zT(t)eA
TtCTCeAtz(t)
⇒ˆ T
0yT(t)y(t)dt =
ˆ T
0(xT(0)eA
TtCTCeAtx(0) + 2zT(t)eATtCTCeAtx(0) + zT(t)eA
TtCTCeAtz(t))dt
= xT(0)WT (C,A)x(0) +
ˆ T
0(2zT(t)eA
TtCTCeAtx(0) + zT(t)eATtCTCeAtz(t))dt(49)
Note that ddTWT (C,A) = eA
TtCTCeAt. Therefore,
ˆ T
0yT(t)y(t)dt = xT(0)WT (C,A)x(0) +
ˆ T
0(2zT(t)
d
dTWT (C,A)x(0) + zT(t)
d
dTWT (C,A)z(t))dt(50)
This gives as expression for the output energy in terms of the Observability Gramian.Suppose all eigenvalues of A lie strictly in the LHP. Thus, one can show using similar arguments
used for the Controllability Gramian thatW∞(C,A) satisfies the Lyapunov equationATW∞(C,A)+W∞(C,A)A = −CTC.
Theorem 8.8. Suppose all eigenvalues of A lie strictly in the LHP. Then, (C,A) is observable ⇔there exists a unique positive definite solution W to the Lyapunov equation ATW +WA = −CTC.
Proof. Follows from Theorem 7.14 and Lemma 8.2.
Suppose (C,A) is not observable. We saw that the unobservable subspace is A−invariant andbelongs to the kernel of C. Choose a basis for the unobservable subspace and extend it to a basisfor the entire state space. By arranging the basis vectors such that vectors in the kernel of C areplaced after the vectors which are not in the kernel of C, we get a following decomposition for Aand C ,
A =
[A1 0A21 A2
], C =
[C1 0
].
Note that (C1, A1) is observable.Detectability: This property is dual of the stabilizability property just like observability is dualto controllability.
Definition 8.9. A system x = Ax+ Bu y = Cx+Du is said to be detectable if the unobservablestates converge to zero as t→∞.
21
Observe that (C,A) is detectable ⇔ (AT, CT) is stabilizable. Thus one can show that (C,A) isdetectable ⇔ unobservable eigenvalues of A lie in the LHP. Moreover, there is an analogous PBHtest for detectability and an analogous Lyapunov test for detectability.Kalman decomposition: Let T = [Vc Vc] where columns of Vc form a basis for the controllablesubspace and column of Vc form uncontrollable states. With T as a similarly transform, it leads tothe following form
A =
[Ac A∗0 Ac
], B =
[Bc0
], C =
[Cc Cc
].
Let x =
[xcxc
]. Thus,
xc = Acxc +A∗xc +Bcu(t)
xc = Acxc
Now further subdivide both Vc and Vc into observable and unobservable states i.e. Vc = [Vco Vco]
and Vc = [Vco Vco]. Thus, T = [Vco Vco Vco Vco] and x =
xcoxcoxcoxco
.
With this similarity transform, recalling the observability decomposition,
xc =
[xcoxco
]=
[Aco 0Ac∗ Aco
] [xcoxco
]+
[A∗o 0A∗∗ A∗o
] [xcoxco
]+
[BcoBco
]u
xc =
[xcoxco
]=
[Aco 0Ac∗ Aco
] [xcoxco
](51)
Thus, we have a following form for (A,B,C),
A =
Aco 0 A∗o 0Ac∗ Aco A∗∗ A∗o0 0 Aco 00 0 Ac∗ Aco
, B =
BcoBco00
, C =[Cco 0 Cco 0
]. (52)
It follows from the Kalman decomposition that the transfer function of a system only captures thecontrollable and observable part of the system. This can be verified by substituting (A,B,C) tripleabove in G(s) = C(sI −A)−1B. Thus, G(s) = C(sI −A)−1B = G(s) = Cco(sI −Aco)−1Bco.
9 State estimation
The feedback law u = −Kx can not be implemented when the state x is not directly accessible.For observable systems, one can uniquely determine the state vector. However, to implement afeedback law, we need x(t) for all time steps. Building an observer which estimates the state vectorasymptotically at each time step lets one implement a feedback law. If (C,A) is unobservable butdetectable, it is possible to estimate x from the system output up to an error e→ 0 as t→∞.
Consider a following model for an observer
˙x = Ax+Bu, e = x− xe = x− ˙x = A(x− x) = Ae
22
Thus, if A is asymptotically stable, then the open loop system above estimates states with e→ 0.When A is not stable, it is still possible to estimate x by using a closed loop estimator using anoutput feedback as follows:
˙x = Ax+Bu+ L(y − y), y = Cx, e = x− x (53)
e = x− ˙x = A(x− x)− LC(x− x) = (A− LC)e (54)
Theorem 9.1. If there exists L such that A − LC has eigenvalues in the LHP, then the stateestimation error converges to 0 exponentially for all input signals u.
Proof. Follows from the equations above.
For observable and or detectable systems, it is always possible to choose L such that eigenvaluesof (A− LC) lie strictly in the LHP.
Suppose we want to implement a state feedback law using state estimates x i.e. u = −Kx.
x = Ax+Bu = Ax−BKx = Ax−BK(x− e) = (A−BK)x+BKe (55)
Thus, from Equations (54) and (55),[xe
]=
[A−BK BK
0 A− LC
] [xe
]( Separation Principle) (56)
The above equation is known as separation principal where one can design a stable feedback gainK and an output injection gain L independently such that the closed loop system is stable and thestate estimation error converges to zero. (Eigenvalues of the matrix above are given by the unionof eigenvalues of (A − BK) and (A − LC). If (A,B) is controllable and (C,A) is observable, onecan choose K and L such that both x and e converge to zero at infinity. Moreover, if (A,B) isstabilizable and (C,A) is detectable, one can still construct an observer such that x and e convergeto zero at infinity.) This is also called stabilization through output feedback. This determines theobserver design to implement state feedback laws.
The dynamics of the state estimate x under the feedback u = −Kx are as follows:
˙x = Ax+Bu+ LC(x− x) = Ax−BKx+ LC(x− x) = (A− LC −BK)x+ LCx (57)
This is called a full order observer.Reduced order observer: The observer used above is called a full order observer. Note that theoutput y = Cx (assuming D = 0) already contains some information about the state. Thus, onecan use this fact and build a reduced order observer or a minimal order observer. Suppose by achange of basis, (A,B,C) can be brought into a following form
A =
[A11 A12
A21 A22
], B =
[B1
B2
], C =
[I 0
]Let (A, B, C) be a matrix representation in the old basis. Thus, A = T−1AT , B = T−1B andC = CT . Let T = [T1 T2]. Thus [CT1 CT2] = [I 0]. (Therefore, T1 is the right inverse of C
and T2 is orthogonal complement of rows of C.) Let x =
[x1
x2
]be partition of x according to the
partition [T1 T2]. Observe that y = [I 0]x = x1. Thus, we need to estimate x2.
x1 = A11x1 +A12x2 +B1u⇒ A12x2 = x1 −A11x1 −B1u (58)
x2 = A21x1 +A22x2 +B2u (59)
23
Since y = x1, x1 is observable. Thus, from Equation (58), A12x2 is observable. Define a stateestimator x2 for x2 as follows
˙x2 = A21x1 +A22x2 + L(x1 −A11x1 −B1u−A12x2) +B2u = A21x1 +A22x2 + LA12(x2 − x2) +B2u(60)
⇒ x2 − ˙x2 = A22(x2 − x2)− LA12(x2 − x2) = (A22 − LA12)(x2 − x2)(61)
We need to show that (A12, A22) form an observable pair. Suppose, (A12, A22) is not observ-
able. Thus, there exists λ such that
[A12
λI −A22
]is rank deficient i.e., there exists v such that
[A12
λI −A22
]v = 0. Note that
[C
λI −A
]=
I 0
λI1 −A11 −A12
−A21 λI2 −A22
. Hence,
[C
λI −A
] [0v
]=
I 0
λI1 −A11 −A12
−A21 λI2 −A22
[ 0v
]= 0 which contradicts the observability of (C,A). Hence, (A12, A22)
is observable. Thus, we can design L such that A22 − LA12 is stable and Tx = T
[x1
x2
]gives an
asymptotic state estimate in the old basis. This gives a minimal order observer.
10 Feedback
Feedback can be an output feedback or a state feedback for state space models. Moreover, it canbe static or dynamic. Static feedback does not involve any dynamics. Note that with a staticoutput feedback u = Ky = KCx, due to the presence of the matrix C, one cannot do an arbitrarypole placement. This is a limitation of static output feedback. However, using an output feedbackand (A,B,C,D) matrices, one can build an observer/state estimator (observer based controller)as shown above which asymptotically estimates states and allows us to implement state feedbacklaws/arbitrary pole placement with an error converging to zero. Note that static feedback can beidentified with P (proportional) controller.
Dynamic feedback controller involves dynamics in its system model. Assuming that (A,B)is stabilizable and (C,A) is detectable (where x = Ax + Bu, y = Cx), one can design a lineardynamic output feedback controller
z = Fz +Gy, u = Lz +My (62)
such that the closed loop matrix [A+BMC BL
GC F
]is Hurwitz. Note that if (A,B) is uncontrollable, then the above dynamic output feedback controller
won’t stabilize the system as we can bring A in block upper triangular form
[A1 A12
0 A2
]and
B =
[B1
0
]. Using the dynamic output feedback, only eigenvalues of A1 can be altered. Thus,
if the uncontrollable eigenvalues are unstable, one cannot stabilize the system by either state oroutput feedback. Moreover, if y = Cx + Du, then let y = y −Du = Cx. Let y be the input tothe dynamic controller and the above arguments work.
With a dynamic output feedback, if (C,A) is observable, then one can reconstruct states x(t)using an observer based controller. Thus, for controllable and observable systems, one can doarbitrary pole placement using a dynamic output feedback. (For input output models, proportional,integral and derivative feedback (P, I, D, PI, PID) is used based on the observed output to get the
24
desired characteristics.) Note that integral control where u =´y is an example of dynamic output
feedback. The controller equations are z = y and u = z. For derivative control, u = y which alsoforms a dynamic output feedback.
For controllable systems, with a static state feedback, one can do arbitrary pole placement(assuming that states are accessible i.e., y = x). Dynamic state feedback can be understood usingdynamic output feedback via the relation y = x.Derivative feedback and DAEs: Dynamic output/state feedback involving derivatives of out-puts/states turns the system into an differential algebraic system as shown in the following example.
Example 10.1. Consider a double integrator[x1
x2
]=
[0 10 0
] [x1
x2
]+
[01
]u
y =[
0 1] [ x1
x2
].
Suppose u = x1 + v which is a dynamic state feedback. Thus, x2 − x1 = v. We can write the stateequations as follows: [
1 0−1 1
] [x1
x2
]=
[0 10 0
] [x1
x2
]+
[01
]v.
This is of the for Ex = Ax + Bu. In this example E is invertible hence, using E−1, one canbring this to standard state space form. However, E may not always be invertible. Such systemsare called differential algebraic systems. The solutions of these systems involve use of impulses ordistributions (Dirac delta functions and its derivatives in a distributional sense).
Consider a derivative control u = Ky = KCx. Therefore, x = Ax+BKCx⇒ (I −BKC)x =Ax. Suppose B = e1, C = eT1 and K = 1. Then, I − BKC is not invertible and one obtains a
differential algebraic system. Suppose (A,B) is uncontrollable and
[A1 A12
0 A2
], B =
[B1
0
]. Let
u = Ky = KCx+ v. Therefore,
x =
[A1 A12
0 A2
]x+
[B1
0
](KCx+ v)⇒ (I −
[B1
0
] [KC1 KC2
])x = Ax+Bv
⇒[I1 −B1KC1 −B1KC2
0 I2
]x =
[A1 A12
0 A2
]x+
[B1
0
]v.
Thus, the controllability and stabilizability properties do not change under the derivative feedbackand the system may turn into DAE. 1
To model transformers, we need to use differential algebraic equations (DAEs) as there is analgebraic relation between the primary and secondary currents/voltages. DAEs arise quite naturallyin many practical systems. Dynamic output/state feedback involving derivatives of outputs/statesis not used in general as it may make the system more complicated.
To implement feedback laws using frequency domain methods, loop shaping techniques areused where one uses appropriate blocks of transfer matrices to get the desired closed loop transferfunction. Loop shaping techniques are applicable for both feedback and feed forward control.
11 Well-posed interconnection
Consider an interconnection of a system and a controller with following dynamics
xs = A1xs +B1u1, ys = C1xs +D1u1 (63)
xc = A2xc +B2u2, yc = C2xc +D2u2. (64)
1Thanks to Pragada Shivaramkrishna for this issue.
25
The interconnection implies that u2 = ys and u1 = yc. Therefore, one obtains
u2 = ys = C1xs +D1u1 = C1xs +D1(C2xc +D2u2) = C1xs +D1C2xc +D1D2u2
⇒ u2 = (I −D1D2)−1(C1xs +D1C2xc) (65)
provided that (I −D1D2)−1 exists. Notice that
u1 = yc = C2xc +D2u2 = C2xc +D2(C1xs +D1u1) = C2xc +D2C1xs +D2D1u1
⇒ u1 = (I −D2D1)−1(C2xc +D2C1xs) (66)
provided that (I −D2D1)−1 exists.Observe that (I−D1D2)−1 exists⇔ (I−D2D1)−1 exists. Suppose (I−D1D2)−1 does not exist.
Hence, there exists v 6= 0 such that (I −D1D2)v = 0 i.e., v = D1D2v. Therefore, D2v = D2D1D2vi.e, (I−D2D1)D2v = 0 and D2v 6= 0 because if D2v = 0, then (I−D1D2)v 6= 0. Thus, (I−D2D1)is not invertible. Similarly, one can show the converse as well. An interconnection is said to bewell posed if (I − D1D2)−1 exists. This indicates that u1 and u2 are uniquely determined. Thedynamics of the interconnected system is given by[
xsxc
]=
[A1 +B1(I −D2D1)−1D2C1 B1(I −D2D1)−1C2
B2(I −D1D2)−1C1 A2 +B2(I −D1D2)−1D1C2
] [xsxc
].
An interconnection which is not well posed is called ill posed. It turns out that if either the systemor the controller is strictly proper (either D1 = 0 or D2 = 0), then the interconnection is alwayswell posed. It follows that interconnection of the system with an observer based controller is wellposed since D2 = 0.
12 Applications/Design examples
Example 12.1 (Feedback linearization and a stabilizing control law). Consider an oscillatingpendulum θ + g
l sin(θ) = 0. Find a stabilizing control law u for the system θ + gl sin(θ) = u which
drives the pendulum from an arbitrary initial position with an arbitrary initial velocity to θ = 0 andθ = 0 asymptotically.
Observe that introducing state variables x1 = θ, x2 = θ, this can be converted into a first ordersystem.
x1 = x2, x2 = −gl
sin(θ) + u
(If we consider θ as a variable of interest, this system can be modeled as a first order system wherethe order of the ode is equal to the number of energy storing elements which is one (mass) in thiscase.)
Using u to for feedback linearization, choose u = gl sin(θ) + v. Thus we have the following linear
system:
˙[x1
x2
]=
[0 10 0
] [x1
x2
]+
[0v
]Clearly, the above system is controllable (by the rank test). Thus, an arbitrary pole placement ispossible. Choose v = k1x1 + k2x2. Thus we have,
˙[x1
x2
]=
[0 1k1 k2
] [x1
x2
]Now choose k1, k2 such that the eigenvalues of the feedback matrix lie in the LHP.
26
One can consider other practical examples of non-linear systems too which are feedback lin-earizable and controllable after feedback linearization. Find out such systems in robotics, powerelectronics or systems in other engineering domains and find appropriate control laws for theirdesigns. Build observers for these systems to implement feedback laws. Do simulations for thesemodels.
Example 12.2 (Tracking). Consider a SISO system (A, b, c) which is both controllable and observ-able. Find u such that y tracks a constant reference signal which is unit step. Let u = −Kx + vsuch that v(t) = 1 for t ≥ 0. Thus,
y(t) = Ce(A−BK)tx(0) +
ˆ t
0Ce(A−BK)(t−τ)Bv(τ)dτ = Ce(A−BK)tx(0) +
ˆ t
0Ce(A−BK)(t−τ)Bdτ
limt→∞y(t) = limt→∞{Ce(A−BK)tx(0) + Ce(A−BK)t(
ˆ t
0e−(A−BK)(τ)dτ)B}
Since (A − BK) has eigenvalues strictly in the LHP, it is invertible. Hence,´ t
0 e−(A−BK)(τ)dτ =
−(A−BK)−1(e−(A−BK)t − I). Thus,
limt→∞y(t) = limt→∞{−C(A−BK)−1e(A−BK)t(e−(A−BK)t − I)B}= limt→∞{−C(A−BK)−1(I − e(A−BK)t)B}= −c(A− bK)−1b (67)
Thus, we normalize v by the factor above so that y(t) tracks the unit step signal. (Observe thatthe transfer function for the closed loop system is G(s) = c(sI − A + bK)−1b. Thus G(0) =−c(A− bK)−1b.)
Example 12.3 (Feedback linearization and tracking). Consider Example 1. Suppose y(t) = θ(t).Suppose we want to design a feedback law such that θ(t) tracks the position θ = π and θ = 0.Then one can use feedback linearization to convert the non-linear system to a linear system whichis controllable. Then one can use Example 2 to design an appropriate input signal.
One can combine the above methods to obtain methods which will give a strategy to trackreference signals for more general feedback linearizable non-linear systems. Construct/find suchexamples in applications. One can consider an example of robot arm manipulator which is feedbacklinearizable. One can have a network of single input systems where each system could be linear orfeedback linearizable. Then one can choose an appropriate input for each of these systems.
Example 12.4. Suppose we want to build an observer to implemented state feedback laws. If theoutput is a non-linear function of states, linear observers won’t work. If the output is a linearfunction of states, still there are difficulties due to non-linearities in the state equation and oneneeds to use techniques from non linear control theory. However, there are feedback linearizablesystems for which it is possible to build linear observers to implement state feedback laws. They areof the following form
x1 = x2
.
.
xn−1 = xn
xn = f(x1, . . . , xn) + u
y = x1.
Thus, all states can be derived from the output equation. Thus, one can implement state feedbacklaws for such systems. Find more example of systems where it is possible to build an observer usingtools from linear systems theory.
27
13 Disturbance decoupling problem
Consider a following model for LTI system where q represents either disturbance or noise (Wonham).
x = Ax+Bu+ Sq (68)
y = Cx (69)
We say that a system in the form above is disturbance decoupled w.r.t. the pair (q, y) if for eachinitial state x(0) ∈ X, the output y(t) is the same for every q. Note that with a state feedbacku = Kx,
x = (A+BK)x+ Sq ⇒ x(t) = e(A+BK)tx(0) +
ˆ t
0e(A+BK)(t−τ)Sq(τ)dτ
⇒ y(t) = Ce(A+BK)tx(0) + C
ˆ t
0e(A+BK)(t−τ)Sq(τ)dτ
Hence, disturbance decoupling means that find u = Kx such that the forced response yf (t) =
C´ t
0 e(A+BK)(t−τ)Sq(τ)dτ = 0. In other words, we want to find K such that the transfer function
from q to y is identically zero. Thus, we want T (s) = Y (s)Q(s) = C(sI − (A+BK))−1S = 0. Let K =
Ker(C) and S = Im(S). Let V =< A+BK,S > be a subspace generated by columns of S underthe action of A+BK. It is clear from Cayley-Hamilton theorem that
´ t0 e
(A+BK)(t−τ)Sq(τ)dτ liesin the subspace < A+BK,S >. This is called disturbance decoupling problem (DDP). Thus DDPis solved ⇔ there exists K such that < A+BK,S >= V ⊂K.
Observe that V is (A+BK)−invariant from construction.
Definition 13.1. (Wonham) A subspace W ⊂ X is said to be (A,B)−invariant, if there exists amatrix K such that (A + BK)W ⊂ W. We denote the class of (A,B)−invariant subspaces of X
by γ(A,B,X).
It is clear that all A−invariant subspaces are (A,B)−invariant by choosing K = 0.
Lemma 13.2. Let W ⊂ X and B = Im(B). Then W ∈ γ(A,B,X) ⇔ AW ⊂W + B.
Proof. (⇒) Suppose W ∈ γ(A,B,X). Therefore, there exists K such that (A+BK)W ∈W. Letw1 ∈W. Therefore, (A+BK)w1 = w2 ∈W ⇒ Aw1 = w2 −BKw1 ∈W + B.(⇐) Suppose AW ⊂W + B. Let {w1, . . . , wl} be a basis for W. Therefore, for each wi 1 ≤ i ≤ l,there exists vi ∈ W and ηi such that Awi = vi − Bηi. Define a linear map K on W such thatKwi = ηi (1 ≤ i ≤ l). Extending the basis for W to a basis for X, we can extend K to a linearmap K on X. Then (A + BK)wi = Awi + BKwi = vi − Bηi + Bηi ∈W. Thus, W is (A + BK)invariant.
Lemma 13.3. The class of subspaces γ(A,B,X) is closed under subspace addition.
Proof. Follows from the above lemma.
Lemma 13.4. The class of subspaces γ(A,B,K) has a maximal element V∗.
Proof. Let W1,W2 ∈ γ(A,B,K). Then, by the previous lemma, W1 + W2 ∈ γ(A,B,K). ConsiderW3 ∈ γ(A,B,K) and obtain W1 + W2 + W3 ∈ γ(A,B,K) such that W1 + W2 ⊂ W1 + W2 + W3.Continuing this procedure, we obtain a chain of subspaces of higher dimensions. Since, K is finitedimensional, this process terminates after finite steps and we no longer generate a subspace ofhigher dimension. This gives a maximal element V∗.
Theorem 13.5. DDP is solvable ⇔ < A+BK,S >= V ⊆V∗ = sup(γ(A,B,K)).
Proof. (⇒) DDP is solvable ⇒ there exists K such that < A + BK,S >= V ⊂ K ⇒ < A +BK,S >= V ⊆V∗ = sup(γ(A,B,K)). (⇐) Obvious.
28
13.1 Output stabilization problem (OSP)
x = Ax+Bu, y = Cx
Find if possible a feedback matrix K such that y(t) = Ce(A+BK)t → 0 as t → ∞. From theprevious DDP problem, find V∗ = sup(γ(A,B,Ker(C) = K)). Let (A + BF )V∗ ⊆ V∗. If allunstable modes of (A + BF,B) lie in the kernel of C, then this solves OSP by choosing u = Fx.Consider the case when there are unstable modes of A+BF lying outside ker(C). If these unstablemodes are controllable, then they can be made to lie in the LHP by an appropriate state feedbacki.e. for x = (A+BF )x+Bv, use v = Kx such unstable (but controllable) eigenvalues of (A+BF )are shifted to the LHP. Thus, OSP is solved. Thus, OSP is solvable ⇔ (unstable modes of A)⊆ < A,B > +V∗. (Union of subspaces is not a subspace in general. Hence we take the sum ofsubspaces to get a bigger subspace containing both the smaller subspaces.)
Uncontrollable modes of A are unaffected under state feedback. If they lie in the kernel of C orare stable, OSP can be solved by shifting all controllable modes to the LHP. If there are unstableuncontrollable modes of A lying outside the kernel of C, then OSP is not solvable.
13.2 Controllability subspaces
Recall that we had seen different canonical forms of (A,B) pairs by generating cyclic subspacesusing columns of B and so on (discussion after Theorem 6.11).
Definition 13.6. Let B1 ⊆ B where B is a subspace generated by column span of B and B1 isa subspace of B. A subspace C(A,B1) generated by the action of A on B1 is called controllabilitysubspace.
Observe that by construction, controllability subspaces are A−invariant. Thus when A is re-stricted to C(A,B1), arbitrary pole placement is possible for restriction of A on this subspace. Acollection of controllability subspaces forming a class of controllability subspaces of the state spaceX is denoted by C(A,B,X). By choosing elements in B, we can generate different controllabilitysubspaces. It is clear from the construction of controllability subspaces that the class C(A,B,X) isclosed under subspace addition (just like the class γ(A,B,X) of (A,B)−invariant subspaces) andit contains a unique maximal element.
Remark 13.7. Consider a controllability subspace C(A,B1) such that the initial condition x0 ∈C(A,B1). Let {b1, . . . , bk} be a basis for B1. Let B1 = [b1, . . . , bk]. Taking action of A on thisset and taking only linearly independent vectors, generate a basis for C(A,B1) and extend it to abasis of X. Let A1 be the restriction of A to C(A,B1). Clearly, (A1, B1) is controllable and usinga feedback F1, we can do arbitrary pole placement for (A1 +B1F1, B1). Note that (A,B1) need notbe controllable and w.r.t. the chosen basis for X, we have an upper triangular decomposition of(A,B1) into the controllable and uncontrollable part. Since the initial condition x0 ∈ C(A,B1), thecomponent of the intial condition in uncontrollable part of the state space is zero. This implies thatthe evolution of x(t) remains inside the controllable subspace C(A,B1) if x0 ∈ C(A,B1). This isan important property of controllability subspaces.
DDP with stability (DDPS): We want to solve DDP using a state feedback such that thefeedback matrix is stable. For an (A,B)−invariant subspace W, we know from the definition thatthere exists K such that (A + BK)W ⊂ W. Let F(W) denote the set of matrices K such that(A+BK) has eigenvalues strictly in the LHP. Consider a class Γ(A,B,K) ⊂ γ(A,B,K) such thatif W ∈ Γ(A,B,K), then F(W) 6= ∅. Thus, we have the following natural analogue of the DDPresult.
Theorem 13.8. DDPS is solvable ⇔ < A+BK,S >= V ⊆V∗ = sup(Γ(A,B,K)).
29
14 Markov parameters, Minimal realizations, Transfer functions
Note that (A−KC,B) need not be controllable and (C,A−BF ) need not be observable in general.
Definition 14.1. A realization (A,B,C,D) of a transfer function G(s) (i.e. C(sI−A)−1B+D =G(s)) is said to be minimal if there does not exist a realization of a smaller order.
Definition 14.2. Two state space systems are said to be zero state equivalent if they realize thesame transfer function.
Observe that (sI − A)−1 = L(eAt) = L{∑∞
i=0ti
i!Ai}. Note that L( t
i
i! ) = s−(i+1). Therefore,
(sI −A)−1 =∑∞
i=0 s−(i+1)Ai. Hence, G(s) = C(sI −A)−1B +D = D +
∑∞i=0 s
−(i+1)CAiB.
Definition 14.3. CAiB for i ≥ 0 are called Markov parameters of a realization (A,B,C,D).
Observe that the impulse response g(t) is given by
g(t) = L−1(G(s)) = CeAtB +Dδ(t),d
dtg(t) = CAeAtB +
d
dtDδ(t), . . .
di
dtig(t) = CAieAtB +
di
dtiDδ(t)
Thus, taking the derivatives from the right as t→ 0+, g(0+) = CB, ddtg(t)|t=0+ = CAB and so on.
Theorem 14.4. Two realizations (A,B,C,D) and (A, B, C, D) are zero state equivalent ⇔ theyhave the same Markov parameters and D = D.
Proof. (⇒) Zero state equivalence implies that D = D. Moreover, since G(s) = G(s), Markovparameters must be the same. Conversely, if two realizations have the same Markov parametersand D = D, then they realize the same transfer function hence, are zero state equivalent.
Theorem 14.5. A realization is minimal ⇔ it is controllable and observable.
Proof. (⇒) Suppose a minimal realization is either uncontrollable or unobservable. Thus, byKalman decomposition, one can find a smaller order realization that realizes the same transferfunction. This contradicts minimality.
(⇐) Suppose a realization (A,B,C,D) is controllable and observable but not minimal. Let(A, B, C, D) be a realization of order n < n. Let C = C(A,B) and O = O(C,A) be the control-lability and observability matrices of (A,B,C,D) and let C = C(A, B) and O = O(C, A) be thecontrollability and observability matrices of (A, B, C, D). Note that since both C and O have rankn, OTOCCT has rank n since it is a product of two positive definite matrices. Therefore, OC hasrank n because if the rank of OC is less than n, then OTOCCT will not have rank equal to n.Similarly, OC has rank n < n.
OC =
CB CAB . . . CAn−1BCAB CA2B . . . CAnB. . . . . .. . . . . .
CAn−1B CAnB . . . CA2n−2B
, OC =
CB CAB . . . CAn−1BCAB CA2B . . . CAnB. . . . . .. . . . . .
CAn−1B CAnB . . . CA2n−2B
.
Since (A,B,C,D) and (A, B, C, D) realize the same transfer function, they have the same Markovparameters. Therefore, OC = OC. But rank(OC) = n < n = rank(OC), which is a contradiction.Therefore, the realization (A,B,C,D) is minimal.
Corollary 14.6. (A−KC,B) is controllable⇔ (C,A−KC,B) is minimal. Similarly, (C,A−BF )is observable ⇔ (C,A−BF,B) is minimal.
Proof. Follows from the theorem above.
30
Corollary 14.7. OC remains invariant under a change of basis.
Proof. By the proof previous theorem, OC = OC where O and C are observability and controlla-bility matrices respectively in a different basis.
Observe that the controllability/observability matrix and the corresponding Gramian matricesdo not remain invariant under a change of basis. However, their rank remains invariant.
Suppose a matrix A is full column rank. Then ATA is positive definite, hence invertible. LetAl = (ATA)−1AT. Then AlA = I and Al is called the left inverse of A. Similarly, if A is full rowrank, then AAT > 0 and Ar = AT(AAT)−1 is called the right inverse of A i.e. AAr = I.
Theorem 14.8. All minimal realizations of a transfer function are algebraically equivalent.
Proof. Let (A,B,C,D) and (A, B, C, D) be two minimal realizations. By the previous theorem,they are both controllable and observable and OC = OC. Note that C and C have full row rankand O, O have full column rank. Let T = CCr.
OC = OC ⇒ OCCr = OCCr ⇒ OlO = (OlO)(CCr) = I ⇒ T−1 = (OlO) (70)
OlOC = C = OlOC = T−1C (71)
OCCr = O = OCCr = OT. (72)
Thus, from previous two equations, T−1B = B and CT = C. Since OC = OC and both realizationshave the same Markov parameters, OAC = OAC ⇒ OlOACCr = (OlO)A(CCr). This impliesthat A = T−1AT . Thus, T provides a similarity transform between the two realizations and D = Dsince they realize the same transfer function.
System norms: Substituting s = jω in the transfer function G(s), we obtain the frequencyresponse for an LTI system. The 2−norm is given by
‖G‖2 := (
ˆ ∞−∞‖G(jω)‖2Fdω)
12 = (
ˆ ∞−∞
trace(G(jω)G∗(jω))dω)12 (73)
If G(s) is stable, then from Parseval’s theorem for Fourier transforms,
‖G‖2 = (
ˆ ∞−∞‖g(t)‖2Fdt)
12 ) = (
ˆ ∞−∞
trace(CeAtBBTeATtCT)dt)
12 . (74)
Similarly, the infinity norm of G is defined as
‖G‖∞ := supωσ1(G(jω)). (75)
where σ1 is the maximum singular value of G(jω). Note that infinity norm is finite ⇔ G(s) hasno poles on the purely imaginary axis. For 2−norm to be finite, apart from the poles condition,G(s) also has to be strictly proper. (We refer the reader to Doyle, Francis and Tannerbaum fordetails.) System norms are useful in robust control to design controllers which minimize the effectof the disturbance on the output by minimizing the norm of the corresponding transfer function(from the disturbance to the output).Matrix-fraction Descriptions (MFDs) Given a matrix transfer function G(s), one can writeG(s) = NR(s)D−1
R (s) or G(s) = D−1L (s)NL(s). This is called right and left matrix fraction respec-
tively and it is not unique. Kailath shows that it is possible to obtain a controllable realization forright MFDs and observable realization for left MFDs. For more on this topic, we refer the readerto Kailath (Chapter 6). MFD is useful for design problems where one wants desired transfer func-tions/loop transfer functions. Using MFDs, one can shape the transfer function of an underlyingsystem by choosing compensators with appropriate MFDs so that one obtains a desired transferfunction.
31
15 Poles-zeros and Smith McMillan form, Transmission zeros, In-variant zeros, System inverse
Refer Hespanha. Applications of Invariant zeros: Modeling, detection and correction of cyberattacks (Refer works of Pasqualetti, Fawzi)G(s) = C(sI − A)−1B + D = 1
pA(s)(CAdj(sI − A)B + DpA(s)). One can write G(s) = 1pA(s)N(s)
and from the Smith form Σ(s) of N(s) where Σ = diag(σi(s)) such that σi|σi+1, we obtain theSmith-McMillan form SMG of G(s), where the nonzero diagonal entries of SMG are ni
ψisuch that
niψi
= σipA
and ni+1
ψi+1= σi+1
pA. Therefore, ni|ni+1 and ψi+1|ψi. Roots of Πini(s) are called zeros of G(s)
and roots of Πiψi(s) are called poles of G(s). The McMillan degree is given by∑
i deg(ψi).
Lemma 15.1. Poles of G(s) ⊆ eigenvalues of A.
Proof. Follows from the fact that G(s) = 1pA(s)(CAdj(sI −A)B +DpA(s)).
For minimal realizations, poles of G(s) = eigenvalues of A.
Theorem 15.2. Let SMG(s) be the Smith-McMillan form for G(s). Then the order n of a minimalrealization of G(s) is equal to the McMillan degree of G(s).
Proof. Let (A,B,C,D) be a minimal realization of G(s). Thus, poles of G(s) are equal to eigen-values of A. Suppose G(s) has n number poles (and by definition this is the McMillan degree).Therefore, the order of a realization must be greater than or equal to n in general for any arbitraryrealization. But for minimal realizations, the order of realization is equal to the number of polessince there are no pole zero cancellations. The number of poles of G(s) is equal to its McMillandegree.
The Rosenbrock system matrix is given by P (s) =
[sI −A B−C D
]. Suppose it has rank r.
Invariant zeros are complex numbers where P (s) loses its rank. Transmission zeros ⊆ Invariantzeros and both of them block certain input signals with a frequency given by transmission/invariantzero and an appropriate initial condition. When Transmission zeros = Invariant zeros?
The Rosenbrock system matrix can be factored as
P (s) =
[sI −A B−C D
]=
[sI −A 0−C I
] [I (sI −A)−1B0 G(s)
]where G(s) is the transfer function. It is clear from this factorization that the rank of P (s) drops attransmission zeros of G(s). Moreover, if there are pole-zero cancellations, then there are eigenvaluesof A which are not poles of G(s). At these complex numbers, P (s) again loses rank. Thus, if thereare no pole-zero cancellations i.e., for minimal realizations, transmission and invariant zeros arethe same.
Example 15.3. Consider the following system
A =
[1 00 2
], B =
[11
], C =
[1 0
], D = 0
P (s) =
s− 1 0 10 s− 2 1−1 0 0
.Rank(P (s)) = 3. At s = 2, P (s) loses its rank. Thus, there is an invariant zero at s = 2. Notethat G(s) = 1
s−1 . Therefore, there are no transmission zeros hence the set of transmission zeros= ∅ ⊂ the set of invariant zeros.
If we choose C =[
1 1]
in the above example, then there are no invariant zeros.
32
Suppose there is a transmission zero at z0 then, G(z0) is rank deficient i.e., there exists u0 suchthat G(z0)u0 = 0. Suppose x0 = (z0I − A)−1Bu0. Let u = ez0tu0. We will see that the output iszero for this input and the intial condition x0.
x(t) = eAt(x0 +
ˆ t
0e−Aτez0τBu0dτ = eAt(x0 +
ˆ t
0e(z0I−A)τdτBu0
= eAt(x0 + (z0I −A)−1(e(z0I−A)t − I)Bu0)
= eAtx0 + eAte−Atez0t(z0I −A)−1Bu0 − eAt(z0I −A)−1Bu0
= eAtx0 + ez0tx0 − eAtx0 = ez0tx0
y(t) = Cez0tx0 = ez0tCx0 = ez0tG(z0)u0 = 0.
Thus, for the initial condition x0 = (z0I −A)−1Bu0, a change in the input u(t) of the form u(t) +ez0tu0 goes undetected at the output. Therefore, malicious attacks made at system transmissionzeros go undetected.
Note that if there is an invariant zero at z0, then P (z0) is rank deficient i.e., there exists x0, u0
such that
P (z0)
[x0
−u0
]=
[zI −A B−C D
] [x0
−u0
]= 0
⇒ (z0I −A)x0 −Bu0 = 0⇒ x0 = (z0I −A)−1Bu0, and − Cx0 −Du0 = 0.
Let u(t) = ez0tu0 and x(0) = x0. Thus, from the arguments seen previously, x(t) = ez0tx0 andy(t) = Cez0tx0 + Dez0tu0 = 0. For a connection between the set of invariant zeros and detectionand correction of cyber attacks, we refer the reader to [11], [12]. Absence of invariant zeros impliesthat attacks are not possible. Attacks can be made on states via B matrix and on the output viaD matrix. We show below that invariant zeros are invariant under feedback.
Consider the Rosenbrock system matrix P (s). By a unimodular transformation,[sI −A B−C D
] [I 0−K I
]=
[sI −A−BK B−C −DK D
]Similarly, one can show that[
I L0 I
] [sI −A B−C D
]=
[sI −A− LC B − LD
−C D
].
Thus, invariant zeros do not change under a state feedback/output feedback as pre and post mul-tiplication by unimodular matrices do not affect invariant zeros of P (s). Note that the Smith formof P (s) gives all invariant zeros of P (s). These are precisely those complex numbers where P (s)loses rank.
Remark 15.4. Observe that the Rosenbrock system matrix P (s) =
[sI −A B−C D
]captures all
information about the underlying linear dynamical system.
• The controllability and stabilizability property can be checked using the PBH test from theblock matrices in the first row of P (s).
• The observability and detectability property can be checked using the PBH test from the blockmatrices in the first column of P (s).
• Invariant zeros are given by the zeros of the Smith form associated with P (s).
33
• Taking Schur complement w.r.t. the block (sI − A), we obtain the transfer function G(s) =C(sI − A)−1B + D from which one can deduce the impulse response, frequency response,Markov parameters, poles and transmission zeros. (From the knowledge of the transfer func-tion, on can draw Nyquist/Bode plots too.)
• The state transition matrix can be found from the inverse Laplace transform of (sI −A)−1.
Definition 15.5. A system (A,B,C,D) with transfer function G(s) is said to be invertible if thereexists a system (A, B, C, D) with a proper transfer function G(s) such that GG = GG = I.
For an inverse to exist, #inputs = #outputs.
Theorem 15.6. A system inverse exists ⇔ D is invertible
Proof. Refer Hespanha. If D is invertible, (A, B, C, D) can be constructed explicitly using D−1 andsystem equations. If D is not invertible, we get a contradiction by choosing v such that Dv = 0.
Theorem 15.7. A system inverse exists ⇔ the Rosenbrock system matrix is invertible for almostall complex numbers.
Proof. Suppose that if z0 is not an eigenvalue of A and not an invariant zero of the Rosenbrocksystem matrix P (s). Therefore,[z0I −A B−C D
]=
[I 0
−C(z0I −A)−1 I
] [z0I −A 0
0 C(z0I −A)−1B +D
] [I (z0I −A)−1B0 I
].
Thus, P (z0) is invertible⇔ G(z0) = C(z0I−A)−1B+D is invertible⇔ D is invertible. Therefore,the statement follows from the previous theorem.
16 LQR/LQG
In this section, we elaborate a little on some topics from Hespanha.
Definition 16.1 (Feedback invariants). A functional H(x, u) that involves system’s input and stateis called a feedback invariant if computed along solution of the system, it depends only on the initialcondition x(0) and not on the input signal u.
16.1 Finite horizon LQR
Proposition 16.2. For every symmetric matrix P (t), the following functional
H(x, u) := xT(t)P (t)x(t)−ˆ t
0(Ax+Bu)TP (t)x+ xTP (t)(Ax+Bu) + xTP (t)xdt
is a feedback invariant.
Proof. H can be rewritten as
H(x, u) = xT(t)P (t)x(t)−ˆ t
0xTPx+ xTPx+ xTP xdt
= xT(t)P (t)x(t)−ˆ t
0
d
dt(xTPx)dt
= xT(t)P (t)x(t) + xT(0)P (0)x(0)− xT(t)P (t)x(t)
Thus, H(x, u) = xT(0)P (0)x(0) along the state trajectory.
34
LQR problem formulation: We want to make states to go the origin in finite time. How-ever, there is a cost attached to input signals and deviation of the state from the origin. This ismathematically formulated as follows:
Minimizeu{JLQR(x, u)} subject to x = Ax+Bu, (76)
where
JLQR(x, u) := xT(tf )F (tf )x(tf ) +
ˆ tf
0xT(t)Qx(t) + uT(t)Ru(t) + 2xT(t)Nu(t)dt (77)
such that Q ≥ 0 and R > 0. Note that the term xT(tf )F (tf )x(tf ) represents the cost associatedwith the deviation of x(tf ) from the origin. The terms inside the integral give cost attached toinputs and deviation of x(t) from the origin along the state trajectory from t = 0 to t = tf .
Using the feedback invariant obtained above, we want to represent JLQR(x, u) as
JLQR(x, u) = H(x, u) +
ˆ tf
0Λ(x, u)dt (78)
where minuΛ(x, u) = 0. In this case, u(t) = argminuΛ(x, u) minimizes JLQR and the minimumcost is equal to H(x, u). Note that there are infinitely many choices for a feedback invariantdepending on P (0) and P (t). We will see below that there is a specific feedback invariant for whichthe corresponding P (t) which satisfies a matrix differential equation (differential Riccati equation(DRE)) which will be our choice of feedback invariant to find minu{JLQR(x, u)}.
JLQR = xT(tf )F (tf )x(tf ) +
ˆ tf
0xT(t)Qx(t) + uT(t)Ru(t) + 2xT(t)Nu(t)dt
= H(x, u) + xT(tf )F (tf )x(tf ) +
ˆ tf
0xT(t)Qx(t) + uT(t)Ru(t) + 2xT(t)Nu(t)dt−H(x, u)
= H(x, u) + xT(tf )F (tf )x(tf )− xT(tf )P (tf )x(tf ) +
ˆ tf
0(xT(t)Qx(t) + uT(t)Ru(t) + 2xT(t)Nu(t) +
(Ax+Bu)TPx+ xTP (Ax+Bu) + xTP x)dt
= H(x, u) + xT(tf )F (tf )x(tf )− xT(tf )P (tf )x(tf ) +ˆ tf
0xT(ATP + PA+Q+ P )x+ uTRu+ 2uT(t)(BTP +NT)xdt (79)
Let K := R−1(BTP +NT).
(uT + xTKT)R(u+Kx) = uTRu+ xT(PB +N)R−1(BTP +NT)x+ 2uT(BTP +NT)x (80)
Adding and subtracting the term xT(PB+N)R−1(BTP +NT)x in Equation (79) and using Equa-tion (80),
JLQR = H(x, u) + xT(tf )F (tf )x(tf )− xT(tf )P (tf )x(tf ) +ˆ tf
0xT(ATP + PA+Q+ P − (PB +N)R−1(BTP +NT))x+ (uT + xTKT)R(u+Kx)dt
If we find P (t) such that ATP (t) + P (t)A+Q+ P (t)− (P (t)B +N)R−1(BTP (t) +NT) = 0 andP (tf ) = F (tf ), then Λ(x, u) = (uT + xTKT)R(u+Kx) which has a minimum equal to zero whenu = −Kx. This leads to a closed loop system x = (A−BR−1(BTP (t) +NT))x. Therefore, choosethe feedback invariant among all possible choices such that P (t) satisfies the differential Riccatiequation with boundary conditions.
35
Theorem 16.3. Assume that there exists a symmetric solution to differential Riccati equationATP (t)+P (t)A+Q+ P (t)−(P (t)B+N)R−1(BTP (t)+NT) = 0 satisfying the boundary conditionP (tf ) = F (tf ).Then the feedback law u(t) = −Kx(t) where K := R−1(BTP + NT) minimizesJLQR(x, u).
Proof. Proof: Follows from the equations above.
If states are not directly accessible, we use a state estimator using an output feedback to im-plement the feedback law of LQR.
Recall that we had considered an optimization problem on linear systems earlier as follows:
minimizeu
ˆ t
0uT(τ)u(τ)dτ
subject to x = Ax+Bu, xf = eAtx(0) +
ˆ t
0eA(t−τ)Bu(τ)dτ. (81)
We solved this optimization problem using tools from linear algebra. Cost functions of the aboveproblem and the LQR problem are different. Substituting Q = 0, F (tf ) = 0, N = 0 and R = I in theLQR problem, we get the same cost function as above but the constraints are still different as thereis an additional constraint on (81). Therefore, clearly the optimal solutions to the two problemsare also different as there are different cost functions and different constraints. Both problems canbe solved independently using two different approaches as we have seen. However, they are closelyrelated to one another. They can be solved using Hamilton-Jacobi-Bellman equations (HJB) fromoptimal control. Note further that for the LQR problem, we only gave sufficient conditions for anoptimal solution. HJB theory says that these conditions are also necessary for an optimal solution.Infinite horizon LQR We refer the reader to Hespanha. The above approach is inspired fromHespanha’s infinite horizon LQR approach.
16.2 Minimum energy estimator (MEE)
If states are not directly available for measurement, we need a state estimator/observer to imple-ment feedback laws for LQR. Moreover, there could be noise and measurement errors. Thus weneed a minimum energy estimator (MEE) using an output feedback. When noise and measure-ment errors are modeled as zero mean white noise Gaussian stochastic processes, we have an LQGproblem (a cousin of LQR). Separation principal holds here as well.
Consider the following model (Hespanha)
x = Ax+Bu+ Bd, y = Cx+ n (82)
where d represents disturbance and n represents measurement noise with both of them unknown.Thus any estimate of x would work for sufficiently large d and n. Thus, previous state estimator orobserver does not work in the presence of disturbance and noise. MEE consists of finding a statetrajectory
˙x = Ax+Bu+ Bd, y = Cx+ n (83)
that starts at rest at t→ −∞ and is consistent with the past measured output y and control inputu for the least amount of noise n and disturbance d, measured by
JMEE :=
ˆ t
−∞(n(τ)TQn(τ) + d(τ)TRd(τ))dτ (84)
36
where Q,R > 0. Once x is found, the minimum energy state estimate x is the most recent value ofx i.e. x(t) = x(t). The MEE problem is solved by minimizing the quadratic cost
JMEE :=
ˆ t
−∞((Cx− y)TQ((Cx− y)) + dTRd)dτ (85)
by appropriately choosing d(.). Note that the dynamic constraint on x is given by Equation (83).
Proposition 16.4. Suppose u(.) and y(.) are given up to some time t > 0. For every symmetricmatrix P , differentiable signal β : (−∞, t] → Rn and a scalar H0 that does not depend on (d, x),the functional
H(x, d) := H0 +
ˆ t
−∞(( ˙x− β)TP (x− β) + (x− β)TP ( ˙x− β))dτ − (x(t)− β(t))TP (x(t)− β(t))(86)
is a feedback invariant for (83) if limτ→−∞(x(τ)− β(τ)) = 0.
Proof.
H(x, d) := H0 +
ˆ t
−∞
d(x(τ)− β(τ))TP (x(τ)− β(τ))
dτdτ − (x(t)− β(t))TP (x(t)− β(t))
= H0
since limτ→−∞(x(τ)− β(τ)) = 0.
Add and subtract the feedback invariant term to JMEE . Thus,
JMEE := H(x, d)−H0 + (x(t)− β(t))TP (x(t)− β(t)) +
ˆ t
−∞((Cx− y)TQ((Cx− y)) + dTRd)dτ
−ˆ t
−∞(( ˙x− β)TP (x− β) + (x− β)TP ( ˙x− β))dτ
= H(x, d)−H0 + (x(t)− β(t))TP (x(t)− β(t)) +
ˆ t
−∞((Cx− y)TQ((Cx− y)) + dTRd)dτ
−ˆ t
−∞((Ax+Bu+ Bd− β)TP (x− β) + (x− β)TP (Ax+Bu+ Bd− β))dτ (87)
= H(x, d)−H0 + (x(t)− β(t))TP (x(t)− β(t)) +
ˆ t
−∞((Cx− y)TQ((Cx− y)) + dTRd)dτ −
ˆ t
−∞[xTATPx+ uTBTPx+ dTBTPx− βTPx− xTATPβ − uTBTPβ − dTBTPβ + βTPβ
+xTPAx+ xTPBu+ xTPBd− xTP β − βTPAx− βTPBu− βTPBd+ βTP β]dτ
= H(x, d)−H0 + (x(t)− β(t))TP (x(t)− β(t)) +
ˆ t
−∞(xTCTQCx+ yTQy − 2xTCTQy + dTRd)dτ
−ˆ t
−∞[xT(ATP + PA)x+ 2xTPBu+ 2dTBTPx− 2xTP β − 2xTATPβ − 2βTPBu−
2dTBTPβ + 2βTP β]dτ
= H(x, d)−H0 + (x(t)− β(t))TP (x(t)− β(t)) +
ˆ t
−∞[xT(−ATP − PA+ CTQC)x+ yTQy +
−2xT(−ATPβ + PBu+ CTQy − P β) + 2βTP (Bu− β) + dTRd− 2dTBTP (x− β)]dτ. (88)
37
Now, we complete the square in terms involving d:
dTRd− 2dTBTP (x− β) = dTRd− 2dTBTP (x− β) + (x− β)TPBR−1BTP (x− β)−(x− β)TPBR−1BTP (x− β)
= dTRd− 2dTRR−1BTP (x− β) + (x− β)TPBR−1BTP (x− β)−(x− β)TPBR−1BTP (x− β)
= (d−R−1BTP (x− β))TR(d−R−1BTP (x− β))−(x− β)TPBR−1BTP (x− β) (89)
Substituting (89) in (88),
JMEE = H(x, d)−H0 + (x(t)− β(t))TP (x(t)− β(t)) +
ˆ t
−∞[xT(−ATP − PA+ CTQC − PBR−1BTP )x
−2xT(−ATPβ − PBR−1BTPβ + PBu+ CTQy − P β) + yTQy + 2βTP (Bu− β)− βTPBR−1BTPβ
+(d−R−1BTP (x− β))TR(d−R−1BTP (x− β))]dτ. (90)
Choose
1. P such that −ATP − PA+ CTQC − PBR−1BTP = 0.
2. β such that −ATPβ−PBR−1BTPβ+PBu+CTQy−P β = 0. Using −ATP−PBR−1BTP =PA− CTQC,
PA− CTQC + PBu+ CTQy − P β = 0
⇒ β = (A− P−1CTQC)β +Bu+ P−1CTQy (91)
3. H0 =´ t−∞ y
TQy + 2βTP (Bu− β)− βTPBR−1BTPβ.
Therefore,
JMEE = H(x, d) + (x(t)− β(t))TP (x(t)− β(t)) +
ˆ t
−∞(d−R−1BTP (x− β))TR(d−R−1BTP (x− β))]dτ.(92)
Thus, JMEE can be minimized by choosing d = R−1BTP (x− β) and x(t) = β(t).We need to show that limτ→−∞β(τ) = 0. We need to solve Equation (91) backwards in time.
Observe that as τ → −∞, u(τ) = 0 and y(τ) = 0. Thus, we eventually have an “autonomoussystem” with time running backwards as after a finite time running back, both u and y become ar-bitrarily small. Thus, limτ→−∞β(τ) = 0 if −(A−P−1CTQC) is asymptotically stable. (The minus
sign is obtained when we take the limit limτ→−∞ of the state transition matrix e(A−P−1CTQC)τ .)Moreover, we need to show that limt→∞(x(τ) − β(τ)) = 0. Note that at time t, x(t) = β(t).
Using equations for ˙x and β and d,
˙x− β = (A+ BR−1BTP )(x− β) + P−1CTQ(Cβ − y). (93)
Solving this in backward time and using the fact that limτ→−∞y(τ) = 0 and limτ→−∞β(τ) = 0,limt→∞(x(τ) − β(τ)) = 0 if −(A + BR−1BTP ) is asymptotically stable. The minimum value ofJMEE is given by H(x, d) = H0. This proves the following theorem (Hespanha)
Theorem 16.5. Assume that there exists a symmetric solution P to the ARE: −ATP − PA +CTQC − PBR−1BTP = 0 such that matrices −(A + BR−1BTP ) and −(A − P−1CTQC) areasymptotically stable. Then MEE estimator for (83) is given by
˙x = (A− LC)x+Bu+ Ly, L := P−1CTQ, (94)
where d = R−1BTP (x− β).
38
Proof. Follows from the discussion above.
This is a deterministic estimator. Let e = x− x. Then e = x− ˙x = (A−LC)e+ Bd−Ln whereA− LC is asymptotically stable. Thus, we have BIBO stability of MEE estimator w.r.t. d and nwhich ensures that the estimate x doesn’t diverge from x. In the absence of both d and n, we haveasymptotically converging estimates.
16.3 LQG (Kalman-Bucy filter)
Suppose d and n are known and both are uncorrelated zero mean Gaussian white-noise stochasticprocesses with covariance matrices
E[d(t)dT(t)] = δ(t− τ)R−1, E[n(t)nT(t)] = δ(t− τ)Q−1, Q,R > 0. (95)
Then the MEE estimate x minimizes JLQG := limt→∞E[‖x(t)− x(t)‖2] which is same as the traceof E(x(t)− x(t))(x(t)− x(t))T. The MEE estimator for the above models of d and n is also calledKalman-Bucy filter.
The proof of the following result is taken from Murray (Lecture notes).
Theorem 16.6 (Kalman-Bucy filter). Let x = Ax + Bu + Bd, y = Cx + n where d and n areuncorrelated zero mean Gaussian white-noise stochastic processes with covariance matrices
E[d(t)dT(t)] = δ(t− τ)R−1, E[n(t)nT(t)] = δ(t− τ)Q−1, Q,R > 0. (96)
The optimal estimator has the following form of a linear observer
˙x = Ax+Bu+ L(t)(y − Cx) (97)
where L(t) = P (t)CTQ and P (t) = E{(x− x)(x− x)T} satisfies
P (t) = AP + PAT − PCTQCP + BR−1BT , P (0) = E{x(0)x(0)T} (98)
Proof. Let e = x− x and ξ = Bd− Ln. Therefore,
e = (A− LC)e+ ξ
⇒ P (t) = (A− LC)P + P (A− LC)T + BR−1BT + LQ−1LT. (99)
For the explanation of the last equation, we refer the reader to Appendix, Equation (108) (Thisderivation is taken from Friedland: Section 10.7, derivation of equation (10.58)).
P (t) = AP + PAT − LCP − PCTLT + BR−1BT + LQ−1LT
= AP + PAT + BR−1BT + (LQ−1 − PCT)Q(LQ−1 − PCT)T − PCTQCP
We need to find L such that P (t) is as small as possible. Since (LQ−1−PCT)Q(LQ−1−PCT)T ≥ 0,to make P (t) small, the positive term in the expression for P can be made zero. Thus, P (t) =AP + PAT − PCTQCP + BR−1BT and P (0) = E{x(0)x(0)T}.
Suppose we design a state feedback controller u = −Kx that solves an LQR problem andconstructed an LQG/MEE state estimator ˙x = (A− LC)x+Bu+ Ly. If states are not available,we may use u = −Kx. Therefore,
˙x = (A− LC −BK)x+ Ly
x = (A−BK)x+BKe+ Bd
e = (A− LC)e+ Bd− Ln˙[xe
]=
[A−BK BK
0 A− LC
] [xe
]+
[BB
]d+
[0−L
]n. (100)
39
Thus, the separation principle holds for LQG problem as well. By choosing K and L, the closed loopsystem can be made asymptotically stable. This ensures that both x and e do not diverge. Note thatthe separation principle says that we can design optimal estimator and optimal regulator separately,and combining them together solves an optimal regulation problem using output feedback.
16.4 Tracking problem
Consider an LTI system x = Ax + Bu, y = Cx + Du where we want the output y to track areference signal r corresponding to an equilibrium point (x∗, u∗) of the system i.e.
0 = x∗ = Ax∗ +Bu∗, r = Cx∗ +Du∗[−A B−C D
] [−x∗u∗
]=
[0r
](101)
• If the number of outputs is greater than the number of inputs, we have an overdeterminedsystem of equations and it may or may not have a solution.
• If the number of outputs is equal to the number of inputs, then the matrix in the previousequation is the Rosenbrock system matrix evaluated at s = 0. Thus, if the origin is not aninvariant zero, then there is a unique equilibrium point.
• If the number of outputs is less than the number of inputs, we have an under-determined
system of equations and it has multiple solutions. In general, we may write
[−x∗u∗
]=[
FxFu
]r.
Let x = x− x∗, u = u− u∗ and y = y − r. Consider the following cost functional
JLQR =
ˆ ∞0
(yTQy + ρuTRu)dt, ρ > 0. (102)
Note that Q penalizes the deviation from the reference signal and R penalizes deviation from theequilibrium input. Observe that
˙x = x = Ax+Bu = Ax+Bu−Ax∗ −Bu∗ = Ax+Bu
y = Cx+Du− r = Cx+Du− Cx∗ −Du∗ = Cx+Du
Thus, we now have a regulator problem and the optimal solution is given by u = −Kx. Thusu = u+ u∗ = −K(x− x∗) + u∗.
We showed that the tracking problem can be transformed into an equivalent LQR (regulator)problem. If states are not available, then to implement feedback laws, we need an estimator(Hespanha). In the presence of noise and disturbance, we need a linear quadratic estimator toestimate states x and we use u = −Kx.Tracking non constant reference signals: Suppose we want x(t) to track a trajectory r(t).Suppose (A,B) is controllable and there exists u∗(t) such that r = Ar(t) + Bu∗(t). Let x(t) =x(t)− r(t) and u = u− u∗. Therefore,
˙x = Ax(t) +Bu(t)−Ar(t)−Bu∗(t) = Ax(t) +Bu
and one needs to choose u such that the above system is asymptotically stable. For output tracking,suppose there exists x∗, u∗ such that x∗ = Ax∗ + Bu∗ and r(t) = Cx∗ + Du∗. Then, usingx = x− x∗, y = y − y∗ and u = u− u∗, one can solve the problem. We refer the reader to ([7]) forbuilding controllers to track non constant reference signals.
40
17 Appendix
The following definitions are from Papoulis and Pillai [15].
Definition 17.1. A random variable is a function from a sample space to C or R. (A sample spaceis the set of all possible events/outcomes of an experiment.)
All random variables have a probability density function (distribution) which allows us tocompute probabilities of different events. A normal or Gaussian distribution is the one where
the probability density function f is of the form of a Gaussian f(x) = 1√2πσ2
e−(x−µ)2
2σ2 where µ is
the mean and σ2 is the variance.
Definition 17.2. A random variable X is said to be zero mean if its expectation E(X) = 0, for arandom vector X, zero mean ⇒ E(X) = 0.
Definition 17.3. Two random variables X and Y are said to be uncorrelated if their covarianceC(X,Y ) = E(XY ) − E(X)E(Y ) = 0. For random vectors X, Y , they are said to be uncorrelatedif their covariance matrix C(X, Y ) = E(XY T)− E(X)E(Y )T is diagonal.
Definition 17.4. A function x(t, ξ) is said to be a stochastic process if for a fixed t0, x(t0, ξ) is arandom variable.
A stochastic process x(t) is said to be white noise if its values x(ti) and x(tj) are uncorrelatedfor every ti 6= tj . A white noise Gaussian stochastic process is the one where each random variableis zero mean and Gaussian with any two random variables being uncorrelated.A correlation matrix is given by Rx(t, τ) = E{x(t)xT(τ)}.
Consider a following system
x = Ax+ Fv, y = Cx
where v is a white Gaussian noise. Thus, E(v) = 0. Let Φ(t, t0) be the state transition matrix.Thus,
x(t) = Φ(t, t0)x(t0) +
ˆ t
t0
Φ(t, λ)F (λ)v(λ)dλ.
Then,
x(t)xT(τ) = Φ(t, t0)x(t0)xT(t0)ΦT(τ, t0) + Φ(t, t0)x(t0){ˆ τ
t0
Φ(τ, λ)F (λ)v(λ)dλ}T +
ˆ t
t0
Φ(t, λ)F (λ)v(λ)dλ.xT(t0)ΦT(τ, t0) +
ˆ t
t0
ˆ τ
t0
Φ(t, λ)F (λ)v(λ)vT(ξ)FT(ξ)ΦT(τ, ξ)dξdλ. (103)
Taking expectation on both sides in the previous equation and observing that E(v) = 0,
Rx(t, τ) = Φ(t, t0)E(x(t0)xT(t0))ΦT(τ, t0) +
ˆ t
t0
ˆ τ
t0
Φ(t, λ)F (λ)E(v(λ)vT(ξ))FT(ξ)ΦT(τ, ξ)dξdλ.(104)
Let E(v(λ)vT(ξ)) = Qδ(λ− ξ) and E(x(t0)xT(t0)) = P (t0). Thus,
Rx(t, τ) = Φ(t, t0)P (t0)ΦT(τ, t0) +
ˆ t
t0
ˆ τ
t0
Φ(t, λ)F (λ)Qδ(λ− ξ)FT(ξ)ΦT(τ, ξ)dξdλ
= Φ(t, t0)P (t0)ΦT(τ, t0) +
ˆ t
t0
Φ(t, λ)F (λ)QFT(λ)ΦT(τ, λ)dλ (105)
41
Observe that
ΦT(τ, λ) = [Φ(τ, t)Φ(t, λ)]T = ΦT(t, λ)ΦT(τ, t). (106)
Substituting (106) in (105),
Rx(t, τ) = Φ(t, t0)P (t0)ΦT(τ, t0) +
ˆ t
t0
Φ(t, λ)F (λ)QFT(λ)ΦT(t, λ)ΦT(τ, t)dλ
= Rx(t, t)ΦT(τ, t) (107)
Let P (t) := E(x(t)xT(t)) = Rx(t, t). Therefore,
d
dtRx(t, t) =
d
dt(Φ(t, t0)P (t0)ΦT(t, t0) +
ˆ t
t0
Φ(t, λ)F (λ)QFT(λ)ΦT(t, λ)dλ)
⇒ P (t) = AP + PAT + FQFT (108)
where the last equality follows by using differentiation under the integral sign.Loop shaping: The idea behind loop shaping is to manipulate block transfer functions such thatone obtains a desired transfer function (which gives desired characteristics such as time response,stability etc.). Using the Nyquist criterion for stability of a closed loop transfer function usingan open loop transfer, one manipulates open loop transfer function to obtain a desired frequencyresponse. This method is used frequently in design problems. MFDs are used for loop shaping.We refer the reader to Doyle, Francis and Tannerbaum, Skogestad and Postlaithwaite for furtherdetails. While implementing feedback laws using state estimators (e.g. linear quadratic estimator(LQE)), open loop gain gets affected. It is shown in Hespanha (Chapter 24) that one can recoveropen loop gain if certain conditions are satisfied. This is called loop transfer recovery (LTR).Kalman equality:
GTH = 0,K = R−1BTP, L(s) = K(sI −A)−1B, LT(−s) = −BT(sI +AT)−1KT,
T (s) = G(sI −A)−1B +H, TT(−s) = −BT(sI +AT)−1GT +HT
TT(−s)T (s) = −BT(sI +AT)−1GTG(sI −A)−1B +HTH (109)
ATP + PA+GTG− PBR−1BTP = 0⇒ ATP + sP − sP + PA+GTG− PBR−1BTP = 0
pre multiplying by -BT(sI +AT)−1 and post multiplying by (sI −A)−1B,
⇒ −BT(sI +AT)−1((sI +AT)P − P (sI −A) +GTG− PBR−1BTP )(sI −A)−1B = 0
⇒ −BTP (sI −A)−1B +BT(sI +AT)−1PB −BT(sI +AT)−1GTG(sI −A)−1B +
BT(sI +AT)−1PBR−1BTP (sI −A)−1B = 0.
using RK = BTP and L(s) = −K(sI −A)−1B,
−RL(s)− LT(−s)R+ TT(−s)T (s)−HTH − LT(−s)RL(s) = 0
⇒ RL(s) + LT(−s)R+ LT(−s)RL(s) = TT(−s)T (s)−HTH
⇒ RL(s) + LT(−s)R+ LT(−s)RL(s) +R = R+ TT(−s)T (s)−HTH
⇒ (1 + LT(−s))R(1 + L(s)) = R+ TT(−s)T (s)−HTH (110)
Further reading: We recommend the reader to read Hespanha [1]. For abstract linear algebraicapproach and geometric approach using subspaces for LTI systems, we refer the reader to Wonham[2]. Hespanha [1] and Brockett [3] consider both LTI and LTV systems. For a detailed realizationtheory and matrix fraction descriptions and examples, we refer the reader to Kailath [4]. Fornumerical aspects in systems and control one may refer to the lecture notes by Van Dooren [13].
For loop shaping and frequency domain approach, refer [9], [10]. For Kalman filter, stochasticprocesses and more, refer [5]. For linear and optimal control, [8], [14] and [14] in particular forrobust control. Skogestad and Postlaithwaite [9] also discuss robust control in detail. For modelorder reduction and large scale systems refer Antoulas [16].
42
Acknowledgments
Thanks to Shauvik Das for an extremely careful reading of the document and also for his use-ful comments. Thanks to Pragada Shivaramakrishna for many useful discussions and comments.Thanks to Lieut. CDR. Vamsi, Moduga Vijay Babu, Noble Tawra and Arpita Sikder for theiruseful comments.
References
[1] J. Hespanha, Linear systems theory, Princeton university press, 2009.
[2] W.M. Wonham, Linear Multi-variable Control A Geometric Approach, third edition, Springer-Verlag, 1985.
[3] R. W. Brockett, Finite Dimensional Linear Systems, John Wiley and Sons, Inc., 1970.
[4] T. Kailath, Linear systems, Prentice hall inc. 1980.
[5] B. Friedland, Control system design : An introduction to state-space methods, Dover publica-tions inc., 1986.
[6] R. Murray, Optimization-Based Control : Lecture notes
[7] A. Astolfi, Tracking and Regulation in Linear Systems, Encyclopedia of Systems and ControlSpringer-Verlag London, 2014.
[8] H. Kwakernaak, R. Sivan, Linear Optimal Control Systems, Willey Interscience, 1972.
[9] S. Skogestad, I. Postlaithwaite, Multi-variable Feedback Control Analysis and design, John Wil-ley and sons, 2007.
[10] J. Doyle, B. Francis, A. Tannerbaum, Feedback Control Theory, Macmillan Publishing Co.,1990.
[11] F. Pasqualetti, A Systems and Control Perspective on Privacy, Safety, and Security in Large-Scale Cyber-Physical Systems, Presentation slides from Disc summerschool talk, the Hague, theNetherlands, 2017.
[12] H. Fawzi, P. Tabuada, S. Diggavi, Secure state-estimation for dynamical systems under activeadversaries, IEEE Transaction on Automatic Control, vol. 59, no.6, pp. 1454-1467, 2014.
[13] P. M. Van Dooren Numerical Linear Algebra for Signals Systems and Control, Lecture notesfor Graduate School in Systems and Control, 2003.
[14] K. Zhou, J. Doyle, K. Glover, Robust and Optimal Control, Prentice hall, 1996.
[15] A. Papoulis, U. Pillai, Probability, Random Variables and Stochastic Processes, fourth edition,Tata-McGraw hill, 2002.
[16] A. Antoulas, Approximation of Large-Scale Dynamical Systems, Advances in Design and Con-trol, SIAM, 2005.
“There is no charge for awesomeness...” -Po (Dragon warrior)
43