Rational Functions and Optimal

Decentralized Control

Sanjay Lall

Stanford University

Control, Optimization, and Functional Analysis: Synergies and PerspectivesWorkshop in honor of Professor Bill HeltonOctober 3, 2010

2

Outline

• Quadratic invariance

• Convexity of the image of a rational function

• Representations of rationals

• State-space solutions

3

Quadratic Invariance

with M. Rotkowitz

4

Decentralized Controller Synthesis

minimize ‖A + BX(I − DX)−1C‖

subject to X ∈ S

• S is a subspace, e.g.,

S =

{[

x y + 3z0 y

] ∣

∣

∣

∣

x, y ∈ R

}

• A, B, C, D are given matrices

• In control, S corresponds to the set of decentralized controllers

• General problem is computationally intractable

5

Rational Functions

A+BX(I−DX)−1C =

−16 x + 12 y + 3 z + 32 x y + 40 x z + 16 x2

6 x + 8 y + 16 x z − 1

8 x + 4 y − 3 z + 16 x y + 28 x z + 8 x2 − 2

6 x + 8 y + 16 x z − 1

Coefficient matrices Subspace of variables

A =

[

02

]

B =

[

3 10 1

]

C =

434

D =

4 44 02 2

X =

[

x 0 0y z x + 2y

]

Coeffs. and vars. may be functions, e.g., in H2 or H∞.

6

Youla Parameterization

minimize ‖A + BX(I − DX)−1C‖

subject to X ∈ S

Let h(X) = −X(I − DX)−1, and use the change of variables

Q = h(X)

gives equivalent problem

minimize ‖A − BQC‖

subject to h(Q) ∈ S

Works when there is no constraint that X ∈ S

7

Quadratic InvarianceThe subspace S is called quadratically invariant under D if

XDX ∈ S for all X ∈ S

S is quadratically invariant under D if and only if

X ∈ S ⇐⇒ X(I − DX)−1 ∈ S

Equivalently: QI ⇐⇒ h(S) = S.

8

Convex Optimization

minimize ‖A − BQC‖

subject to Q ∈ S

• Convex program

• Given optimal Q, let X = −Q(I − DQ)−1

• Centralized problem (i.e., without X ∈ S constraint) reduces to4-block problem: has state-space solution for H2 and H∞

• No general state-space solution

9

Theorem• U and Y are Banach spaces

• D : U → Y is compact

• S ⊂ L(Y ,U) is a closed subspace

• Let M ={

X ∈ L(Y ,U) ; (I − DX) is invertible}

• Let h(X) = −X(I − DX)−1

Then the subspace S is quadratically invariant under D if and only if

h(S ∩ M) = S ∩ M

10

ExampleSuppose S and D are given by

D =

∗ 0 0 0 0∗ ∗ 0 0 0∗ ∗ ∗ 0 0∗ ∗ ∗ ∗ 0∗ ∗ ∗ ∗ ∗

S =

X | X =

0 0 0 0 00 ∗ 0 0 00 ∗ 0 0 0∗ ∗ ∗ 0 0∗ ∗ ∗ 0 ∗

S is quadratically invariant, since for general X ∈ S

XDX ∼

0 0 0 0 00 ∗ 0 0 00 ∗ 0 0 0∗ ∗ ∗ 0 0∗ ∗ ∗ 0 ∗

11

Example

G1 G2 G3

K1 K2 K3

p

p

c

c

c

p

p

c

c

c

• p is the propagation delay

• c is the communication delay

• Quadratic invariance if c ≤ p

12

Delay StructureIn the above problem D and X are structured according to

D =

∗ λp∗ λ2p∗ λ3p∗

λp∗ ∗ λp∗ λ2p∗

λ2p∗ λp∗ ∗ λp∗

λ3p∗ λ2p∗ λp∗ ∗

X =

∗ λc∗ λ2c∗ λ3c∗

λc∗ ∗ λc∗ λ2c∗

λ2c∗ λc∗ ∗ λc∗

λ3c∗ λ2c∗ λc∗ ∗

Quadratically invariant if c ≤ p, since XDX has the structure

XDX =

∗ λr∗ λ2r∗ λ3r∗

λr∗ ∗ λr∗ λ2r∗

λ2r∗ λr∗ ∗ λr∗

λ3r∗ λ2r∗ λr∗ ∗

where r = min{c, p}

13

Convexity

with Laurent Lessard

14

ConvexityReduced rational optimization to

minimize ‖A − BQC‖

subject to Q ∈ h(S)

Are there any cases when h(S) is convex, but h(S) 6= S?

15

Convexity

S is quadratically invariant under D iff h(S) is convex.

• Hence if h(S) is convex, then h(S) = S

• S is quadratically invariant iff {X(I − DX)−1 | X ∈ S} is convex

16

TheoremSuppose

• U and Y are Banach spaces, D ∈ L(U ,Y)

• Let M ={

X ∈ L(Y ,U) ; (I − DX) is invertible}

• Let h(X) = −X(I − DX)−1

If

• S ⊂ L(Y ,U) is a closed double-cone

• T ⊂ L(Y ,U) is convex

• h(S ∩ M) = T ∩ M

Then T ∩ M = S ∩ M .

17

Algebraic Framework

with Laurent Lessard

18

Algebraic Version• R is a commutative ring with 2 a unit

• D ∈ Rm×n

• S is an R-module

• Let M ={

X ∈ Rm×n | (I − DX) is invertible}

If S is quadratically invariant, then

h(S ∩ M) = S ∩ M

• Removes compactness requirements on D

• Necessary for particular rings, e.g. proper rational functions

19

2 a unitIf 2 6∈ U(R) then the above can fail. On the integers, let

S =

2x y z

y z 0z 0 0

∣

∣

∣

∣

∣

∣

x, y, z ∈ Z

, D =

0 0 00 0 10 1 0

S is QI, since

XDX =

2yz z2 0

z2 0 00 0 0

But X ∈ S does not imply h(X) ∈ S, e.g.,

X =

0 0 10 1 01 0 0

=⇒ h(X) =

1 1 11 1 01 0 0

20

General Rationals

• QI tells us about convexity of

{X(I − DX)−1 | K ∈ S}

• What about convexity of

C = {A + BX(I − DX)−1C | X ∈ S}

• QI depends on the representation used for C.

• How to test convexity independent of representation?

21

General RationalsDefine

C ={

A + BX(I − DX)−1C | X ∈ S}

We’d like to solve

minimize ‖X‖

subject to X ∈ C

If S is quadratically invariant under D, then C is linear

C ={

A − BQC | Q ∈ S}

22

Example

[

A B

C D

]

=

a b1 b2 b2c1 d1 0 0c1 d1 0 0c2 d2 d3 d3

X =

x1x2

x3

[

A B

C D

]′

=

a b1 b2c1 d1 0c2 d2 d3

X ′ =

[

x1x2 x3

]

• Both examples have the same set C

• The first is not QI, but the second is.

23

Internal Quadratic Invariance

S is called internally quadratically invariant under

[

A B

C D

]

if

W2SW1 is QI with respect to D

• W1 and W2 are projectors satisfying

range W1 = range[

C D]

null W2 = null

[

B

D

]

• Independent of choice of W1 and W2 when S is a module

• Projectors can always be chosen to be proper

24

Feedback TransformationsW1 and W2 are projectors satisfying

range W1 = range[

C D]

null W2 = null

[

B

D

]

Add Wi into the feedback loop without changing the closed-loop map.

z w

X

W1 W2

A B

C D

z w

W2XW1

A B

C D

25

Theorem• P is rational and proper

• D is strictly proper

• S is a module in the set of proper rationals

• Let h(K) = K(I − DK)−1

If S is internally quadratically invariant, then

Bh(S)C = BSC

G1 G2

K1 K2

z−1

z−1

z−1

z−1

w1

r1

r2

w2

u1 y1 u2y2

26

Delay Example

D =

H11 z−1H12

z−2H21 z−1H22

z−1H21 H22

z−1H11 z−2H12

X =

[

K11 K12 0 00 0 K22 K21

]

S is not quadratically invariant with respect to D, so use

W1 =1

1 + z−2

1 0 0 z−1

0 z−2 z−1 0

0 z−1 1 0

z−1 0 0 z−2

W2 =

[

1 00 1

]

G1 G2

K1 K2

z−1

z−1

z−1

z−1

w1

r1

r2

w2

u1 y1 u2y2

G1 G2

K1 K2

z−1

z−1

z−1

z−1

w1

r1

r2

w2

u1 y1 u2y2

27

Delay Example

quadratically invariant internally quadratically invariant

28

State-Space

with John Swigart

29

State-Space Solutions

Is there a state-space method for solving

minimize ‖A − BQC‖

subject to Q ∈ H2

Q ∈ S

• Riccati equations

• Separation structure

• Order of optimal controller

S1 S2

K1 K2

30

Two-Player LQR

[

x1(t + 1)x2(t + 1)

]

=

[

A11 0A21 A22

] [

x1(t)x2(t)

]

+

[

B11 0B21 B22

] [

u1(t)u2(t)

]

+ v(t)

Minimize

limN→∞

1

N + 1E

N∑

t=0

‖Cx(t) + Du(t)‖2

Objective: pick controller of the form

u1(t) = γ1(

t, x1(0), . . . , x1(t))

u2(t) = γ2(

t, x1(0), . . . , x1(t), x2(0), . . . , x2(t))

w

u

z

y

P11 P12P21 P22

K

31

LFT Formulation

minimize ‖P11 + P12K(I − P22K)−1P21‖

subject to K is stabilizing

K is block lower

• P =

[

C

I

]

(zI − A)−1 [

I B]

+

[

0 D

0 0

]

32

A Standard HeuristicOptimal centralized solution

[

u1u2

]

=

[

F11 F12F21 F22

] [

x1x2

]

where F = −(DTD + BTXB)−1BTXA

Common heuristic can be unstable:

u1 = F11x1 + F12xest2 Player 1 replaces x2 with xest

2

u2 = F21x1 + F22x2

33

What is known?

We know

• optimal controller is linear

• optimal controller is rational

• convergent numerical algorithms for impulse response

We do not know

• How many states each controller has?

• Is there a separation structure?

34

Optimal Decentralized Solution

u1 = F11x1 + F12xest2

u2 = F21x1 + F22xest2 + J

(

xest2 − x2

)

Here

• State estimator: xest2 (t) = E

(

x2(t) | x1(0), . . . , x1(t))

• Both players need to run the estimator.

• Even player 2 estimates his own state.

• Both controllers have n2 states.

• Player 2 corrects mistakes made by Player 1

• If xest2 = x2 (no noise) then same action as centralized case

35

Optimal Controller

u1 = F11x1 + F12xest2

u2 = F21x1 + F22xest2 + J

(

xest2 − x2

)

Riccati equations

X = CTC + ATXA − ATXB(DTD + BTXB)−1BTXA

Y = CT2 C2 + AT

22Y A22 − AT22Y B22(D

T2 D2 + BT

22Y B22)−1BT

22Y A22

Gains

F = (DTD + BTXB)−1BTXA

J = (DT2 D2 + BT

22Y B22)−1BT

22Y A22

0 50 100 150 200 250 300 350 400 450 500

−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

x2x2 − x1x1

0 50 100 150 200 250 300 350 400 450 500

−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

x2x2 − x1x1

Decentralized

Centralized

36

Example

• Two point masses

• Cost

1000(x1 − x2)2

+ (x21 + x2

2 + x21 + x2

2)

+ 10(u21 + u2

2)

• At time 50, force pulseapplied to mass 1

• time 200; mass 2

• time 300; both masses

0 50 100 150 200 250 300 350 400 450 500

−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

x2x2 − x1x1

0 50 100 150 200 250 300 350 400 450 500

−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

x2x2 − x1x1

Decentralized

Centralized

0 50 100 150 200 250 300 350 400 450 500−5

−4

−3

−2

−1

0

1

2

3

u2u1

0 50 100 150 200 250 300 350 400 450 500−5

−4

−3

−2

−1

0

1

2

3

u2u1Decentralized

Centralized

37

38

Trade-off

0 10 20 30 40 50 60 70 80 90 1000

20

40

60

80

100

120

140

centralizeddecentralized

Mean square relative position error

Mea

nsq

uare

effor

t

39

Workload Distributioneffort ratio about 3.5, varies little with position on curve

0 10 20 30 40 50 60 70 80 90 1000

20

40

60

80

100

120

140

player 1player 2

Mean square relative position error

Mea

nsq

uare

effor

t

40

Summary• Optimal-norm synthesis subject to quadratically invariant information

constraints is a convex optimization problem.

• Algebraic approach

• IQI class is strictly larger than QI class

• Two-player LQR

• Found optimal state-space solution to simple two-player network

• Estimator required for both systems; not classical certainty equiv-alence

• Optimal controller order is the size of A22