Linear-Quadratic-Gaussian Problem

Study Guide for ES205

Yu-Chi HoJonathan T. LeeFeb. 5, 2001

2

Outline Linear-Quadratic Problem Calculus of Variation Approach Dynamic Programming Approach Kalman Filter Linear Feedback Control

3

Linear-Quadratic Problem

subject to linear system dynamics

given the initial state x(0)where x(i) is the state variables at time iu(i) is the control variable at time iA(i) and B(i) are the cost matrices at time i

1

0 21

21

21min

N

i

TTt

T

iuiuiBiuixiAixNxNANxJ

f

iuiixiix 1

N

4

Calculus of Variation Approach Let

N

1

0

11N

i

T ixiuiixiiJH

NxNANixiAiiixH T with ,10

10 1 iiiBiuuH T

0 with ,10 xiuiixiixH

5

Dynamic Programming Approach Cost-to-go

withwhere

and N

121

21

min

01

0

iNxJ

iNuiNBiNu

iNxiNAiNx

iNxJ

i

T

T

iNui

1112110

1 iNxiNSiNxiNxJ Ti

11211 iNAiNiNMiNiNS T

21

1211

12221

iNSiN

iNiNSiNiNB

iNiNSiNSiNM

T

T

6

DP Approach (cont.) Set We have

Let

andThen

1 1 1u N i B N i N i M N i N x N i

0

0

iNuiNxJ iN

iNAiNiNMiNiNS T 1

iNxiNSiNxiNxJ Ti

210

N

1

1

1111

iNSiN

iNiNSiNiNB

iNiNSiNSiNM

T

T

7

DP Approach (cont.) By induction, we have the optimal

solution to be

whereand

with boundary conditionwith

1 1 1Tu i B i i M i N x i iAiiMiiS T 1

NANS

N

11

111111

iSiiiSiiB

iiSiSiMTT

0,...,1Ni

8

Static State

subject to state transition

with initial conditionwhere

and is noise

ixix 1 00 x

ixiz

N

ix

xizJ0

2

ˆˆ

21min

N

9

Static State (cont.) Set

Thus,

0ˆ0

N

i

xiz

0ˆ

xJ

N

i

izN

x01

1ˆ

N

10

Static State (cont.)

New Estimate = Old Estimate + Confidence Factor Correction Term

11

111

1ˆ1

izi

jzii

iixi

j

11

1ˆ1

izi

ixii

Term Correction

Factor Confidence

ˆ11

1ˆ ixizi

ix

N

11

Dynamic State

subject to state transition

with disturbance (i)where

and is noise

iixix 1

N

i

N

iix

ixizixJ0

21

0

22

21

210

21min

ixiz

N

12

Dynamic State (cont.) Cost-to-go

Set

where We have

N

22

0

00 00

210

21min0ˆ xzxxJ

x

00

00

xJ

002100

210ˆ xzxzx

0410ˆ 20

0 zxJ

00 x

13

Dynamic System (cont.) Cost-to-go

Substitute

04111

210

21min

0ˆ11210

21min1

222

1

00

22

1

01

zxz

xJxzxJ

x

x

0ˆ10 xx

041

1121

0ˆ121

min12

22

1

01

z

xzxxxJ

x

N

14

Dynamic System (cont.) Set

Let Then, we have

01

01

xJ

0ˆ1210ˆ

20ˆ11ˆ xzxxzx

0ˆ1 xx

112111ˆ xzxx

N

15

Dynamic System (cont.) Cost-to-go

Substitute

1ˆ

21

21min 0

1220 ixJixiziixJ iixi

1ˆ ixixi

1ˆ211ˆ

21

minˆ0

1

220

ixJ

ixizixixixJ

iixi

N

16

Dynamic System (cont.) Set

Let and Then, we have

00

ixJ i

1ˆ211ˆ

21ˆˆ

ixizixixizix

1ˆ ixix

ixizixix 21ˆ

00 x

N

17

With Control

subject to state transition

with disturbance (i)where

and is noise

iwiuixix 1

N

ixiz

N

i

N

iix

ixizixJ0

21

0

22

21

210

21min

18

With Control (cont.) Cost-to-go

Set

We haveN

22

0

00 00

210

21min0ˆ xzxxJ

x

00

00

xJ

0210ˆ zx

0410ˆ 20

0 zxJ

19

With Control (cont.) Cost-to-go

Substitute

04111

210

21min

0ˆ11210

21min1

222

1

00

22

1

01

zxz

xJxzxJ

x

x

00ˆ10 uxx

041

112100ˆ1

21

min12

22

1

01

z

xzuxxxJ

x

N

20

With Control(cont.) Set

Let Then, we have

01

01

xJ

00ˆ12100ˆ

200ˆ11ˆ

uxzux

uxzx

00ˆ1 uxx

112111ˆ xzxx

N

21

With Control(cont.) Cost-to-go

Substitute

1ˆ

21

21min 0

1220 ixJixiziixJ iixi

11ˆ iuixixi

1ˆ2111ˆ

21

minˆ0

1

220

ixJ

ixiziuixixixJ

iix

i

N

22

With Control(cont.) Set

Let andThen, we have

00

ixJ i

11ˆ2111ˆ

211ˆˆ

iuixiziuix

iuixizix

11ˆ iuixix

ixizixix 21ˆ

N

00 x

23

Linear Feedback Control The predicted state

with initial statebased on the estimator

112111ˆ ixHizixix

11ˆ iuixix

00 xx

N

24

Linear Feedback Control (cont.) Optimal control based on the

estimated state:

whereand

with boundary conditionwith

N

ixiSNiMiiBiu T ˆ111

iAiiMiiS T 1

NANS

0,...,1Ni

11

111111

iSiiiSiiB

iiSiSiMTT

25

Applications Apollo program: control of the space

craft. Airplane controllers: autopilot. Neighboring Optimal Control for

Non-Linear Systems Chemical process controller Guidance and control

N

26

References:

• Bryson, Jr., A. E. and Y.-C. Ho, Applied Optimal Control: Optimization, Estimation, and Control, Taylor & Francis, 1975.

• Ho, Y.-C., Lecture Notes, Harvard University, 1997.