Automatic Control

Linear Quadratic Regulator (LQR)

Consider a linear vector space over .

A norm on is a function : with the following properties:

For all , and for all ,

1. 0 (non-negativity)

2. = (positive homogeneity)

3. + + (triangular inequality)

4. = 0 if and only if = 0 (separates point)

Norms Basic definition

Consider a vector .

- 2 norm: 2 = 2=1 (Euclidean norm)

- 1 norm: 1 = =1

- norm: = max=1,,

Norms Vector norms

These norms give rise to the 2, 1 and vector Banach spaces.

Consider the vector = 1 2 3 3.

- 2 norm: 2 = 1 2 + 2 2 + 3 2 = 14 = 3.7417

- 1 norm: 1 = =1 = 1 + 2 + 3 = 6

- norm: = max=1,,

1 , 2 , 3 = 3

Norms Vector norms: example

Consider the vector = 1 2 3 3.

- 2 norm: norm(f)=norm(f,2)=3.7417

- 1 norm: norm(f,1)=6

- norm: norm(f,inf)=3

Norms Vector norms in Matlab

Norms Signal norms

These norms give rise to the 2, 1 and functional Banach spaces.

Consider a function (signal) :+ , where + = [0,].

- 2 norm: 2 = () 2

0

- 1 norm: 1 = ()

0

- norm: = sup+

()

Norms Signal norms

These norms give rise to the 2, 1 and functional Banach spaces.

Consider a function (signal) :+ , where + = [0,].

- 2 norm: 2 = () 22

0= ()()

0

- 1 norm: 1 = () 1

0

- norm: = sup+

()

Very common are the 2 and norms:

- 2 norm: 22= () 2

2

0 = energy of the signal

- norm: = sup+

() = amplitude of the signal

Norms Signal norms

Consider the function () = 1 0 20 > 2

Norms Signal norms: examples

22 2

2

0 0

[0,2]

( ) 2 area of the red region energy of

max ( ) max ( ) 1 largest value of ( ) amplitude of tt

f f t dt dt f

f f t f t f t f

R

0 2 4 6 8 10-0.5

0

0.5

1

1.5

t

f(t)

0 2 4 6 8 10-0.5

0

0.5

1

1.5

t

f(t)2

Consider the function () = 3

Norms Signal norms: examples

2 2 6 6

200 0

3

1 1( )

6 6

max ( ) max 1

t t

t

t t

f f t dt e dt e

f f t e

R R

0 0.2 0.4 0.6 0.8 1-0.5

0

0.5

1

1.5

t

f(t)

0 0.2 0.4 0.6 0.8 1-0.5

0

0.5

1

1.5

t

f(t)2

Consider the function () = sin()

Norms Signal norms: examples

2 2 2

22

0 0

( ) sin( ) unbounded norm

max ( ) max sin( ) 1 bounded normt t

f f t dt t dt

f f t t

R R

0 2 4 6 8 10

-1

0

1

t

f(t)

0 2 4 6 8 10-0.5

0

0.5

1

1.5

t

f(t)2

Consider the function () = ()

Norms Signal norms: examples

2

22

0 0

( ) ( ) 1 bounded norm

max ( ) max ( ) unbounded normt t

f f t dt t dt

f f t t

RR

-1 -0.5 0 0.5 10

5000

10000

t

f(t)

-1 -0.5 0 0.5 10

5

10x 10

7

t

f(t)2

2

Norms Signal norms in Matlab

Consider the function () = 3

dt=0.0001;

t=0:dt:10;

f=exp(-3*t);

norm(f)*sqrt(dt) % L2 norm

norm(f,1)*dt % L1 norm

norm(f,inf) % Linf norm

For vector-valued functions, norm computation can be slightly

more complicated.

Consider a function (signal) :+ , where + = [0,].

- 2 wighted norm: ,2 = ()()

0

Norms Signal weighted norms

dt=0.0001;

t=0:dt:10;

f=exp(-3*t);

sqrt(sum(f*Q*f)*dt) % L2 weighted norm

LQR control Introduction

Suppose that the following LTI system has to be controlled:

How to choose the desired eigenvalues for the controlled system?

Two of the most common techniques are the following:

- Method of II order dominant poles: The desired eigenvalues are

chosen in such a way that the controlled system is similar to a

second-order system with desired damping and natural frequency.

- Optimal control (Linear Quadratic Regulator, LQR): The desired

eigenvalues are chosen on the basis of suitable optimality criteria.

)()(

)()()(

txty

tButAxtx

LQR control Introduction

The method of II order dominant poles consists in placing the poles in order to obtain the desired behavior.

This placement is carried out assuming that the controller can provide an arbitrarily large amount of energy.

Problems: In most practical situations:

- any actuator is subject to energy and amplitude limitations;

- large control actions can drive the system out of the linearity region.

LQR control Introduction

Certain limitations have to be imposed to the control in practical implementations.

A typical approach is to require a control input with a small weighted energy:

,22= ()()

0 small

The performance requirements can be met imposing a small weighted energy for the state:

,22= ()()

0 small

The matrices and determine the trade-off between performances (minimization of 2) and command activity (minimization of 2).

LQR control Problem formulation

J(u,x) is a functional, i.e. a function of functions:

For any function (signal) x() and any function (signal) u(), we have a given value of J(u,x).

Define the following cost functional:

Q,R : fixed weight matrices.

2

2,

2

2,),(

RQuxxuJ

The functional associates to any couple of functions (u(),x()) a real number:

R Jxu )(),(

LQR control Problem formulation

For given initial conditions, the state signal depends on the input signal ():

input signal u(t) state signal x(t) J(x,u)R

0)0(

)()()(

xx

tButAxtx

This implies that, for given initial conditions, the cost functional depends uniquely on the input signal:

(, ) (, {})

LQR control Problem formulation

The LQR (Linear Quadratic Regulator) control law is the solution of the

following optimization problem:

0

,

*

)0(

),0[),()()(

subject to

),(minarg

xx

ttButAxtx

xuJuxu

The law is obtained minimizing the weighted energy of and subject to the constraint that and must satisfy the dynamic equations.

Stability is not explicitly accounted for. Minimizing 2 ensures closed-loop stability.

LQR control Main result

Theorem. Assume that (, ) is controllable. For any initial condition 0, the optimal LQR controller is given by

= ()

where

and is obtained solving the following Algebraic Riccati Equation:

The optimal controller can be easily designed in Matlab:

K = lqr(A,B,Q,R);

1 TK R B P

1 0T TA P PA Q PBR B P

LQR control Design procedure

Initial choice: and diagonal with

Calibration:

change the values of Qii according to the following indications, until the

requirements are satisfied in simulation:

Qii large energy of xi small

Rii large energy of ui small

1 in the presence of requirements on

0 otherwise

1 in the presence of requirements on

0 otherwise

i

ii

i

ii

xQ

uR

LQR control Example: feedback control of satellite attitude

Differential equation:

Initial conditions: x(0)=[0.3;0.02], disturbance: MD=0.

1

0,

00

10,,),()()( BA

J

MdFuxtButAxtx dc

)(),(minarg)(,

* tKxxuJtuxu

LQR control law:

)()()( tMtdFtJ Dc

Earth d

Fc

MD

LQR control Example: feedback control of satellite attitude

41.111,00

01

KRQ

0.79 0.321,00

010/1

KRQ

)()(* tKxtu LQR control law:

No control: = [00]

51.216.310

1,

00

01

KRQ

0 1

LQR control Discussion

The LQR controller has been derived for the below control structure, where no reference is used:

x u*

K

P -

LQR control Discussion

The LQR controller can be used in all the other considered control structures:

x u*

K

P -

d

LQR control Discussion

The LQR controller can be used in all the other considered control structures:

P

K

x + -

r kr C y

d

u*

P x

K0

+

-

r Kc/s

+

- C

y

d

u*

LQR control Discussion

The LQR controller can be used with an observer:

u*

O

P y

K

+

- r

If the observer O is the Kalman filter, then we obtain an LQG (Linear Quadratic Gaussian) control.