athans, m. - the matrix mininum principle (nasa)

8/3/2019 Athans, M. - The Matrix Mininum Principle (NASA)

1/23

I

AUGUST, 1967 REPORT E S l - R a 7M A T . PROJECT DSR 7626sNASA R a ~ r r b rant NGR-22-009(12

THE MATRIX M I W U M PRINCIPLE

M i W Atbam

GPO PRICECFSTI PRICE(S)6

Hard copy (HC)Microfiche ( M F )

ff 653 July 85

III

- N 67- 9 5 74(CODE)

Electroroic Systems LaboratoryMASSACHUSETTS INSTITUTE OF TECHNOLOGY, CAMBRIDGE, MASSACHUSFCTS ozis9Departmerst of Electrical EngineSring


2/23

. .,. 1TJI' '"I R ep or t ES L- R- 3 17 COPY1 August 1967

' THE MATRIX MINIMUM PR IN CIP LE ' - 'by

Michael Athans 1

T h i s r e s e a r c h w a s c a r r i e d o ut at the M . I . T . El ec t ron i c Sys t emsLa bo rat ory wi th s upport extended by the Nat ional Aeronaut ics andSpace Adm inis t ra t ion under re se ar ch g rant NGR-22 -0Oq 24) ,DSR Pro je ct No. 76265. 1

.-.,r-/-E1 c tr ni c System s L abor a tor y ".+-Department of Elec t r i ca l Engineer ingMassachuset ts Inst i tute of TechnologyCambr idge , Massachuse t t s 22139


3/23

ABSTRACT

The purpose of th i s repo r t i s to provide an al ternate s ta tement of thePont ryagin maximum pr inc ip le as appl ied to sy s te m s which a r e mo s tconvenient ly and natura . lly desc r ibed by m atr i x, ra th er than vector ,di f ferent ia l or di f ference equat ions .faci l i t a tes the manipulat ion of the res ul tant equat ions .applied to the solution of a sim ple optim ization pro ble m.The use of gradient m at r ic esThe theory i s

i i i


4/23

CHAPTERCHAPTER

CHAPTER

CHAPTER

II1

111

IV

CHAPTER V

CHAPTER VIAPPENDIX A

CONTENTS

INTRODUCTION paReTHE MATRIX MINIMUM PRINCIPLE(CONTINUOUS TIME)THE MATRIX MINIMUM PRINCIPLE(DISCRETE TIME)JUSTIFICATION O F THE MATRIXMINIMUM PRINCIPLEAPPLICATION TO A LINEAR CONTROLPROBLEMCONCLUSIONSA PARTIAL LIST OF GRADIENT MATRICES

1

6

8

10

131718

REFERENCES 19

V


5/23

I. INTRODUCTIONThe purpo se of t his re po rt i s to provide (with no proofs) a

state me nt of the ne ce ss ar y conditions fo r opt imali ty for a c la ssof problems that appear to be impor tan t a s ev idenced by rece n tr e s e a r c h e f fo r t s.the fact that the plant equat ions a r e m ost convenient ly des crib edby m atr ix d i f feren t ia l equat ions . F o r such p rob lem s , i t i s im -portant to have a com pact s tate me nt of the min imum pr incip les o a s to aid both intui tion and mathe matica l manip ulat ions; th isprovided the motivation o r this study.

This cla ss of problem s is distinguished by

In the rem ain de r of th is re por t the fol lowing topics ar e t re ate d:1. the relat ion of the mat r ix min imum pr incip leto the ord in ary min imum pr incip le ;2. a sta tem ent of the ne ce ss ar y conditions fo ropt imali ty as provided by the matr ix mini-m-Jm principle;3 . the solution of a v ery s imple p rob lem whichinvolves the deter min atio n of the li ne ar t ime -varying gains which optimlze the res po nse ofa l inear sy s te m with quadrat ic pe r forman ceindex.

The m os t common fo rm of the m inim um principle p erta ins to theopt imal con t ro l of sys tem s descr ibed by vector d i f feren t ia l equat ions- c C c',..,V I L A I C IVI111

(where -( t ) i s a co lumn n-vector , -( t ) i s a co lumn r -ve c to r , and -( . )i s a vector-valued funct ion) .by Pont ryag in e t a l .with m ode rn control theo ry. The descript ion of plants by Eq. 1 is ave ry common one; however , there a r e p rob lems in which theevolu t ion- in- t ime of thei r var iab les i s m ost natura l ly descr ibed byme ans of ma tr ix different ial equat ions.c o n Q i d e r a s ys t em -phose s ta te var iab les a re x

These ar e the type of sy ste ms con side red1 and treated in most of the available books dealing

To ma ke t h i s mo r e p r e c i s e ,with i = 1 , 2 , . . . ,ai j '

4-8. Supersc r ip t s r e f e r t o numbered i t ems in the Refe rences .- 1 -


6/23

- 2 -and j = 1 , 2 , . . . , m , and whose con tro l va r i ab le s a re u aa = 1, 2,. . , r and p = 1 , 2 , . . . q .th ink of the "s ta te mat r ix ' ' -( t ) whose e l emen t s a re the s t a t evar iab les x . . ( t ) and of the "control m atr ix ' ' -( t ) whose e lements

, withIn such pro blem s, one may

1.1a r e the con


7/23

- 3 -where K[ e ] and L[ - 3 a r e scalar-valued functions of the ir a rgu me nt.One ma y seek the opt imal contro l -ma tr ix g* ( t ) , which m ay be con-s t ra ine d by

which minimizes the cost functional - C).I t should be c le ar that the tools a r e avai lable to tackle this opt i -

mi za t i on p rob l em.f ir st ord er equa tions

After all, one can decompose Eq. 2 into a se t of

and proc eed with the application of the fam i l ia r minimum pr inc ip le .What happens, how ever, i s tha t one ma y get lo st i n a lo t of equationsand it may become a lm os t imposs ib le to de te rmine any s t ruc ture andpr op er ti es of the solution.for es t f ro m the t re es " which has provided the mot iva tion for dea lingwith pro ble ms involving the t im e -evolution of m atr ic es by cons truc t inga sys tem at ic notat ional approach.

It i s t h i s possibi l i ty that "one m ay lose the

The f i r s t s tep towards this goal is to real iz e that the s et of a l l ,s ay , n x m r ea l ma t r i ce s fo rm s a l i nea r vec t o r space wit h we l l-defined opera tions of addit ion and multiplication. Denote this vec torspace by S . Then, it i s possible t o define an inner productnmt h i s space .-E snm , the i r inner product i s defined by the t race operat ion

Thus , i f _A and ,B a re n x m m a t r i c es , i . e . , ,AEn m- -&_B) = t r [ A B ' ]_ = 1 a . . bj i j

It i s t r i v i a l t o ve r i fy tha t Eq. 10 indeed def ines an in ner product .Using this notation one can fo rm the Hamiltonian function for the o pti-miza t ion problem.var iab le assoc ia ted wi th x. . (t) then the Hamiltonian must take thef o r m

F i r s t of all, note that i f p. . ( t ) i s the costate1J1J

n nH = L[_X(t),g(t),1 -t1 A . . ( t)p. ( t) ( 1 1)lJ 1Ji = l j= 1


8/23

-4-Using Eq. 10, i t follows that the Hamiltonia n can be writ te n a s

w h e r e - ( t ) i s the cos ta te ma t r ix assoc ia ted with the s ta te mat r ixX( t) , in the sens e that the costa te var iable p. . ( t ) i s the ij th e lem ent

1J-of l?(t).Using the no tation of A thans and Fal b, it i s known that the costate

va r ia ble s sa t isfy the di f ferent ia l equat ions

Th is type of equation le ad s to the definition of the s o-ca lled gra di en tm a t r i ~ . ~ndeed i t ma y be ar gu ed that the use of gradient ma tr ic es fo rpure ly m anipula to ry purpo ses i s the key concept tha t ma kes the useof the ma t r ix m inimu m pr inc ip le su i tab le and s t ra ight forw ard .

A gradien t m at r i x i s def ined a s fol lows: Suppose that f (X)-s asca lar-v alue d funct ion of the elem ents x. . of 5 . Then the gradien tm a t r i x of 1.li s denoted by

and i t i s a mat r ix whose i j th e lement is simply given byr 1

A brief tab le of som e gra dient m atr ice s i s given in Appendix A .Using the notion of the g radient ma tri x, i t i s r ead il y s een tha t

Eq . 13 can be wr i t t en a s

s ince the Hamil tonian H i s a scalar-valued funct ion.Once thi s notation ha s been establ ishe d, one can state al l the known

neces sa ry cond i t i ons f o r opt imal i ty fo r vector - type pro blem s to theequiva len t s t a tements for the mat r ix- type problems . In the following


9/23

-5 -sec t ion , t he nec es sa ry cond it ions fo r op t imal i ty a r e s t a t ed fo r t hef ixed- t ime op t imizat ion prob lem with ter m ina l cos t .


10/23


11/23

- 7 -At the term ina l t ime ( t ra nsv ers al i ty condi tions)

(ii i) Minimization of the Ham iltonian:

fo r every _V E and for each t E[ t o , T] . (24)Note that i f - ( t ) i s unconst ra ined , then E q . 24 i m -

p l ies the ne ce ss ar y condi tion

i . e . , t he g ra di en t ma t r i x of the Hamil tonian with respectto the cont ro l mat r ix _V must van ish .


12/23

111. THE MATRIX MINIMUM PRIN CIPL E (DISCRETE TIM E)The re a r e p rob le ms fo r which the evolu tion of the per t inen t va r iab les

i s mos t na tu ra l ly desc r ibe d by a se t of m at r ix d i f ference equat ions .F o r s uc h p r o b le m s , it i s possible to extend the res ul ts of the "vector"d i sc re t e min imum p r inc ip led i s c r e t e m a t r i x mi n imu m p r i nc i p le ,

to obtain the equivalent f or m of the

Consider the d isc re te op t imizat ion prob lem defined by a s ys te m ofmatr ix d i f ference equat ions

Consider the sc al ar4 3 'with -kR I , zk Snm for all k, and -k E Scos t funct ionalN -1P

It i s a s s u me d t h at F (.), _K(.) , and L ( - ) sat i s fy the condit ions r e -qu i red by the d i sc re t e min imum p r inc ip le .-k k

Define the Hamiltonian function

w h e r e Zk s t he cos t a t e mat r ix .i s t he op t imal s t a t e , t hen the d i sc re t e m at r ix min imum p r inc ip l esta tes t hat t he re ex i s t s a cos t a t e mat r ix E:, k=O, 1, . . . N, such thatthe fol lowing relat ions hold

Tf U?; k=O, 1 : . . . N -1 i s the optimal control and Xz, k=O, 1 , . . . N,-k - A

( i) Canonical Equations:

p* p* = aH *kt1 -k- 8 -


13/23

-9 -(ii) Boundary Conditions :

At the initial "time" (k=O)x* = x-0 -0

At the terminal " t ime" (k=N)

(i i i) Minimization of the Hamiltonian:F or e v e r y ,V E R and each k=O, , . . . N-1

If the -k a re unconst ra ined then Eq. 3 3 yields the neces-s a ry condi tion


14/23

IV . JUSTIFICATION OF THE MATRIX MINIMUM PRINCIPLE

X l mx2 1x2 2. .X 2m. . .. . .X nm- -

The extension of the vector minim um principle to the m at r i x ca seF r o m a the oreti cal point of view it hinges on thes s t ra igh t forward .

existence of a mapping which rela tes the se t of n x m re al ma tr ic esto the se t of (nm)-d imensional vectors .

As befo re, l e t Snm denote the set of all r ea l nXm mat r i ces . L e tR denote the (nm) dimen sional Euc lidea n ve cto r spac e. Define(nm) into Rnm (nm)mapping z+b f r o m S

(nm)+b : S n m + Rs o that i f - E Snm i s the m at r ix

x =-of

(nm)hen the image X E R

dimensional column vector

l m12 ... x11x2 1 x22 . . . x m. . . . . . . . . . . . .X X ... xn l n2 nm

x =-

X under-

( 3 5)

the mapping z+b i s the (n m)-

( 3 )

- 10-


15/23

-11-It i s ea sy t o ve r i fy t ha t :

1) $(a) i s a l i nea r mapp i ng2 ) $(.) i s one-to-one and onto, hence $-l ex is ts3) $ ( - ) pr ese rve s t he i nne r p roduct because i f ,X, ,YE S andn m

5 , y E R s o that 5 = $(X),y $(z),hen the inner product

i s :(nm)while the inner product in Rn me , y > =

i = 1 j = 1( 3 9 )

and R ar e a lgebra ica l ly and topo-hus , the two spa ces Snmlogical ly equivalent . (nm)

In the continuous t ime case, one s t a r t s f r om t he ma t r i x d i f f e ren t ia lequation

Through the mapping~ $ this equat ion becomes- = J ( 5 , g ; t ) (42)

Sim ila r ly the in tegra nd of the cost funct ional L[,X, ,V, t]in to L[x, V, t] .t h e r e i s a cos t a t e vec t o r E E R

i s changedThen, by the ord inary vec tor minim um pr inc ip le ,L e t(nm)assoc ia ted with 5 E R(nm)


16/23

- 1 2 -

E =

Then the Hamil tonian

- -p11p12

P l mP21p2 2

P2m

...

...

. . .. . .Pnm- -

function in the vec tor

(43)

c a s e i sH = L ( x , U , t ) t < k , ~ > (44)

-1 PEnmince $ (.) ex ist s one ca n f ind a unique co state m at r i x

so that the Hamil tonian H can be wri t ten a s

ii i the riiatri;i 2862. Thus , t h e fact that ,!I prese rves the inner p roduc t(involve d in the definition of the Ham iltonian) coupled with the specificdefini t ion of the gradient m at r i ce s yields the m at r i x minimu m principlein the cont inuous - t ime ca se .

Caution: If - i s const ra in ed to be sy m m et r ic , then the mapping$ (* ) i s not i nver t ib l e .m a t r ic e s and the form ulae of Appendix A a r e n o t v a li d s o that the s ta te-me n t s in Sec t ions 2 and 3 mu st be modif ied in ord er to obtain the co r-r e c t a n s w e r s.

In th is ca se , the definitions of the gra dien t


17/23

V. APPLICATION TO A LINEAR CONTROL PROBLEMIn th is sect ion the ma t r ix min imum pr incip le i s used to de te rmine

the so lu t ion to the s imple op t imal l inear regula tor p rob lem.a l in ea r time-vary ing sy s te m with s ta te vector x(t)- ( t ) rel ate d by the vecto r differe nt ial equation

Considerand cont ro l vector

- ( t ) = - ( t ) s ( t ) t - -(t) u (t) (4 )where -( t ) i s a n n x n ma t r i x a nd ,B(t) a n n x r ma t r i x . C o n si d erthe quadrat ic cost funct ional

where -(t) and _R(t) ar e sym me tr ic posi tive defin ite m at r i ce s . Thes tan dard op t imization prob lem i s to find the c ontro l u ( t ) , t < t < T ,s o a s to minim ize the cos t functional J . 0- -

Instead of deal ing with this s tan dard problem , consider the fol-lowing vari at ion . Suppose that one imp ose s the constra int that thecon t ro l - ( t ) be g ene rate d by using a l in ea r t ime-va rying feedback lawof the form---I---.. P I + \ is 22 r ) in tirAe-y.rar)eng ~ l g ~ i f i l ~-a-trix !the e lemen t s ofl 1 G J . G _ u \ L I- ( t ) spe cify the t ime -varying feedback gains which mult iply the a p-p rop r i a t e s t a t e va r i ab les ) . In th is case , t he sy s t em sa t i s f ies t heclo se d -loop equation

- = [_AM-B(t)_G(t)l t )and the cost functional J reduces to

- 1 3 -


18/23

- 14-To complete the t ransform at ion of the prob lem in to the f rame work

requ i red by the ma t r ix min imum pr incip le , def ine the n x m "s ta tema t r i x ' ' z(t) as the ou ter vector p roduct o f the s ta te vector x(t )with i tself , i . e . ,

Noting that

E O ,F( t )x ( t ) = tr[_F(t)_X(t)] = tr[S(t)_F(t)] (54)i t fol lows f ro m Eqs . 52 and 50 that

TT - f tr[ Q!t)tG'It)_R(t)_G(t))g(t)]dt (58)" J

The sy st em (56) and the co st funct ional (58) a re in the fo rm r e -qu i re d to u se the mat r ix min imum p r inc ipl e . So l e t ,P(t) be then x n c o s t a t e ma t r i x a s s o ci a te d with z(t).H fo r t h i s p rob lem is*

The Hamiltonian function

H = tr[ -g t tr[ S ' R G X ]-- t tr [--cXPI] - tr[c---GXP'] t t r [-cA ' P ' ] - t r [--G'B'P']-(59 )

.l The t ime dependence i s suppresse d for s impl ic i ty


19/23

- 1 5 -The canonical equat ions yield (using the gradient m at r ix f orm ulae of Ap-pendix A)

The boundary condi t ions are- ( to) = % ( t o )x ' ( t o ) ; = 0

Since _G i s unconst r ai ned , i t i s nec es sa ry tha t

Note that both - ( t) and -( t ) a r e sym met r i c . To s ee t h i s , not e t ha tthe solu tion of Eq . 6 0 i s :

w h e r e -( t , to) i s the t ran s i t ion ma t r ix of [ _A(t)-B(t)G(t)]- . The sym-m e t r y of X ( t ) follows f ro m Eq. 64 and the s ym m etr y of X ( t ) . Asimilar a rgumen t can be used t o e s t ab l i sh the sym me t ry of _P(t).sym me t ry p rope r t i e s and Eq . 63 y ie ld

0- These

*If t hi s equation i s to hold for all z ( t ) , then one deduces_G(t) = _R-l(t)s'(t)_p(t) ( 66 )

To complete ly specify the gain m at r i x _G(t) one mu s t de t e rmi net he cos t a t e ma t r i x - ( t ) . By subst i tut ing Eq . 6 6 into E q. 61 one findst ha t t he cos t a t e ma t r i x -( t ) is the so lutio n of the fam il ia r Ri cca tim at r i x d i f fe ren t ial equa t ion

4 Th is i s the sam e argument tha t one us es in the vec tor case to obtainthe feedback solut ion; se e Ref. 8, p. 761.


20/23

-(T ) = 0 (68)I t should be c le ar tha t the ne ce ss ar y condit ions provided by the

m at r i x min imu m principle yield the sa me answe r that one would obtainin the vec tor fo rmulat ion . It i s , of course, well known that the answeri s indeed the unique optimal one.

The fact that the cos ta te m at r ix -( t) i s the solution of the Ric-cat i equat ion sheds some l ight in i t s physical in terpreta t ion .view, as requ i red by the Hamil ton-Jacobi -Bel lman theory , the cos ta tematrix as the grad ien t m at r ix of the cos t wi th resp ect to the s ta te , i . e . ,

If we

it is evident that the R icca ti equation defines the evolution of the p art ialde r iva t ives a J / a x . .(t) for tc[ to , T] .1Jr e a c h e d as rea di ly in the vector formulat ion of the problem.Th is conclusion cannot be


21/23

VI. CONCLUSIONSI t has been shown that sys tem s d esc r ibed by m at r i x d i f feren tia l

and difference equat ions can be opt imized by the ma tr i x vers ion of themin imu m principle of Pontryagin.m a t r i x of a scalar-valued funct ion of a ma t r ix f ac i l i ta t es the man ipu -lat ion of the nec es sar y condi tions fo r optimali ty a s i l lus t r a ted by thepro ble m of optimizing the gains of a l i ne ar sy st em .

The definition of the gra die nt

-17-


22/23

APPENDIX AA PARTIAL LIST OF GRADLENT M ATRICES

The formulae appea r ing below have been calculated in the u n -published repo r t by Athans and Schweppe.' Some of them have a l sobeen calculated by Kleinman using a di f feren t approach (Appendix F ofReference 5). The interes ted reader should consu lt these rep or t s fo rdeta i l s . The results a r e s ta ted in this appendix fo r th e sake of r e f e r -ence ; the calculations involved a r e s t ra igh t fo rwar d bu t leng thy .

In the formulae below ,X i s an n x m matr ix . The r eader i scautioned that th e formulae a re not val id i f the elements x ij of - a r enot independent..

- t r [X] = Lax --tr[AX] = ,A 'ax ---tr[ A X ' ] = ,Aax - -a

ax - - -

-

--tr[ A X B] = _A'_B'-aax ---r [ AX'B] = --A-

-tr[AX] = ,Aa g --aax! r[ AX!] = - '--

-tr[AXB] =El3a_xl ---- t r [AX'B] =,A'EJ'ax! --aax --r [ XX] 2 x 1-aax --~ [ x x I ]2~-

(A. 16)X X'- r [ e-] = e -ax-tr[ x-l]=-(X -1X -1) '= - (X-2 ) ' (A. 17)--ax -aax -- -tr[ AX-lB] =-(X-'BAX-')I--- (A , 18)-

(A. 20)log det [ X] =(X-')'ax - -adet[ AXB] =(det[AXB])(_X-')' (A .21)ax ----

--et[ Xn] =n(det[X])n(X-l)' (A . 23)a_x - - -- 18-


23/23

REFERENCES1 . P o n t r y a g i n , L . S . e t al. , The Math ema t ical T heo ry of Opt imalP ro ce ss es , New York , Wi ley , 1962.2 . A t ha n s, M . , a nd T s e , E . , "A Dir ect D er ivat ion of the Opt imalW E E T r a n s . o ni nea r F i l t e r Us ing t he Max i mum Pr i n c i p l e ,Auto ma tic Co ntro l , Vol. AC-12, No. 6 , December 1967, ( toa p p e a r ) .3 . Athans , M . , and Schweppe , F . C . , "On Optimal Wavefo rm De-s ign v ia Cont ro l Theo re t i c Concepts , ' ' Informat ion and Co nt ro l ,Vol. 10 , Apr il 1967, pp. 335-377.4 . Schweppe , F . C . , "Opt imizat ion of Signals , M. I . T . Linco lnLa bor a t o ry Repor t 1965-4 (unpub li shed) Lex i ng t on , M as s . , 1965 .5 . Kleinman, D. L . , "Subopt imal Design of L in ea r Re gul atorSys tem s Subjec t to Computer S torage Lim i ta t io ns , I Ph . D.Th es i s , Dept . o f E l ec t r i ca l Eng i nee ri ng , M. I . T . , Feb rua ry1 9 6 7; a ls o M . I . T . E l e c t r o n i c S y s t em s L a b o r a t o r y R e p o r tESL-R-297 , Cambr i dge , M ass . , Feb ru a ry , 1967 .6 . Schweppe , F . C . , "On t he Di s t ance Be tween G auss i an P r oc es se s ,subm i t ted to Informat ion and Cont ro l.7 . T se , E . , "App li ca ti on of Pon t r ayg i n ' s Mi n i mum Pr i nc i p l e t oF i l t e r in g P r o b l e m s , S . M. Th es i s , Dept. of Ele c t r i c a l Eng ine-

e r i n g , M . I . T . , C a m br id g e, M a s s . , June 1967.8 . A t h an s , M . , a n d F a l b , P . L . , Op timal Con trol , New York,McG raw Hil l Book Go. , 1966.9 . Athans , M . , and Schweppe , F. C. , "Grad i en t Ma t r i ce s andh . 4 3 t r i x Calc ii l at ions , M. I .T . Lincoln La b. T echnica l Note 1965-53 ,Lexington , M a ss . , Novem ber , 1965, (unpubli shed) .10 . Kle inman, D. L . , and Athans , M . , "The Dis cre t e MinimumPr in c ip le wi th Applica t ion to the Li ne ar R egula tor Prob lem ,M . I. T . E l e c t r o n i c S y s te m s L a b o r a t o r y R e p o r t E S L -R - 26 0 ,

Cambr i dge , Mass . , 1 966.11 . Hol tzman, J . M . , and H alkin, H . , "Direc t ional Convexi ty andM a x i m u m P r i n c i p l e f o r D i s c r e te S y s t e m s ,Vol . 4, No. 2, May 1966, pp. 263-275. J . SIAM on Control ,

12 . Ho l t zman , J . M ., "Convexity and the Maximum Pr in c ip le forD i s c r e t e S y s t e m s , ' IEEE T ran s . on Automat ic Cont ro l , Vol ,AC-11; Jan ua ry 1966, pp. 30-35.

athans, m. - the matrix mininum principle (nasa)

Documents