the solution classical feedback optimal control …file.scirp.org/pdf/ojop_2013032914423399.pdf ·...

Open Journal of Optimization, 2013, 2, 16-25 http://dx.doi.org/10.4236/ojop.2013.21003 Published Online March 2013 (http://www.scirp.org/journal/ojop)

The Solution Classical Feedback Optimal Control Problem for m-Persons Differential Game with Imperfect

Information

Jaykov Foukzon1, Elena Men’kova2, Alex Potapov3 1Department of Mathematics, Israel Institute of Technologies, Haifa, Israel

2All-Russian Research Institute for Opto-Physical Measurements, Moscow, Russia 3V.A. Kotel’nikov Institute of Radio Engineering and Electronics, Russian Academy of Sciences, Moscow, Russia

Email: [email protected], [email protected], [email protected]

Received January 12, 2013; revised February 11, 2013; accepted March 5, 2013

ABSTRACT

The paper presents a new approach to construct the Bellman function ,t x and optimal control directly by

way of using strong large deviations principle for the solutions Colombeau-Ito’s SDE. The generic imperfect dynamic models of air-to-surface missiles are given in addition to the related simple guidance law. A four examples have been illustrated, corresponding numerical simulations have been illustrated and analyzed.

,u t x

Keywords: Optimal Control; Bellman Equation

1. Mathematical Challenge: Creating a Game Theory That Scales

What new scalable mathematics is needed to replace the traditional Partial Differential Equations (PDE) approach to differential games?

Let be a probably space. Any stochastic process on is a measurable mapping

, , d

: 0,X T

,u t x

n . Many stochastic optimal control problems essentially come down to constructing a function that has the properties

1) ,, inf ,xt Du t x E J X t ,

2) , ' 0,, inf x

s D ,s t

u t x E J X

',' 0,, ,x

t Ds t',s u t x t t

n

,t U U , where J is the termination payoff

functional, is a control and is some t , 0

xt D t

x

Markov process governed by some stochastic Ito’s equation driven by a Brownian motion of the form

3) , ,0, d ,

tx xt D s Dx x g x t s DW t ,

where ,W t is the Brownian motion. Traditionally the function has been computed by way of ,u t xsolving the associated Bellman equation, for which vari- ous numerical techniques mostly variations of the finite difference scheme have been developed. Another ap-

proach, which takes advantage of the recent develop- ments in computing technology and allows one to construct the function ,u x t by way of backward induc- tion governed by Bellman’s principle such that described in [1]. In paper [1] Equation (3) is approximated by an equation with affine coefficients which admits an explicit solution in terms of integrals of the exponential Brow- nian motion. In approach proposed in paper [2,3] we have replaced Equation (3) by Colombeau-Ito’s Equation (4)

, ,, , , ,0

, ,

, ,

tx xt D s D , d

,

x x g x t s

D w t w t

', 0,1 , 1, 2 , where ,w t is the

white noise on n , i.e., d,

dw t W t

t, almost

surely in , and 'D ' ,w t is the smoothed white noise on n i.e.,

' ', , ,w t w t s t ,

and ' is a model delta net [2,4]. Fortunately in con-trast with Equation (3) one can solve Equation (4) with-out any approximation using strong large deviations principle [4]. In this paper we considered only quasi sto-chastic case, i.e. 0, 0D . General case will be con-sidered in forthcoming papers.

Statement of the novelty and uniqueness of the pro-

Copyright © 2013 SciRes. OJOp

J. FOUKZON ET AL. 17

posed idea: A new approach, which is proposed in this paper allows one to construct the Bellman function

and optimal control directly, i.e., without any reference to the Bellman equation, by way of using strong large deviations principle for the solutions Colombeau-Ito’s SDE (CISDE).

,V t x ,t x

2. Proposed Approach

Let us consider an m-persons Colombeau-Ito’s differential game ; , , , n

m TCDG f g y G with a stochastic nonlinear dynamics:

, , ,

, , , 0,1

x t f t x t Dw t t

w t

;

00, : , 0 , ,nt T x t x x f f

, , ,ng g f g G (1)

1 , , , , 1, , ,ikm i it t t t U i m

and m-persons Colombeau-Ito’s differential game

with imperfect infor-

mation about the system [5-8]:

; , , , ,nm TCIDG f g y G t

, , , , ,

, , , 0,1

x t

f t x t Dw t t t t

w t

;

00, : , 0 , ,nt T x t x x f f

, , ,ng g f g G (2)

1 , , , , 1, , ,ikm i it t t t U i m

Here is the algebra of Colombeau generalized functions [9], is the ring of Colombeau’s generalized numbers [10-12],

nG

n ; it t is the control chosen by the i-th player, within a set of ad-missible control values , and the playoff for the i-th player is:

iU

1 2

1 2

, , ,10

1

2

,1

, , ,

, , d

T

i i

m

n

i ii

, ;nJ E E g t x t x t

t t t

E E x T y

. (3)

where is the trajectory

of the Equation (1). Optimal control problem for the i-th player is:

,1 , ', , nt x t x t

,

, min maxi j t j i

it

J ,iJ

. (4)

Let us consider now a family ,tx

of the solu-

tions Colombeau-Ito’s SDE:

, ,

,0 0

, ;

, 0, , , 0,1

t t

n

d x b x t dW t

x x t T

(5)

where W t is n-dimensional Brownian motion,

0, , :n nb t b G n is a polynomial,

i.e.

0, 0, 1

0

, ,

, 1, , .

ii k

k

jj

b x t b t x i i

i i n

, , ,

Definition 1. CISDE (5) is -dissipative if exist Lyapunov candidate function and Colombeau

,V x tconstants 0,C C

r r , such that:

1) 0,1 , 0,1 , , :x x r

, ; , ;V x t b C V x t

,1

, ,, ; ,

n

ii i

V x t V x tV x t b b x

t x

2) lim infrr x r

V x V x

.

Theorem 1. Main result (strong large deviations principle) [5,13]. For any solution t 1, ,, ,t t nx x x of dissipative CISDE (5) and valued parameters 1, , n , there exist Colombeau constant

, 0C C R C

,

such that 1, , , n :

2 2,

0liminf ,tE x C U t

. (6)

where a function 1, , , , nU t U t U t, is the solution of the master equation:

0, , , , 0,U t J b t U b t U x , (7)

where ,J J b t the Jacobian, i.e. J is a n n - matrix:

0,, ,i jJ b t b x t x x .

Remark.1. We note that 0,1 :

, 0,

0lim inf 0t tt E x x

.

Example 1.

3 2

0

sin

, 1,0 , 0 , 0,

n mt t t t

kx a x b x cx t t t

W t a x x t T

.



From a general master Equation (7) one obtain the next linear master equation:

2 3

0

3 2

sin , 0n m k

u t a b c u t a b c

t t t u x

2

. (8)

From the differential master Equation (8) one obtain transcendental master equation

20

3 2

0

2

exp 3 2

sin

exp 3 2 d 0

tn m

x t a t b t t

a t b t

a t b t t

. (9)

Numerical simulation: Figures 1 and 2.

0

3 20 0 0 0

0

1, 5, 1, 2, 2, 0,

5, 0.01,

sin , 0 , 0,

nt t t t

m k

a b c m n x

T R T

x a x b x cx t

t t x x t T

. (10)

Here 20

0lim inf .t tt E x x

Let us consider now

an m-persons Colombeau stochastic differential game

with nonlinear dynamics ; , , , nm TCDG f g y G

, ,

, 0

1

1

, , , ,

: , 0 , 0, , 0,1

, , , , 1, , .

, , , , 1, , .

i

i

n

km i i

km i i

;

, 1,

x t f t x t t dW

t x t x x t T

t t t t U i m

t t t t U i m

t

(11)

Here , n it t is the control chosen by the i-th player, within a set of admissible control val-ues , and the playoff of the i-th player is iU

, , ,0

2

, ;1

, ; d

,

T

i i

n

i ii

J E g x t t t

E x T y

(12)

where 1, , ny y y and ,t x t is the trajec-tory of the Equation (11).

Theorem 2. For any solution

,

, ,1, , 1

,

, , , , ,

t

t n t m

X t

X X t

t

of the dissipative and ; , 0, , nm TCDG f y G val-

Figure 1. The solution of the Equation (8) in a comparison with a corresponding solution x t of the ODE (10).

Figure 2. r versus R.

ued parameters 1, , n , there exists Colombeau con-

stant ,C C C 0, such that:

1

2 2,

0

, , ,

liminf ,

n

tE x C U t

. (13)

where 1, , , , nU t U t U t, the trajectory of the corresponding master game

0

,

, , , , , 0,

,

U t

J f t U f t U

x

2

iJ U T (14)

Example 2. 1) 3

1 2 2 2 1 2, ,x x x kx t t k 0,

1 1 1 2 2, , ,t t 2 ,

1 01 2

2 21 2

0, , 0 , 0 ;

, 1, 2;i

t T x x x x

J x T x T i

02

optimal control problem for the first player:

1 1 2 2 2 2

2 21 2

, ,min max

t tx T x T

and optimal control problem for the second player:

1 1 12

2 21 2

,,max min .

u u ttx T x T

From Equation (14) we obtain corresponding master game:

2) 2 31 2 2 2 2 2 2 1 2, 3u u u k u k t t ,



1 1 1 2 2 2 1 01

2 22 02 2 1 2

, , , , 0

0 ; , 1, 2;i

t t u

u x J u T u T i

1,x

optimal control problem for the first player is:

1 1 2 2 2 2

2 21 2

, ,min max

t tu T u T

and optimal control problem for the second player is:

1 1 12

2 21 2

,,max min .

u u ttu T u T

Having solved by standard way [14,15] linear master game (2) one obtain optimal feedback control of the first player:

1 1 1 2

1 1 2

, ,

sign

t t x t x t

x t t x

t

and optimal feedback control of the second player [5]:

2 2 1 2

2 1 2

, ,

sign .

t t x t x t

x t t x t

Here , ,t t t t

ceil 1 ,t

t t

where ceil x is a part-whole of a number x .

Thus, for numerical simulation we obtain ODE:

1 2 ,x t x t

32 2 1 1 2

2 1 2

sign

sign .

x t kx t x t t x t

x t t x t

Numerical simulation: Figures 3-6

1

1 2

22

1, 400, 100, 5,

0 300 m, 0 30 m sec,

80 sec, sin .

k A

x x

T t A t

Theorem 3. For any solution

,

, ,1, , 1

,

, , , , ,

t

t n t m

X t

X X t

t

Figure 3. Optimal trajectory: 1 1 0.4 mx t x T .

Figure 4. Optimal velocity: /2 2 0.4 m secx t x T .

Figure 5. Optimal control of the first player.

Figure 6. Control of the second player.

of the dissipative ; , 0, , ,nm TCIDG f y G and

valued parameters 1, , n , there exists Colombeau constant ,C C C

0, such that:

2 2

0lim inf ,tE x C U t

. (15)

where 1, , , , nU t U t U t, the trajectory of the corresponding master game

0

, , ,

, , , 0,

,

U t J f w t t t U

f w t t t U

x

(16)

1

2

, , , , 1, , ,

.

ikm i i

i

t t t t U R i

J U T

m

Example 3. Game with imperfect measurements.



1) 1 2 ,x t x t

3 22 1 2 2 2

1 1 2

2 1 2 1 2

1 1 1 2 2 2

, ,

, , , 0, ,

, , , ,

x t k x t k x t

t x t x t t

t x t x t k k

t t

,

2 21 1 , 1, 2iJ x T x T i . From Equation (16) one

obtain corresponding master game: 2) 1 2 2 ,u u

2 32 1 2 2 2 2 1 2 2

1 1 2

2 1 2

1 1 1 2 2 2

2 21 2

3 2

, ,

, , ,

, , , ,

, 1, 2.i

u k k u k k

t u t u t t

t u t u t

t t

J u t u t i

22

Having solved by standard way linear master game (2) one obtain local optimal feedback control of the first player [5]:

1 1 1 1 2sign nt x t t t x t t

t

and local optimal feedback control of the second player:

2 2 1 1 2sign .nt x t t t x

t

t

Thus, finally we obtain global optimal control of the next form [5]:

1 1 1 2

2 2 1 2

sign ,

sign .

t x t t x t

t x t t x t

Here , ,t t t

1 ,t

t t ceil

where is a part-whole of a number ceil x x . Thus, for numerical simulation we obtain ODE: 1 2 ,x x

2 1 2 2 2 1 1 2

2 1 2 .

3 2x k x k x sign x t t x t t

sign x t t x t

Numerical simulation: Figures 7-12. Game with imperfect measurements: red curves 1 2,x t x t . Classical game: blue curves 1 2, .y t y t t 2sint A .

3. Homing Missile Guidance with Imperfect Measurements Capable to Defeat in Conditions of Hostile Active Radio-Electronic Jamming

Homing missile guidance strategies (guidance laws) dic-tate the manner in which the missile will guide to inter-cept, or rendezvous with, the target. The feedback nature of homing guidance allows the guided missile (or, more

generally, the pursuer) to tolerate some level of (sensor) measurement uncertainties, errors in the assumptions used to model the engagement (e.g., unanticipated target maneuver), and errors in modeling missile capability (e.g., deviation of actual missile speed of response to guidance commands from the design assumptions). Nev-ertheless, the selection of a guidance strategy and its subsequent mechanization are crucial design factors that can have substantial impact on guided missile perform-ance. Key drivers to guidance law design include the type of targeting sensor to be used (passive IR, active or semi-active RF, etc.), accuracy of the targeting and iner-tial measurement unit (IMU) sensors, missile maneuver-ability, and, finally yet important, the types of targets to be engaged and their associated maneuverability levels.

Figure 13 shows the intercept geometry of a missile in planar pursuit of a target. Taking the origin of the refer-ence frame to be the instantaneous position of the missile, the equation of motion in polar form are [16]:

2 , , ,

, , , .

2 , ,

, , ,

r rM T

r r r r rM M M T T T

n nM T

n n n n nM M M T T T

R R a t R t R t a t

a t a a a t a a

R R a t t t a t

a t a a a t a a

,

.

r

n

(17)

1) The variable R R t denotes a true target-to- missile range TM

2) The variable R t .

R R t denotes the it is real meas-ured target-to-missile range . TMR t

3) The variable t denotes a true line-of-sight angle (LOST) i.e., the it is true angle between the con-stant reference direction and target-to-missile direction.

4) The variable t denotes the it is real meas- ured line-of-sight angle (LOSM) i.e., the it is true angle between the constant reference direction and target-to- missile direction.

5) The variable denotes , ,n nM Ma t a t R t R t

the missiles acceleration along direction which perpen-dicularly to line-of-sight direction.

6) The variable , ,r rM Ma t a t t t

denotes

the missile acceleration along target-to-missile direction. 7) The variable n

Ta t denotes the target acceleration along direction which perpendicularly to line-of-sight direction.

8) The variable rTa t denotes the target acceleration

along target-to-missile direction. Using replacement z R into Equation (17) one

obtain:

2

, , ,

, , ,

r rM T

r r r r rM M M T T

zR a t R t R t a t

R

a t a a a t a a

.rT

(18)



Figure 7. Uncertainty of speed measurements t .

Figure 8. Cutting function . 11. 10t

Figure 9. Optimal trajectory.

Figure 10. Optimal velocity.

Figure 11. Optimal control of the first player.

Figure 12. Optimal control of the second player.

Figure 13. Planar intercept geometry.

, , ,

, , ,

r nM T

n n n n nM M M T T

Rzz a t z t z t a t

R

a t a a a t a a

.nT

(18)

,

.

z t R t t

z t R t t R t t

Suppose that:

1 2, .R t R t t t t t

Therefore



1 1 1, .R t R t t R t t t 1

2 2, .t t t t t t

1 2

1 2

1 2 2

2 ,

z t R t t R t t t t

R t t t t t R t t

z t t t t R t t

z t t

2

2 1 2 2 .t t t t R t t (20)

1 2

1 2

1 2

2 2 1 2

2 2 1 2

3

3 2 2 1 2

,

.

z t R t t R t t

R t t t t

R t t t t

R t t R t t t t t

R t t R t t t t t

z t R t t R t t t t t

z t t

t R t t R t t t t t

Let us consider antagonistic Colombeau differential game 2

2; ,0, , , , ,TIDG f y G w 1 2 3 4, , , ,

2 4,w with non-linear dy-namics and imperfect measurements [6]:

2

31

1 2

,

,

, ,

; ,

, , , .

r

r rr M T

r rM M r r

r r

r r r r r rM M M T T T

R V

zV a t a t

R

a t a t R t V t k V t

R t R t t V t V t t

a t a a a t a a

,z w (21)

3

2

3 4

21

,

, ,

, ,

, , ,

, 1,2.

n nrM T

n nM M

n n n n nM M M T T T

i

V ww a t a t

R

a t a t w t w t k w t

z t z t t z t z t t

a t a a a t a a

J R t i

,

.n

Optimal control problem of the first player is:

1, , ,

21

, , ,

min

max .

r r n n nM M M M M M

r r r n n nT T T T T T

S t a a a t a a

a t a a a t a a

J

R t

(22)

2 max, , ,

21

, , ,min

Optimal control problem of the second player is:

.

r r r n n nJ

T T T T T T

r r r n n nM M M M M M

a t a a a t a a

a t a a a t a aR t

(23)

From Equations (21)-(23) one obtain corresponding linear master game:

2 ,rr v2 3

1 2 1 2

1 1 2 2

3 ,

, , ,

; ,

, , , .

r rr r M T

r rM M r

r r

r r r r r rM M M T T T

v k v t k a t a t

a t a t r t v t

r t r t t v t v t t

a t a a a t a a

(24)

2 31 2 3 1 2 3

3 1 3 1 4

21

3 ,

, , ,

,

, , , ,

, 1,2.

n nM T

n nM M

n n n n n nM M M T T T

i

z k z t k a t a t

a t a t z t z t

z t z t t z t z t t

a t a a a t a a

J r t i

From Equation (24) we obtain quasi optimal solution for the antagonistic differential game

22; ,0, , , ,TIDG f y G w given by Equations (21)-

(23). Quasi optimal control ,r of the first M Mt ttrol player and quasi optimal con of the

second player are:

signr rt

t t,rT T

1

3

2 1 2

3

3

4 2 4

sign

.

M M

r r

n nM M

R t t

t V t t k V t t

t z t t

t z t t k z t t

(25)

t

t

1 2

3

1 2

3 4

3

2 4

ˆ ˆsign

ˆ ,

ˆ ˆsign

ˆ .

r rT T r

r

n nT T

t R t t t V t

k V t t

t z t t t z t

k z t t

Thus, for numerical simulation we obtain ODE:

2

1

3

2 1 2

3

3

4 2 4

,rV

sign

sign

.

rr M

rr r T

nrM

nT

R

zV R t t

R

t V t t k V t t a t

V zz z t t

R

t z t t k z t t a t



Example 4: Figures 14-24.

3 21 2

2

0.001, 10 , 0.001, 20m sec ,

20 m sec , 0 200 m, 0 10 m sec,

0 60, 0 40, sin ,

sin , 50,

sin ,

rT

T r

pr rT T

q

T T

p

k k a

a R V

z z a t a t

a a t

w t t t

20, 2, 1.p q

4. Conclusions

Supporting Technical Analysis: Let us consider optimal control problem from Example 1, corresponding Bellman type equation is:

Figure 14. Cutting function:

t .

Figure 15. Uncertainty of measurements of a variable

:R t t .

Figure 16. Target-to-missile range R t . 37.2 10 m R t .

1 1 2 2 2 2

32 2 1 2

, ,1 2

min max

0,

V V Vx x

t x x

2 21 2, ,V T x x x t T 0, (27)

Figure 17. Speed of rapprochement missile-to-target: R t .

Figure 18. Variable z t R .

Figure 19. Variable 2.172z T . z t



0 0.3 t . Figure 20. Variable

Figure 21. Missile acceleration along target-to-missile direction:

rMa t .

Figure 22. Missile acceleration along direction which perpendicularly to line-of-sight direction: n

Ma t .

Figure 23. Target acceleration along target-to-missile direction: r

Ta t .

Figure 24. Target acceleration along direction which perpendi larly to line-of-sight direction: na t . cu

Complete constructing the exact analytical solution for

PDE (27) is a complicated unresolved classical problem, because PDE (27) is not amenable to analytical treat-ments. Even the theorem of existence classical solution for boundary Problems such (27) is not proved. Thus, even for simple cases a problem of construction feedback optimal control by the associated Bellman equation com-plicated numerical technology or principal simplification is needed [17]. However as one can see complete con-structing feedback optimal control from Theorems 1-2 is simple. In study [6], the generic imperfect dynamic mod-els of air-to-surface missiles are given in addition to he related simple guidance law.

[1]

T

t

REFERENCES A. Lyasoff, “Path Integral Methods for Parabolic Partial

Differential Equations with Examples from Computational Finance,” Mathematical Journal, Vol. 9, No. 2, 2004, pp. 399-422.

[2] D. Rajter-Ciric, “A Note on Fractional Derivatives of Colombeau Generalized Stochastic Processes,” Novi Sad Journal of Mathematics, Vol. 40, No. 1, 2010, pp. 111- 121.

[3] C. Martiasa, “Stochastic Integration on Generalized Func- tion Spaces and Its Applications,” Stochastics and Sto- chastic Reports, Vol. 57, No. 3-4, 1996, pp. 289-301. doi:10.1080/17442509608834064

[4] M. Oberguggenberger and D. Rajter-Ciric, “Stochastic Differential Equations Driven by Generalized Positive Noise,” Publications de l’Institut Mathématique, Nouvelle Série, Vol. 77,

[5] J. Foukzon, “T nd Quantum Feed-

ith Imperfect Measurements and Imperfect

No. 91, 2005, pp. 7-19.

he Solution Classical aback Optimal Control Problem without the Bellman Equation,” 2009. http://arxiv.org/abs/0811.2170v4

[6] J. Foukzon and A. A. Potapov, “Homing Missile Guid- ance Law wInformation about the System,” 2012. http://arxiv.org/abs/1210.2933


J. FOUKZON ET AL.


25

Class of Stochastic Dynamical Games with I[7] P. Bernhard and A.-L. Colomb, “Saddle Point Conditions

for a mper- fect Information,” IEEE Transactions on Automatic Con- trol, Vol. 33, No. 1, 1988, pp. 98-101. http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=367

[8] A. V. Kryazhimskii, “Differential Games of Approach in Conditions of Imperfect Information about the System,” Ukrainian Mathematical Journal, Vol. 27, No. 4, 1975, pp. 425-429. doi:10.1007/BF01085592

[9] J. F. Colombeau, “Elementary Introduction to New Gen- eralized Functions,” North-Holland, Amsterdam, 1985.

[10] J. F. Colombeau, “New Generalized Functions and Mul- tiplication of Distributions,” North-Holland, Amsterdam, 1984.

[11] H. Vernaeve, “Ideals in the Ring of Cized Numbers,” 2007. http://arx

olombeau Geiv.org/abs/0707.0698

an, “On Optimal Guidance for Homing Missiles,”

neral-

[12] E. Mayerhofer, “Spherical Completeness of the Non- Archimedian Ring of Colombeau Generalized Numbers,” Bulletin of the Institute of Mathematics Academia Sinica

(New Series), Vol. 2, No. 3, 2007, pp. 769-783.

[13] J. Foukzon, “Large Deviations Principles of Non-Freidlin- Wentzell Type,” 2008. http://arxiv.org/abs/0803.2072

[14] S. GutmJournal of Guidance and Control, Vol. 2, No. 4, 1979, pp. 296-300. doi:10.2514/3.55878

[15] V. Glizer and V. Turetsky, “Complete Solution of a Dif- ferential Game with Linear Dynamics and Bounded Con- trols,” Applied Mathematics Research Express, Vol. 2008, 2008, p. 49. doi:10.1093/amrx/abm012

[16] M. Idan and T. Shima, “Integrated Sliding Model Auto- pilot-Guidance for Dual-Control Missiles,” Journal of Guidance, Control and Dynamics, Vol. 30, No. 4, 2007, pp. 1081-1089. doi:10.2514/1.24953

[17] W. Cai and J. Z. Wang, “Adaptive Wavelet Collocation Methods for Initial Value Boundary Problems of Nonlin- ear PDE’s,” Pentagon Reports, 1993. http://www.stormingmedia.us/44/4422/A442272.html10.1093/amrx/abm012

http://dx.doi.org/10.2514/3.55878

http://dx.doi.org/10.2514/3.55878

the solution classical feedback optimal control …file.scirp.org/pdf/ojop_2013032914423399.pdf ·...

Documents