Introductory Lecture 2Suzanne Lenhart

University of Tennessee, Knoxville

Departments of Mathematics

Supported by NIH grant

Lecture2 – p.1/25

Salvage Term

� �� � ��� �� � � � � �� �� � �� �� � �� � ��

� � � � �� � � � � � � �� � � � �where

� �� �� � �

is final payoff. What change results?

� ��� � � � � �� � � �� �� � �� � � � � �� � � � �� � � � �

...� �� � �� � � � � � �� � � � �

same as before ! �� � � �� � �� � � � � � ��� � �� � � � �� � �� � � �

Lecture2 – p.2/25

Only change

" # $ % & #(' ) # $ $Example

* +, -.' / # $ 01

#324 ' 4 5 $6 2

#(' $ % -' / & % 78 '

" # $ % 78 ' ) # $

Lecture2 – p.3/25

Example

9 :�; <

Number of cancer cells at time

;

(exponential growth)State

= :�; <

Drug concentration Control

> 9>; ? @ 9 :�; <BA = :�; <

9 :C < ? 9ED known initial data

F GIH 9 :J <LK MD = N :�; < >;

where the first term is number of cancer cells at final time

J

and the second term is the harmful effects of drug on body.

Lecture2 – p.4/25

O P Q R S(T UWV P X

YY P O Z PV R O [

at P \ P \ O RZ

R ] O VY

Y U O V T R R O R_^ ` a b c

R S X O d

transversality conditionS U X O Ufe ] S U X O d

U S X g^ P Q S3h Xi h

here

S U X O Ufj

Lecture2 – p.5/25

Contd.

k l knm o p q rs k t u l v k m l o q w

k l o p q x r p w y

z { l | zW} ~ l | z } o p q x r p w y�

z { } | z l } o p q x r p w y�

� o p q r z � { l } o p � q r o q w�

z � t3� u l o q r z m o q w t o p q r } o q r u� |

Lecture2 – p.6/25

Well Stirred Bioreactor

Contaminant and bacteria present in spatiallyuniform time varying concentrations

� �3� � � concentration of contaminant

� �3� � � concentration of bacteria

bioreactor rich in all nutrients except one

� �3� � � concentration of input nutrient

bacteria degrades contaminant via co-metabolism.

Lecture2 – p.7/25

� ���� ��� ���� � ���� �B� � � ���� � � �

where

���� ��� ����� �� � ��� � � � � ���� � ���� �

where� �� �

is control and ��� �� ��� �are known.

Objective functional:

 ��� ��� ¡¢ � � ���� � � � ��� � �£ �

Find� ¤

to maximize

  ��� ¤ ��� ¥ ¦§  ��� �

maximize bacteria and minimize input nutrient cost.Lecture2 – p.8/25

¨ ©3ª « ¬ ¨®­ ¯° ± ²³

­ ´ ©(µ «¶ µ

³­ ´ ©µ «¶ µ ¬ ² ·¹¸ ¨ © «¨3­

º ©(» «

penalizes large values of ¨ at final time .

Can eliminate ¨ variable and work with ´ ©ª «

only.

Lecture2 – p.9/25

¼ ½W¾ ¿ À ¿ ½¿ ¾ ½ Á

ÂÂ ¿ ¼ ¾ Ã À ½ ÂÂ ¿

¿ ½¿ ¼ Ä

at ¿ Å

¾ à À ½Æ ¿ Ç Á ¼ Ä À ½ ¼ Æ ¿ Ç Á

Æ À ½ Ç È É Á ¼ ¿

¿ Å ¼ Æ À ½ Ç È É Á ¾

Lecture2 – p.10/25

Ê ËfÌ Í ÎÏÎÑÐ Ì Í Ê ÒÔÓÏ Õ Ó Í Ö × Ð Õ Ø

Ê ÙÚ Û Ì ÜÊ ËfÌ Í Ê Ò Ý Ù Ê Ð Ò Ï Û Þ ßà Í Ï á

Ï Õ â Ù Ê Ð Ò Ï Û Þ ßà Í Ï ã Í Ö × Ð Õ Ø

Ð ËäÌ Ò Ý Ù Ê Ð Ò Ï Û Þ ßà Í Ï á

Ï Õ Ù Ê Ð Ò Ï Û Þ ßà Í Ï Í × Ð à

Ð Ù Ü Û Ì ÐEå known æSolve for Ðèç Ê

numerically.

Lecture2 – p.11/25

Problems

é ê ë ìí î ï ð ñò óWhat if:

ìí î ï ð ñò ë ô õ

í î ô õ

ìí î ï ð ñò ó ö ô õ

Need additional constraintô é ì3÷ ï ø

Lecture2 – p.12/25

Fishery Model

ù úfû üù ý þù ÿ þ �ù

ù ý�� ÿ

population level of fish

� ý�� ÿ

harvesting control

Maximizing net profit:

�� � � �� � � �ù þ ý �ù ÿ þ � � � � ��

where � � �� is discount factor, ��� � � � terms represent profit

from sale of fish, diminishing returns when there is a large

amount of fish to sell and cost of fishing. � � � � � � are

positive constants.Lecture2 – p.13/25

Contd.

� � � �� ��� � �� � � � �� ! � � " � � #

$ � � � ! � �� !

$ % � �&

&� � � ' � � �� � � � � ( � � � � � !

$ � ( � � � !)

&& � � � � �� � � � � ( � � �� � � " � ! $ � � ! � *

� + � � $ � + � � �� � � � + � " � !

( � � �� � � � + ! �

Lecture2 – p.14/25

Contd.

Solve for , -. / -. 0

numerically.

Need control bounds

1 , 243 5 687Ref:B D Craven bookControl and Optimization

Lecture2 – p.15/25

Interpretation of Adjoint

9 :;<=?>

=?@A4BC DC E FG B H A D8I C B I F

( Definition of value function )

D J K L A4BC DC E F

D AB I F K D I

MM D

A D I C B I F K N A4B I F

OP 9Q R IA D I SC B I F T A DUI C B I F

S

Units: money/unit item in profit problems.Lecture2 – p.16/25

V W4XZY [

= marginal variation in the optimal objectivefunctional value of the state value at

X Y .“Shadow price”\ additional money associated with addi-tional increment of the state variable

]]_^

W ^ ` WX [a X [ b V W4X [for all

X Y X Xdc

“If one fish is added to the stock, how much is thevalue of the fishery affected ?"

Lecture2 – p.17/25

ee_f

g f8h i jh k l m g jh kApproximate

g f h n i jh k o g f4h i jh k

n p m g jh k

g f h n i jh k p g f h i jh k m g jh k

New value Original value + adjoint

Lecture2 – p.18/25

Principle of Optimality

If q rs t r

is an optimal pair onuZv u uZw and

uZv xu uZw ,then q rs t r

is also optimal for the problem onxu u uw :

yz{|}�~

�}� us ts q �� u t � � � � us ts q �

t � xu � � t r � xu �t

x 0

t 0 t t 1

(t, x*(t ))

x

Lecture2 – p.19/25

Existence of Optimal Controls

“Sufficient conditions to guarantee existence ofOC"Suppose � �� � �� �

satisfy

� � � � �4� � �� � � � �4�Z� � � ��

� � � � �� � �� � � �4�Z� � � �

is maximized w.r.t. � at � �

plusset of controls compact

� � jointly concave in � and �

bounded state functions

Lecture2 – p.20/25

For details about existence of OC seeMacki and Strauss book

Fleming and Rishel book

Back to exercise example

��

�Z� � �� �

� �   ¡ ¢ � £ � �¥¤ �   ¡

To guarantee the maximum value of

¦ � � � wouldbe finite, need a priori bound on state � , control �.

Lecture2 – p.21/25

Optimality System

State system coupled with adjoint system- optimal control’s expressions substituted in

Uniqueness of Optimality System § Uniqueness ofOptimal Control

Uniqueness of Optimality System - only for small time

¨

due to opposite time orientations

BUT Uniqueness of Optimal Control © Uniqueness ofSolutions of Optimality System

To get uniqueness of OC directly,

need strict concavity of

ª «­¬¯® ° «¬ ± ±

.

Lecture2 – p.22/25

Optimality System

State system coupled with adjoint system- optimal control’s expressions substituted in

Uniqueness of Optimality System - only for small time

²

due to opposite time orientations

Numerical Solutions by Iterative Method- with Runge Kutta 4, Matlab or favorite ODE solver

(Characterization of OC non-smooth)- guess for controls, solve forward for states- solve backward for adjoints- update controls, using characterization- repeat forward and backwards sweeps and controlupdates until convergence of iterates

Lecture2 – p.23/25

Idea of Runge Kutta

Give handouts.

Lecture2 – p.24/25

2 BC on one state

For Lab example

Suppose ³ ´¥µ ¶ · ³8¸ and ³ ´ ¶ · ³8¹are BOTH GIVEN

Then

º

doesnot have a boundary condition.

Needs a type of shooting method.

Lecture2 – p.25/25