weak dynamic programming principle toward viscosity solutions · 2013-04-16 · weak dynamic...
TRANSCRIPT
![Page 1: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/1.jpg)
Weak dynamic programming principle towardviscosity solutions
Bruno BouchardCeremade and Crest
University Paris Dauphine and ENSAE
Ann Arbor 2011
Joint works with M. Nutz and N. Touzi
![Page 2: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/2.jpg)
Motivation
Provide an easy to prove Dynamic Programming Principle forstochastic optimal control problems in standard form :
v(t, x) := supν∈U
F (t, x ; ν) with F (t, x , ν) := E[f (X ν
t,x(T ))].
Weaker than the usual one, but just enough to provide the usualPDE characterization.(joint work with N. Touzi - SIAM Journal on Control andOptimization, 49 (3), 2011)
Extend it to optimal control problems :- with constraints in expectation : E
[g(X ν
t,x(T ))]≤ m.
- with strong state constraints : X νt,x ∈ O on [t,T ].
(joint work with M. Nutz - preprint)
![Page 3: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/3.jpg)
Motivation
Provide an easy to prove Dynamic Programming Principle forstochastic optimal control problems in standard form :
v(t, x) := supν∈U
F (t, x ; ν) with F (t, x , ν) := E[f (X ν
t,x(T ))].
Weaker than the usual one, but just enough to provide the usualPDE characterization.(joint work with N. Touzi - SIAM Journal on Control andOptimization, 49 (3), 2011)
Extend it to optimal control problems :- with constraints in expectation : E
[g(X ν
t,x(T ))]≤ m.
- with strong state constraints : X νt,x ∈ O on [t,T ].
(joint work with M. Nutz - preprint)
![Page 4: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/4.jpg)
The case without constraint
v(t, x) := supν∈U
F (t, x ; ν) with F (t, x , ν) := E[f (X ν
t,x(T ))]
Weak Dynamic Programming Principle for Viscosity Solutions, with NizarTouzi, SIAM Journal on Control and Optimization, 49 (3), 2011.
![Page 5: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/5.jpg)
Standard approach for the DPP
Either use a measurable selection argument : (t, x) 7→ νε(t, x) ∈ Usuch that
F (t, x ; νε(t, x)) ≥ v(t, x)− ε
Or use the regularity of v and F (·; ν) to construct one :
F (·; νti ,xiε ) ≥
lscF (ti , xi ; ν
ti ,xiε )− ε ≥ v(ti , xi )− 2ε ≥
uscv − 3ε on Bi
with (Bi )i≥1 a partition of the state-space. Then, one constructs ameasurable selection by setting
νε(t, x) :=∑i≥1
νti ,xiε 1Bi (t, x) .
![Page 6: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/6.jpg)
Standard approach for the DPP
Either use a measurable selection argument : (t, x) 7→ νε(t, x) ∈ Usuch that
F (t, x ; νε(t, x)) ≥ v(t, x)− ε
Or use the regularity of v and F (·; ν) to construct one :
F (·; νs,yε ) ≥
lscF (s, y ; νs,y
ε )− ε ≥ v(s, y)− 2ε ≥usc
v − 3ε on Bs,y
with (Bi )i≥1 a partition of the state-space. Then, one constructs ameasurable selection by setting
νε(t, x) :=∑i≥1
νti ,xiε 1Bi (t, x) .
![Page 7: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/7.jpg)
Standard approach for the DPP
Either use a measurable selection argument : (t, x) 7→ νε(t, x) ∈ Usuch that
F (t, x ; νε(t, x)) ≥ v(t, x)− ε
Or use the regularity of v and F (·; ν) to construct one :
F (·; νti ,xiε ) ≥
lscF (ti , xi ; ν
ti ,xiε )− ε ≥ v(ti , xi )− 2ε ≥
uscv − 3ε on Bi
with (Bi )i≥1 a partition of the state-space. Then, one constructs ameasurable selection by setting
νε(t, x) :=∑i≥1
νti ,xiε 1Bi (t, x) .
![Page 8: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/8.jpg)
Standard approach for the DPP
In both cases : for ν = ν1[0,θ) + 1[θ,T ]νε(θ,X νt,x(θ))
v(t, x) ≥ F (t, x ; ν) = E[E[f (X
νε(θ,Xνt,x (θ))
θ,Xνt,x (θ) (T )) | Fθ]]
= E[F (θ,X ν
t,x(θ); νε(θ,X νt,x(θ)))
]≥ E
[v(θ,X ν
t,x(θ))]− 3ε
Most of the time, proofs are based on a regularity argument :
F (·; νti ,xiε ) ≥
lscF (ti , xi ; ν
ti ,xiε )− ε ≥ v(ti , xi )− 2ε ≥
uscv − 3ε
on Bi 3 (ti , xi ), with (Bi )i≥1 a partition of the state-space.
In any case a minimum of regularity is required.
![Page 9: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/9.jpg)
Standard approach for the DPP
In both cases : for ν = ν1[0,θ) + 1[θ,T ]νε(θ,X νt,x(θ))
v(t, x) ≥ F (t, x ; ν) = E[E[f (X
νε(θ,Xνt,x (θ))
θ,Xνt,x (θ) (T )) | Fθ]]
= E[F (θ,X ν
t,x(θ); νε(θ,X νt,x(θ)))
]≥ E
[v(θ,X ν
t,x(θ))]− 3ε
Most of the time, proofs are based on a regularity argument :
F (·; νti ,xiε ) ≥
lscF (ti , xi ; ν
ti ,xiε )− ε ≥ v(ti , xi )− 2ε ≥
uscv − 3ε
on Bi 3 (ti , xi ), with (Bi )i≥1 a partition of the state-space.
In any case a minimum of regularity is required.
![Page 10: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/10.jpg)
Standard approach for the DPP
In both cases : for ν = ν1[0,θ) + 1[θ,T ]νε(θ,X νt,x(θ))
v(t, x) ≥ F (t, x ; ν) = E[E[f (X
νε(θ,Xνt,x (θ))
θ,Xνt,x (θ) (T )) | Fθ]]
= E[F (θ,X ν
t,x(θ); νε(θ,X νt,x(θ)))
]≥ E
[v(θ,X ν
t,x(θ))]− 3ε
Most of the time, proofs are based on a regularity argument :
F (·; νti ,xiε ) ≥
lscF (ti , xi ; ν
ti ,xiε )− ε ≥ v(ti , xi )− 2ε ≥
uscv − 3ε
on Bi 3 (ti , xi ), with (Bi )i≥1 a partition of the state-space.
In any case a minimum of regularity is required.
![Page 11: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/11.jpg)
DPP for what ?
To derive a PDE characterization, typically in the viscosity solutionsense.
Assuming v continuous, take a test function touching v from belowat (t, x)
ϕ(t, x) = v(t, x) ≥ E[v(θ,X ν
t,x(θ))]≥ E
[ϕ(θ,X ν
t,x(θ))]
And the other way round for the subsolution property :
ϕ(t, x) = v(t, x) ≤ supν∈U
E[v(θ,X ν
t,x(θ))]≤ sup
ν∈UE[ϕ(θ,X ν
t,x(θ))].
We never use : v(t, x) = supν∈U E[v(θ,X ν
t,x(θ))].
![Page 12: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/12.jpg)
DPP for what ?
To derive a PDE characterization, typically in the viscosity solutionsense.Assuming v continuous, take a test function touching v from belowat (t, x)
ϕ(t, x) = v(t, x) ≥ E[v(θ,X ν
t,x(θ))]≥ E
[ϕ(θ,X ν
t,x(θ))]
And the other way round for the subsolution property :
ϕ(t, x) = v(t, x) ≤ supν∈U
E[v(θ,X ν
t,x(θ))]≤ sup
ν∈UE[ϕ(θ,X ν
t,x(θ))].
We never use : v(t, x) = supν∈U E[v(θ,X ν
t,x(θ))].
![Page 13: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/13.jpg)
DPP for what ?
To derive a PDE characterization, typically in the viscosity solutionsense.Assuming v continuous, take a test function touching v from belowat (t, x)
ϕ(t, x) = v(t, x) ≥ E[v(θ,X ν
t,x(θ))]≥ E
[ϕ(θ,X ν
t,x(θ))]
And the other way round for the subsolution property :
ϕ(t, x) = v(t, x) ≤ supν∈U
E[v(θ,X ν
t,x(θ))]≤ sup
ν∈UE[ϕ(θ,X ν
t,x(θ))].
We never use : v(t, x) = supν∈U E[v(θ,X ν
t,x(θ))].
![Page 14: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/14.jpg)
DPP for what ?
To derive a PDE characterization, typically in the viscosity solutionsense.Assuming v continuous, take a test function touching v from belowat (t, x)
ϕ(t, x) = v(t, x) ≥ E[v(θ,X ν
t,x(θ))]≥ E
[ϕ(θ,X ν
t,x(θ))]
And the other way round for the subsolution property :
ϕ(t, x) = v(t, x) ≤ supν∈U
E[v(θ,X ν
t,x(θ))]≤ sup
ν∈UE[ϕ(θ,X ν
t,x(θ))].
We never use : v(t, x) = supν∈U E[v(θ,X ν
t,x(θ))].
![Page 15: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/15.jpg)
Main idea : prove the DPP on test functionsdirectly
Only assume that (t, x) 7→ F (t, x ; ν) is l.s.c. for ν fixed.
For all usc function ϕ ≤ v
F (·; νti ,xiε ) ≥
lscF (ti , xi ; ν
ti ,xiε )−ε ≥ v(ti , xi )−2ε ≥ ϕ(ti , xi )−2ε ≥
uscϕ−3ε
on Bi 3 (ti , xi ).One can reproduce the usual argument (but on ϕ) :
v(t, x) ≥ F (t, x ; ν) = E[E[f (X
νε(θ,Xνt,x (θ))
θ,Xνt,x (θ) (T )) | Fθ]]
= E[F (θ,X ν
t,x(θ); νε(θ,X νt,x(θ)))
]≥ E
[ϕ(θ,X ν
t,x(θ))]− 3ε
i.e. v(t, x) ≥ supν∈U E[ϕ(θ,X ν
t,x(θ))].
![Page 16: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/16.jpg)
Main idea : prove the DPP on test functionsdirectly
Only assume that (t, x) 7→ F (t, x ; ν) is l.s.c. for ν fixed.For all usc function ϕ ≤ v
F (·; νs,yε ) ≥
lscF (s, y ; νs,y
ε )−ε ≥ v(s, y)−2ε ≥ ϕ(s, y)−2ε ≥usc
ϕ−3ε
on Bs,y .
One can reproduce the usual argument (but on ϕ) :
v(t, x) ≥ F (t, x ; ν) = E[E[f (X
νε(θ,Xνt,x (θ))
θ,Xνt,x (θ) (T )) | Fθ]]
= E[F (θ,X ν
t,x(θ); νε(θ,X νt,x(θ)))
]≥ E
[ϕ(θ,X ν
t,x(θ))]− 3ε
i.e. v(t, x) ≥ supν∈U E[ϕ(θ,X ν
t,x(θ))].
![Page 17: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/17.jpg)
Main idea : prove the DPP on test functionsdirectly
Only assume that (t, x) 7→ F (t, x ; ν) is l.s.c. for ν fixed.For all usc function ϕ ≤ v
F (·; νti ,xiε ) ≥
lscF (ti , xi ; ν
ti ,xiε )−ε ≥ v(ti , xi )−2ε ≥ ϕ(ti , xi )−2ε ≥
uscϕ−3ε
on Bi 3 (ti , xi ).
One can reproduce the usual argument (but on ϕ) :
v(t, x) ≥ F (t, x ; ν) = E[E[f (X
νε(θ,Xνt,x (θ))
θ,Xνt,x (θ) (T )) | Fθ]]
= E[F (θ,X ν
t,x(θ); νε(θ,X νt,x(θ)))
]≥ E
[ϕ(θ,X ν
t,x(θ))]− 3ε
i.e. v(t, x) ≥ supν∈U E[ϕ(θ,X ν
t,x(θ))].
![Page 18: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/18.jpg)
Main idea : prove the DPP on test functionsdirectly
Only assume that (t, x) 7→ F (t, x ; ν) is l.s.c. for ν fixed.For all usc function ϕ ≤ v
F (·; νti ,xiε ) ≥
lscF (ti , xi ; ν
ti ,xiε )−ε ≥ v(ti , xi )−2ε ≥ ϕ(ti , xi )−2ε ≥
uscϕ−3ε
on Bi 3 (ti , xi ).One can reproduce the usual argument (but on ϕ) :
v(t, x) ≥ F (t, x ; ν) = E[E[f (X
νε(θ,Xνt,x (θ))
θ,Xνt,x (θ) (T )) | Fθ]]
= E[F (θ,X ν
t,x(θ); νε(θ,X νt,x(θ)))
]≥ E
[ϕ(θ,X ν
t,x(θ))]− 3ε
i.e. v(t, x) ≥ supν∈U E[ϕ(θ,X ν
t,x(θ))].
![Page 19: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/19.jpg)
Main idea : prove the DPP on test functionsdirectly
Only assume that (t, x) 7→ F (t, x ; ν) is l.s.c. for ν fixed.For all usc function ϕ ≤ v
F (·; νti ,xiε ) ≥
lscF (ti , xi ; ν
ti ,xiε )−ε ≥ v(ti , xi )−2ε ≥ ϕ(ti , xi )−2ε ≥
uscϕ−3ε
on Bi 3 (ti , xi ).One can reproduce the usual argument (but on ϕ) :
v(t, x) ≥ F (t, x ; ν) = E[E[f (X
νε(θ,Xνt,x (θ))
θ,Xνt,x (θ) (T )) | Fθ]]
= E[F (θ,X ν
t,x(θ); νε(θ,X νt,x(θ)))
]≥ E
[ϕ(θ,X ν
t,x(θ))]− 3ε
i.e. v(t, x) ≥ supν∈U E[ϕ(θ,X ν
t,x(θ))].
![Page 20: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/20.jpg)
An abstract setting
• Filtration : F ts := σ(Zr − Zt ; t ≤ r ≤ s) for some càdlàg process
Z with independent increments.
• Admissible controls : Ut the subset of U whose elements arepredictable with respect to Ft := (F t
s )s≥t .
• Controlled process : ν ∈ U 7→ X νt,x a càdlàg adapted process with
values in Rd (could be a separable metric space).
![Page 21: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/21.jpg)
An abstract setting
• Filtration : F ts := σ(Zr − Zt ; t ≤ r ≤ s) for some càdlàg process
Z with independent increments.
• Admissible controls : Ut the subset of U whose elements arepredictable with respect to Ft := (F t
s )s≥t .
• Controlled process : ν ∈ U 7→ X νt,x a càdlàg adapted process with
values in Rd (could be a separable metric space).
![Page 22: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/22.jpg)
An abstract setting
• Filtration : F ts := σ(Zr − Zt ; t ≤ r ≤ s) for some càdlàg process
Z with independent increments.
• Admissible controls : Ut the subset of U whose elements arepredictable with respect to Ft := (F t
s )s≥t .
• Controlled process : ν ∈ U 7→ X νt,x a càdlàg adapted process with
values in Rd (could be a separable metric space).
![Page 23: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/23.jpg)
Finiteness assumption : E[|f (X ν
t,x(T ))|]<∞ for all ν ∈ U .
Structure assumption : Let (t, x) ∈ [0,T ]× Rd , ν ∈ Ut , τ ∈ T t ,Γ ∈ F t
τ and ν ∈ U‖τ‖L∞ .
There exists a control ν ∈ Ut , denoted by ν ⊗(τ,Γ) ν, such that
X νt,x(·) = X ν
t,x(·) on [t,T ]× (Ω \ Γ);
X νt,x(·) = X ν
τ,Xνt,x (τ)(·) on [τ,T ]× Γ;
E[f (X ν
t,x(T )) | Fτ]
= F (τ,X νt,x(τ); ν) on Γ.
![Page 24: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/24.jpg)
Finiteness assumption : E[|f (X ν
t,x(T ))|]<∞ for all ν ∈ U .
Structure assumption : Let (t, x) ∈ [0,T ]× Rd , ν ∈ Ut , τ ∈ T t ,Γ ∈ F t
τ and ν ∈ U‖τ‖L∞ .
There exists a control ν ∈ Ut , denoted by ν ⊗(τ,Γ) ν, such that
X νt,x(·) = X ν
t,x(·) on [t,T ]× (Ω \ Γ);
X νt,x(·) = X ν
τ,Xνt,x (τ)(·) on [τ,T ]× Γ;
E[f (X ν
t,x(T )) | Fτ]
= F (τ,X νt,x(τ); ν) on Γ.
![Page 25: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/25.jpg)
Easy part of the DPP
• Fix (t, x) ∈ [0,T ]× Rd , ν ∈ Ut . Let θν′, ν ′ ∈ U ⊂ T t and let
ϕ : [0,T ]×S → R be a measurable function such that v ≤ ϕ.Then,
F (t, x ; ν) ≤ E[ϕ(θν ,X ν
t,x(θν))].
Corollary :v(t, x) ≤ sup
ν∈Ut
E[ϕ(θν ,X ν
t,x(θν))]
![Page 26: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/26.jpg)
Easy part of the DPP
• Fix (t, x) ∈ [0,T ]× Rd , ν ∈ Ut . Let θν′, ν ′ ∈ U ⊂ T t and let
ϕ : [0,T ]×S → R be a measurable function such that v ≤ ϕ.Then,
F (t, x ; ν) ≤ E[ϕ(θν ,X ν
t,x(θν))].
Corollary :v(t, x) ≤ sup
ν∈Ut
E[ϕ(θν ,X ν
t,x(θν))]
![Page 27: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/27.jpg)
Proof :
F (t, x ; ν) = E[E[f (X ν
θ,Xνt,x (θ)(T )) | Fθ]]
where
E[f (X ν
θ,Xνt,x (θ)(T )) | Fθ]
(ω) = F (θ(ω),X νt,x(θ)(ω); ν(ωθ(ω), ·))
≤ v(θ(ω),X νt,x(θ)(ω))
≤ ϕ(θ(ω),X νt,x(θ)(ω))
because ν(ωθ(ω), ·) ∈ Uθ(ω).
![Page 28: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/28.jpg)
Proof :
F (t, x ; ν) = E[E[f (X ν
θ,Xνt,x (θ)(T )) | Fθ]]
where
E[f (X ν
θ,Xνt,x (θ)(T )) | Fθ]
(ω) = F (θ(ω),X νt,x(θ)(ω); ν(ωθ(ω), ·))
≤ v(θ(ω),X νt,x(θ)(ω))
≤ ϕ(θ(ω),X νt,x(θ)(ω))
because ν(ωθ(ω), ·) ∈ Uθ(ω).
![Page 29: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/29.jpg)
“Difficult” part
• Assume that F (·; ν) is lsc for all ν ∈ U (on the left in time).
• Fix (t, x) ∈ [0,T ]× Rd . Let θν′ , ν ′ ∈ U ⊂ T t and letϕ : [0,T ]× S → R be a u.s.c. function such that v ≥ ϕ.Then,
v(t, x) ≥ supν∈Ut
E[ϕ(θν ,X ν
t,x(θν))].
![Page 30: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/30.jpg)
“Difficult” part
• Assume that F (·; ν) is lsc for all ν ∈ U (on the left in time).
• Fix (t, x) ∈ [0,T ]× Rd . Let θν′ , ν ′ ∈ U ⊂ T t and letϕ : [0,T ]× S → R be a u.s.c. function such that v ≥ ϕ.Then,
v(t, x) ≥ supν∈Ut
E[ϕ(θν ,X ν
t,x(θν))].
![Page 31: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/31.jpg)
Proof of the difficult partFix νs,y ∈ Us such that
F (s, y ; νs,y ) ≥ v(s, y)− ε .
Fix Bs,y := (s − rs,y , s]× B(y , rs,y ) such that
F (·; νs,y ) ≥ F (s, y ; νs,y )− ε ≥ ϕ(s, y)− 2ε ≥ ϕ on Bs,y .
By the Lindelof property : we can find (si , yi )i≥1 such that∪iBsi ,yi = (0,T ]× Rd .
Take a disjoint sub-covering (Ai )i and setΓi := (θ,X ν
t,x(θ)) ∈ Bi, Γn := ∪i≤nΓi .
Set
νn := (((ν ⊗θ,Γ1 νs1,y1)⊗θ,Γ2 νs2,y2)⊗ · · · )⊗θ,Γn νsn,yn
![Page 32: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/32.jpg)
Proof of the difficult partFix νs,y ∈ Us such that
F (s, y ; νs,y ) ≥ v(s, y)− ε .
Fix Bs,y := (s − rs,y , s]× B(y , rs,y ) such that
F (·; νs,y ) ≥ F (s, y ; νs,y )− ε ≥ ϕ(s, y)− 2ε ≥ ϕ on Bs,y .
By the Lindelof property : we can find (si , yi )i≥1 such that∪iBsi ,yi = (0,T ]× Rd .
Take a disjoint sub-covering (Ai )i and setΓi := (θ,X ν
t,x(θ)) ∈ Bi, Γn := ∪i≤nΓi .
Set
νn := (((ν ⊗θ,Γ1 νs1,y1)⊗θ,Γ2 νs2,y2)⊗ · · · )⊗θ,Γn νsn,yn
![Page 33: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/33.jpg)
Proof of the difficult partFix νs,y ∈ Us such that
F (s, y ; νs,y ) ≥ v(s, y)− ε .
Fix Bs,y := (s − rs,y , s]× B(y , rs,y ) such that
F (·; νs,y ) ≥ F (s, y ; νs,y )− ε ≥ ϕ(s, y)− 2ε ≥ ϕ on Bs,y .
By the Lindelof property : we can find (si , yi )i≥1 such that∪iBsi ,yi = (0,T ]× Rd .
Take a disjoint sub-covering (Ai )i and setΓi := (θ,X ν
t,x(θ)) ∈ Bi, Γn := ∪i≤nΓi .
Set
νn := (((ν ⊗θ,Γ1 νs1,y1)⊗θ,Γ2 νs2,y2)⊗ · · · )⊗θ,Γn νsn,yn
![Page 34: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/34.jpg)
Proof of the difficult partFix νs,y ∈ Us such that
F (s, y ; νs,y ) ≥ v(s, y)− ε .
Fix Bs,y := (s − rs,y , s]× B(y , rs,y ) such that
F (·; νs,y ) ≥ F (s, y ; νs,y )− ε ≥ ϕ(s, y)− 2ε ≥ ϕ on Bs,y .
By the Lindelof property : we can find (si , yi )i≥1 such that∪iBsi ,yi = (0,T ]× Rd .
Take a disjoint sub-covering (Ai )i and setΓi := (θ,X ν
t,x(θ)) ∈ Bi, Γn := ∪i≤nΓi .
Set
νn := (((ν ⊗θ,Γ1 νs1,y1)⊗θ,Γ2 νs2,y2)⊗ · · · )⊗θ,Γn νsn,yn
![Page 35: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/35.jpg)
Proof of the difficult partFix νs,y ∈ Us such that
F (s, y ; νs,y ) ≥ v(s, y)− ε .
Fix Bs,y := (s − rs,y , s]× B(y , rs,y ) such that
F (·; νs,y ) ≥ F (s, y ; νs,y )− ε ≥ ϕ(s, y)− 2ε ≥ ϕ on Bs,y .
By the Lindelof property : we can find (si , yi )i≥1 such that∪iBsi ,yi = (0,T ]× Rd .
Take a disjoint sub-covering (Ai )i and setΓi := (θ,X ν
t,x(θ)) ∈ Bi, Γn := ∪i≤nΓi .
Set
νn := (((ν ⊗θ,Γ1 νs1,y1)⊗θ,Γ2 νs2,y2)⊗ · · · )⊗θ,Γn νsn,yn
![Page 36: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/36.jpg)
Then
F (t, x ; νn) = E
f (X νt,x(T ))1Γnc +
∑i≤n
E[f (X
νsi ,yiθ,Xνt,x (θ)(T )) | Fθ
]1Γi
where, on Γi (with Ai ⊂ Bi ),
E[f (X
νsi ,yiθ,Xνt,x (θ)(T )) | Fθ
](ω) = F (θ(ω),X ν
t,x(θ)(ω); νsi ,yi )
≥ ϕ(θ(ω),X νt,x(θ)(ω))− 3ε.
Pass to the limit in n→∞ and ε→ 0.
![Page 37: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/37.jpg)
Then
F (t, x ; νn) = E
f (X νt,x(T ))1Γnc +
∑i≤n
E[f (X
νsi ,yiθ,Xνt,x (θ)(T )) | Fθ
]1Γi
where, on Γi (with Ai ⊂ Bi ),
E[f (X
νsi ,yiθ,Xνt,x (θ)(T )) | Fθ
](ω) = F (θ(ω),X ν
t,x(θ)(ω); νsi ,yi )
≥ ϕ(θ(ω),X νt,x(θ)(ω))− 3ε.
Pass to the limit in n→∞ and ε→ 0.
![Page 38: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/38.jpg)
The weak DPP (summing up)
Theorem : Assume that F (·; ν) is lsc for all ν ∈ U (on the left intime). Let ϕ− ≤ v ≤ ϕ+ be two smooth functions. Then
supν∈Ut
E[ϕ−(θν ,X ν
t,x(θν))]≤ v(t, x) ≤ sup
ν∈Ut
E[ϕ+(θν ,X ν
t,x(θν))]
Remark : if v is locally bounded and (θν ,X νt,x(θν)), ν ∈ Ut is
bounded then
supν∈Ut
E[v∗(θν ,X ν
t,x(θν))]≤ v(t, x) ≤ sup
ν∈Ut
E[v∗(θν ,X ν
t,x(θν))]
Remark : v = v∗ if F (·; ν) is lsc. If v is usc, then one retrieves theusual DPP.
![Page 39: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/39.jpg)
The weak DPP (summing up)
Theorem : Assume that F (·; ν) is lsc for all ν ∈ U (on the left intime). Let ϕ− ≤ v ≤ ϕ+ be two smooth functions. Then
supν∈Ut
E[ϕ−(θν ,X ν
t,x(θν))]≤ v(t, x) ≤ sup
ν∈Ut
E[ϕ+(θν ,X ν
t,x(θν))]
Remark : if v is locally bounded and (θν ,X νt,x(θν)), ν ∈ Ut is
bounded then
supν∈Ut
E[v∗(θν ,X ν
t,x(θν))]≤ v(t, x) ≤ sup
ν∈Ut
E[v∗(θν ,X ν
t,x(θν))]
Remark : v = v∗ if F (·; ν) is lsc. If v is usc, then one retrieves theusual DPP.
![Page 40: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/40.jpg)
The weak DPP (summing up)
Theorem : Assume that F (·; ν) is lsc for all ν ∈ U (on the left intime). Let ϕ− ≤ v ≤ ϕ+ be two smooth functions. Then
supν∈Ut
E[ϕ−(θν ,X ν
t,x(θν))]≤ v(t, x) ≤ sup
ν∈Ut
E[ϕ+(θν ,X ν
t,x(θν))]
Remark : if v is locally bounded and (θν ,X νt,x(θν)), ν ∈ Ut is
bounded then
supν∈Ut
E[v∗(θν ,X ν
t,x(θν))]≤ v(t, x) ≤ sup
ν∈Ut
E[v∗(θν ,X ν
t,x(θν))]
Remark : v = v∗ if F (·; ν) is lsc. If v is usc, then one retrieves theusual DPP.
![Page 41: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/41.jpg)
Other examples of application
• Set of controls depending on the state process : Bouchard, Dangand Lehall, Optimal control of trading algorithms : a generalimpulse control approach, to appear in SIAM Journal on FinancialMathematics.
• Game problem : Bayraktar and Hang, On the Multi-dimensionalcontroller and stopper games, preprint 2010.
![Page 42: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/42.jpg)
The case with constraint in expectation
v(t, x) := supν∈U(t,x ,m)
F (t, x ; ν) with F (t, x , ν) := E[f (X ν
t,x(T ))]
and
U(t, x ,m) := ν ∈ Ut : G (t, x ; ν) := E[g(X ν
t,x(T ))]≤ m
Weak Dynamic Programming for Generalized State Constraints, withMarcel Nutz, preprint 2011.
(compare with Bouchard, Elie, Imbert, SIAM Journal on Control andOptimization, 48 (5), 2010. )
![Page 43: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/43.jpg)
Problem reformulation towards DPP
• State space augmentation : LetMt,m be a set of càdlàgmartingales M = M(s), s ∈ [t,T ] with initial value M(t) = m,adapted to Ft .
• Martingale representation assumption : We assume that, forall (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :
∃ Mνt [x ] ∈Mt,m such that Mν
t [x ](T ) = g(X νt,x(T )),
with m := E[g(X ν
t,x(T ))].
• Reformulation : We set
M+t,x ,m(ν) := M ∈Mt,m : M(T ) ≥ g(X ν
t,x(T ))
![Page 44: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/44.jpg)
Problem reformulation towards DPP
• State space augmentation : LetMt,m be a set of càdlàgmartingales M = M(s), s ∈ [t,T ] with initial value M(t) = m,adapted to Ft .
• Martingale representation assumption : We assume that, forall (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :
∃ Mνt [x ] ∈Mt,m such that Mν
t [x ](T ) = g(X νt,x(T )),
with m := E[g(X ν
t,x(T ))].
• Reformulation : We set
M+t,x ,m(ν) := M ∈Mt,m : M(T ) ≥ g(X ν
t,x(T ))
![Page 45: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/45.jpg)
Problem reformulation towards DPP
• State space augmentation : LetMt,m be a set of càdlàgmartingales M = M(s), s ∈ [t,T ] with initial value M(t) = m,adapted to Ft .
• Martingale representation assumption : We assume that, forall (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :
∃ Mνt [x ] ∈Mt,m such that Mν
t [x ](T ) = g(X νt,x(T )),
with m := E[g(X ν
t,x(T ))].
• Reformulation : We set
M+t,x ,m(ν) := M ∈Mt,m : M(T ) ≥ g(X ν
t,x(T ))
![Page 46: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/46.jpg)
We set
M+t,x ,m(ν) := M ∈Mt,m : M(T ) ≥ g(X ν
t,x(T ))
Lemma : Let (t, x) ∈ [0,T ]× Rd and m ∈ R. Then
U(t, x ,m) =ν ∈ Ut : M+
t,x ,m(ν) 6= ∅.
Example : In a Brownian filtration, we can take
Mt,m = m + Mαt,0(T ) :=
∫ T
tαsdWs , α ∈ At
where At is the set of predictable Rd -valued processes such thatMα
t,0 is a Ft-adapted martingale.Then,
U(t, x ,m) =ν ∈ Ut : ∃ α ∈ At s.t. Mα
t,m(T ) ≥ g(X νt,x(T ))
.
![Page 47: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/47.jpg)
We set
M+t,x ,m(ν) := M ∈Mt,m : M(T ) ≥ g(X ν
t,x(T ))
Lemma : Let (t, x) ∈ [0,T ]× Rd and m ∈ R. Then
U(t, x ,m) =ν ∈ Ut : M+
t,x ,m(ν) 6= ∅.
Example : In a Brownian filtration, we can take
Mt,m = m + Mαt,0(T ) :=
∫ T
tαsdWs , α ∈ At
where At is the set of predictable Rd -valued processes such thatMα
t,0 is a Ft-adapted martingale.
Then,
U(t, x ,m) =ν ∈ Ut : ∃ α ∈ At s.t. Mα
t,m(T ) ≥ g(X νt,x(T ))
.
![Page 48: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/48.jpg)
We set
M+t,x ,m(ν) := M ∈Mt,m : M(T ) ≥ g(X ν
t,x(T ))
Lemma : Let (t, x) ∈ [0,T ]× Rd and m ∈ R. Then
U(t, x ,m) =ν ∈ Ut : M+
t,x ,m(ν) 6= ∅.
Example : In a Brownian filtration, we can take
Mt,m = m + Mαt,0(T ) :=
∫ T
tαsdWs , α ∈ At
where At is the set of predictable Rd -valued processes such thatMα
t,0 is a Ft-adapted martingale.Then,
U(t, x ,m) =ν ∈ Ut : ∃ α ∈ At s.t. Mα
t,m(T ) ≥ g(X νt,x(T ))
.
![Page 49: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/49.jpg)
Heuristic DPP in the Brownian setting• In the caseMt,m = m + Mα
t,0(T ) :=∫ Tt αsdWs , α ∈ At :
v(t, x ,m)?= sup
(ν,α)∈Θ(t,x ,m)E[v(θ,X ν
t,x(θ),Mαt,m(θ))
]with
Θ(t, x ,m) := (ν, α) ∈ Ut ×At : Mαt,m(T ) ≥ g(X ν
t,x(T )).
•Why could we obtain a weak formulation ?
If G (·; ν) is u.s.c, moving a bit moves the m constraint to an m + δconstraint with δ > 0 small.
Guess : for all δ > 0
v(t, x ,m + δ) ≥ sup(ν,α)∈Θ(t,x ,m)
E[ϕ(θ,X ν
t,x(θ),Mαt,m(θ))
].
![Page 50: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/50.jpg)
Heuristic DPP in the Brownian setting• In the caseMt,m = m + Mα
t,0(T ) :=∫ Tt αsdWs , α ∈ At :
v(t, x ,m)?= sup
(ν,α)∈Θ(t,x ,m)E[v(θ,X ν
t,x(θ),Mαt,m(θ))
]with
Θ(t, x ,m) := (ν, α) ∈ Ut ×At : Mαt,m(T ) ≥ g(X ν
t,x(T )).
•Why could we obtain a weak formulation ?
If G (·; ν) is u.s.c, moving a bit moves the m constraint to an m + δconstraint with δ > 0 small.
Guess : for all δ > 0
v(t, x ,m + δ) ≥ sup(ν,α)∈Θ(t,x ,m)
E[ϕ(θ,X ν
t,x(θ),Mαt,m(θ))
].
![Page 51: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/51.jpg)
Heuristic DPP in the Brownian setting• In the caseMt,m = m + Mα
t,0(T ) :=∫ Tt αsdWs , α ∈ At :
v(t, x ,m)?= sup
(ν,α)∈Θ(t,x ,m)E[v(θ,X ν
t,x(θ),Mαt,m(θ))
]with
Θ(t, x ,m) := (ν, α) ∈ Ut ×At : Mαt,m(T ) ≥ g(X ν
t,x(T )).
•Why could we obtain a weak formulation ?
If G (·; ν) is u.s.c, moving a bit moves the m constraint to an m + δconstraint with δ > 0 small.
Guess : for all δ > 0
v(t, x ,m + δ) ≥ sup(ν,α)∈Θ(t,x ,m)
E[ϕ(θ,X ν
t,x(θ),Mαt,m(θ))
].
![Page 52: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/52.jpg)
Heuristic DPP in the Brownian setting• In the caseMt,m = m + Mα
t,0(T ) :=∫ Tt αsdWs , α ∈ At :
v(t, x ,m)?= sup
(ν,α)∈Θ(t,x ,m)E[v(θ,X ν
t,x(θ),Mαt,m(θ))
]with
Θ(t, x ,m) := (ν, α) ∈ Ut ×At : Mαt,m(T ) ≥ g(X ν
t,x(T )).
•Why could we obtain a weak formulation ?
If G (·; ν) is u.s.c, moving a bit moves the m constraint to an m + δconstraint with δ > 0 small.
Guess : for all δ > 0
v(t, x ,m + δ) ≥ sup(ν,α)∈Θ(t,x ,m)
E[ϕ(θ,X ν
t,x(θ),Mαt,m(θ))
].
![Page 53: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/53.jpg)
Additional assumption
Assumption : Let (t, x) ∈ [0,T ]× Rd , ν ∈ Ut , τ ∈ T t , Γ ∈ F tτ ,
ν ∈ U‖τ‖L∞ , and M ∈Mt,0. Then, there exists a processM = M(r), r ∈ [τ,T ] such that
M(·)(ω) =(M ντ(ω)[X ν
t,x(τ)(ω)](·))(ω) on [τ,T ] P− a.s.
and
M1[t,τ) + 1[τ,T ]
(M1Ω\Γ +
[M − M(τ) + M(τ)
]1Γ
)∈Mt,0.
![Page 54: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/54.jpg)
General result
Theorem : Assume the above holds.(i) Let ϕ+ ≥ v be a measurable function. Then
v(t, x ,m) ≤ E[ϕ+(θν ,X ν
t,x(θν),M(θν))]
for some ν ∈ U(t, x ,m) and M ∈M+t,x ,m(ν).
(ii) Assume that F (·; ν) and −G (·; ν) are lsc for all ν ∈ U (on theleft in time). Let ϕ− ≤ v be a usc function and fix δ > 0. Then
v(t, x ,m + δ) ≥ E[ϕ−(θν ,X ν
t,x(θν),M(θν))]
for all ν ∈ U(t, x ,m) and M ∈M+t,x ,m(ν).
![Page 55: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/55.jpg)
General result
Theorem : Assume the above holds.(i) Let ϕ+ ≥ v be a measurable function. Then
v(t, x ,m) ≤ E[ϕ+(θν ,X ν
t,x(θν),M(θν))]
for some ν ∈ U(t, x ,m) and M ∈M+t,x ,m(ν).
(ii) Assume that F (·; ν) and −G (·; ν) are lsc for all ν ∈ U (on theleft in time). Let ϕ− ≤ v be a usc function and fix δ > 0. Then
v(t, x ,m + δ) ≥ E[ϕ−(θν ,X ν
t,x(θν),M(θν))]
for all ν ∈ U(t, x ,m) and M ∈M+t,x ,m(ν).
![Page 56: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/56.jpg)
The Brownian setting
• U : set of square integrable predictable processes with values inU ⊂ Rd .
• X νt,x solves on [t,T ]
dXs = b(Xs , νs)ds + σ(Xs , νs)dWs
with Lipschitz conditions (can be relaxed).
• The set of martingales is given by :
Mt,m = m + Mαt,0(T ) :=
∫ T
tαsdWs , α ∈ At.
![Page 57: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/57.jpg)
The Brownian setting
• U : set of square integrable predictable processes with values inU ⊂ Rd .
• X νt,x solves on [t,T ]
dXs = b(Xs , νs)ds + σ(Xs , νs)dWs
with Lipschitz conditions (can be relaxed).
• The set of martingales is given by :
Mt,m = m + Mαt,0(T ) :=
∫ T
tαsdWs , α ∈ At.
![Page 58: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/58.jpg)
The Brownian setting
• U : set of square integrable predictable processes with values inU ⊂ Rd .
• X νt,x solves on [t,T ]
dXs = b(Xs , νs)ds + σ(Xs , νs)dWs
with Lipschitz conditions (can be relaxed).
• The set of martingales is given by :
Mt,m = m + Mαt,0(T ) :=
∫ T
tαsdWs , α ∈ At.
![Page 59: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/59.jpg)
Domain of definition
• The natural domain D := (t, x ,m) : U(t, x ,m) 6= ∅ isassociated to
w(t, x) := infν∈Ut
E [g(X νt,x(T ))],
through
intD =
(t, x ,m) ∈ [0,T ]× Rd × R : m > w(t, x), t < T,
(w is usc if G (·; ν) is).
• One has
D ⊆
(t, x ,m) ∈ [0,T ]× Rd × R : m ≥ w∗(t, x)
= intD,
where w∗ is the lower semicontinuous envelope of v on [0,T ]×Rd .
![Page 60: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/60.jpg)
Domain of definition
• The natural domain D := (t, x ,m) : U(t, x ,m) 6= ∅ isassociated to
w(t, x) := infν∈Ut
E [g(X νt,x(T ))],
through
intD =
(t, x ,m) ∈ [0,T ]× Rd × R : m > w(t, x), t < T,
(w is usc if G (·; ν) is).• One has
D ⊆
(t, x ,m) ∈ [0,T ]× Rd × R : m ≥ w∗(t, x)
= intD,
where w∗ is the lower semicontinuous envelope of v on [0,T ]×Rd .
![Page 61: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/61.jpg)
DPP for viscosity super-solution
Corollary : Assume the above holds. Let θν,αB be the first exist timeof (·,X ν
t,x ,Mαt,m) from a ball B around (t, x ,m) ∈ intD. Then, for
all δ > 0 and θν,α ≤ θν,αB ,
v(t, x ,m + δ) ≥ sup(ν,α)∈Ut×At
E[ϕ−(θν,α,X ν
t,x(θν,α),Mαt,m(θν,α))
]
![Page 62: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/62.jpg)
Viscosity super-solution property derivation
Let ϕ− be a test function for v∗ at (t, x ,m) Fix(tε, xε,mε, δε)→ (t, x ,m, 0) such that
|v(tε, xε,mε + δε)− ϕ−(tε, xε,mε)| ≤ ε2 → 0
Set (ν, α) = (u, a) ∈ U × Rd , θε := θu,aB ∧ (tε + ε). Then,
ϕ−(tε, xε,mε) ≥ E[ϕ−(θε,X u
tε,xε(θε),Matε,mε(θε))
]− ε2
and therefore
0 ≥ ε−1E
[∫ θu,aB ∧(tε+ε)
tε(∂t + Lu,a
X ,M)ϕ−(s,X utε,xε(s),Ma
tε,mε(s))ds
]− ε
and pass to the limit ε→ 0.[In practice use a proof by contradiction to avoid passages to thelimit]
![Page 63: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/63.jpg)
Viscosity super-solution property derivation
Let ϕ− be a test function for v∗ at (t, x ,m) Fix(tε, xε,mε, δε)→ (t, x ,m, 0) such that
|v(tε, xε,mε + δε)− ϕ−(tε, xε,mε)| ≤ ε2 → 0
Set (ν, α) = (u, a) ∈ U × Rd , θε := θu,aB ∧ (tε + ε). Then,
v(tε, xε,mε + δε) ≥ E[ϕ−(θε,X u
tε,xε(θε),Matε,mε(θε))
]
and therefore
0 ≥ ε−1E
[∫ θu,aB ∧(tε+ε)
tε(∂t + Lu,a
X ,M)ϕ−(s,X utε,xε(s),Ma
tε,mε(s))ds
]− ε
and pass to the limit ε→ 0.[In practice use a proof by contradiction to avoid passages to thelimit]
![Page 64: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/64.jpg)
Viscosity super-solution property derivation
Let ϕ− be a test function for v∗ at (t, x ,m) Fix(tε, xε,mε, δε)→ (t, x ,m, 0) such that
|v(tε, xε,mε + δε)− ϕ−(tε, xε,mε)| ≤ ε2 → 0
Set (ν, α) = (u, a) ∈ U × Rd , θε := θu,aB ∧ (tε + ε). Then,
ϕ−(tε, xε,mε) ≥ E[ϕ−(θε,X u
tε,xε(θε),Matε,mε(θε))
]− ε2
and therefore
0 ≥ ε−1E
[∫ θu,aB ∧(tε+ε)
tε(∂t + Lu,a
X ,M)ϕ−(s,X utε,xε(s),Ma
tε,mε(s))ds
]− ε
and pass to the limit ε→ 0.[In practice use a proof by contradiction to avoid passages to thelimit]
![Page 65: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/65.jpg)
Viscosity super-solution property derivation
Let ϕ− be a test function for v∗ at (t, x ,m) Fix(tε, xε,mε, δε)→ (t, x ,m, 0) such that
|v(tε, xε,mε + δε)− ϕ−(tε, xε,mε)| ≤ ε2 → 0
Set (ν, α) = (u, a) ∈ U × Rd , θε := θu,aB ∧ (tε + ε). Then,
ϕ−(tε, xε,mε) ≥ E[ϕ−(θε,X u
tε,xε(θε),Matε,mε(θε))
]− ε2
and therefore
0 ≥ ε−1E
[∫ θu,aB ∧(tε+ε)
tε(∂t + Lu,a
X ,M)ϕ−(s,X utε,xε(s),Ma
tε,mε(s))ds
]− ε
and pass to the limit ε→ 0.
[In practice use a proof by contradiction to avoid passages to thelimit]
![Page 66: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/66.jpg)
Viscosity super-solution property derivation
Let ϕ− be a test function for v∗ at (t, x ,m) Fix(tε, xε,mε, δε)→ (t, x ,m, 0) such that
|v(tε, xε,mε + δε)− ϕ−(tε, xε,mε)| ≤ ε2 → 0
Set (ν, α) = (u, a) ∈ U × Rd , θε := θu,aB ∧ (tε + ε). Then,
ϕ−(tε, xε,mε) ≥ E[ϕ−(θε,X u
tε,xε(θε),Matε,mε(θε))
]− ε2
and therefore
0 ≥ ε−1E
[∫ θu,aB ∧(tε+ε)
tε(∂t + Lu,a
X ,M)ϕ−(s,X utε,xε(s),Ma
tε,mε(s))ds
]− ε
and pass to the limit ε→ 0.[In practice use a proof by contradiction to avoid passages to thelimit]
![Page 67: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/67.jpg)
PDE characterization
Theorem : Assume the above holds. Then,(i) v∗ is a viscosity super-solution on intD of
−∂tϕ+ H(·,Dϕ,D2ϕ) = 0.
(ii) v∗ is a viscosity sub-solution on clD of
−∂tϕ+ H∗(·,Dϕ,D2ϕ) = 0
whereH(·,Dϕ,D2ϕ) := − sup
(u,a)∈U×RdLu,a
X ,Mϕ.
(See Bouchard, Elie and Imbert 2010 for a discussion on theboundary conditions)
![Page 68: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/68.jpg)
The case with P− a.s. state constraint
v(t, x) := supν∈U(t,x)
F (t, x ; ν) with F (t, x , ν) := E[f (X ν
t,x(T ))]
and
U(t, x) := ν ∈ Ut : X νt,x ∈ O on [t,T ]
with O an open subset.
Weak Dynamic Programming for Generalized State Constraints, withMarcel Nutz, preprint 2011.
![Page 69: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/69.jpg)
A-priori difficulty
• Can not use a (t, x) admissible control in a ball around (t, x) :may exit the domain.
• Can in fact almost do this if O is open : if exits, it should be withsmall probability.
![Page 70: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/70.jpg)
A-priori difficulty
• Can not use a (t, x) admissible control in a ball around (t, x) :may exit the domain.
• Can in fact almost do this if O is open
: if exits, it should be withsmall probability.
![Page 71: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/71.jpg)
A-priori difficulty
• Can not use a (t, x) admissible control in a ball around (t, x) :may exit the domain.
• Can in fact almost do this if O is open : if exits, it should be withsmall probability.
![Page 72: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/72.jpg)
Approximation by constraints in expectations• Additional dimension
Y νt,x ,y (s) := y ∧ inf
r∈[t,s]d(X ν
t,x(r)), s ∈ [t,T ], y > 0.
By continuity, each trajectory X νt,x(r)(ω), r ∈ [t,T ] has strictly
positive distance to Oc whenever it is contained in O :
X νt,x(r)(ω), r ∈ [t,T ] ⊆ O if and only if Y ν
t,x ,y (T )(ω) > 0.
• Equivalent control problem
v(t, x) = v(t, x , y , 0)
wherev(t, x , y ,m) := sup
ν∈U(t,x ,y ,m)F (t, x ; ν)
with
U(t, x , y ,m) := ν ∈ Ut : P[Y ν
t,x ,1(T ) ≤ 0]≤ m.
![Page 73: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/73.jpg)
Approximation by constraints in expectations• Additional dimension
Y νt,x ,y (s) := y ∧ inf
r∈[t,s]d(X ν
t,x(r)), s ∈ [t,T ], y > 0.
By continuity, each trajectory X νt,x(r)(ω), r ∈ [t,T ] has strictly
positive distance to Oc whenever it is contained in O :
X νt,x(r)(ω), r ∈ [t,T ] ⊆ O if and only if Y ν
t,x ,y (T )(ω) > 0.
• Equivalent control problem
v(t, x) = v(t, x , y , 0)
wherev(t, x , y ,m) := sup
ν∈U(t,x ,y ,m)F (t, x ; ν)
with
U(t, x , y ,m) := ν ∈ Ut : P[Y ν
t,x ,1(T ) ≤ 0]≤ m.
![Page 74: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/74.jpg)
Approximation by constraints in expectations• Additional dimension
Y νt,x ,y (s) := y ∧ inf
r∈[t,s]d(X ν
t,x(r)), s ∈ [t,T ], y > 0.
By continuity, each trajectory X νt,x(r)(ω), r ∈ [t,T ] has strictly
positive distance to Oc whenever it is contained in O :
X νt,x(r)(ω), r ∈ [t,T ] ⊆ O if and only if Y ν
t,x ,y (T )(ω) > 0.
• Equivalent control problem
v(t, x) = v(t, x , y , 0)
wherev(t, x , y ,m) := sup
ν∈U(t,x ,y ,m)F (t, x ; ν)
with
U(t, x , y ,m) := ν ∈ Ut : P[Y ν
t,x ,1(T ) ≤ 0]≤ m.
![Page 75: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/75.jpg)
DPP in the state constraint case
• Apply the DPP to v : for all (ν, α) ∈ Θ(t, x , y , 0)
v(t, x , y , 0 + δ) ≥ E[ϕ−(θν ,X ν
t,x(θν),Y νt,x ,y (θν),Mα
t,0(θν))]
If v(t, x , y , 0+) = v(t, x , y , 0), then
v(t, x , y , 0) ≥ E[ϕ−(θν ,X ν
t,x(θν),Y νt,x ,y (θν), 0)
]but v(t, x , y , 0) = v(t, x , 1, 0) = v(t, x).
Hence
v(t, x) ≥ supν∈U(t,x)
E[φ−(θν ,X ν
t,x(θν))]
for φ− ≤ v usc.
![Page 76: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/76.jpg)
DPP in the state constraint case
• Apply the DPP to v : for all (ν, α) ∈ Θ(t, x , y , 0)
v(t, x , y , 0 + δ) ≥ E[ϕ−(θν ,X ν
t,x(θν),Y νt,x ,y (θν),Mα
t,0(θν))]
If v(t, x , y , 0+) = v(t, x , y , 0), then
v(t, x , y , 0) ≥ E[ϕ−(θν ,X ν
t,x(θν),Y νt,x ,y (θν), 0)
]
but v(t, x , y , 0) = v(t, x , 1, 0) = v(t, x).
Hence
v(t, x) ≥ supν∈U(t,x)
E[φ−(θν ,X ν
t,x(θν))]
for φ− ≤ v usc.
![Page 77: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/77.jpg)
DPP in the state constraint case
• Apply the DPP to v : for all (ν, α) ∈ Θ(t, x , y , 0)
v(t, x , y , 0 + δ) ≥ E[ϕ−(θν ,X ν
t,x(θν),Y νt,x ,y (θν),Mα
t,0(θν))]
If v(t, x , y , 0+) = v(t, x , y , 0), then
v(t, x , y , 0) ≥ E[ϕ−(θν ,X ν
t,x(θν),Y νt,x ,y (θν), 0)
]but v(t, x , y , 0) = v(t, x , 1, 0) = v(t, x).
Hence
v(t, x) ≥ supν∈U(t,x)
E[φ−(θν ,X ν
t,x(θν))]
for φ− ≤ v usc.
![Page 78: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/78.jpg)
DPP in the state constraint case
• Apply the DPP to v : for all (ν, α) ∈ Θ(t, x , y , 0)
v(t, x , y , 0 + δ) ≥ E[ϕ−(θν ,X ν
t,x(θν),Y νt,x ,y (θν),Mα
t,0(θν))]
If v(t, x , y , 0+) = v(t, x , y , 0), then
v(t, x , y , 0) ≥ E[ϕ−(θν ,X ν
t,x(θν),Y νt,x ,y (θν), 0)
]but v(t, x , y , 0) = v(t, x , 1, 0) = v(t, x).
Hence
v(t, x) ≥ supν∈U(t,x)
E[φ−(θν ,X ν
t,x(θν))]
for φ− ≤ v usc.
![Page 79: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/79.jpg)
DPP in the state constraint case : sufficientcondition
Measurable selection assumption : There exists a Lipschitzcontinuous mapping u : O → U such that, for all(t, x) ∈ [0,T ]×O, the solution Xt,x of
X (s) = x +
∫ s
tb(X (r), u(X (r))
)dr +
∫ s
tσ(X (r), u(X (r))
)dWr
satisfies Xt,x(s) ∈ O for all s ∈ [t,T ], P− a.s.
Technical assumption : Either f is bounded or the coefficientsb(x , u) and σ(x , u) have linear growth in x , uniformly in u.
Proposition : Under the above assumption,v(t, x , y , 0+) = v(t, x , y , 0).
![Page 80: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/80.jpg)
DPP in the state constraint case : sufficientcondition
Measurable selection assumption : There exists a Lipschitzcontinuous mapping u : O → U such that, for all(t, x) ∈ [0,T ]×O, the solution Xt,x of
X (s) = x +
∫ s
tb(X (r), u(X (r))
)dr +
∫ s
tσ(X (r), u(X (r))
)dWr
satisfies Xt,x(s) ∈ O for all s ∈ [t,T ], P− a.s.
Technical assumption : Either f is bounded or the coefficientsb(x , u) and σ(x , u) have linear growth in x , uniformly in u.
Proposition : Under the above assumption,v(t, x , y , 0+) = v(t, x , y , 0).
![Page 81: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/81.jpg)
DPP in the state constraint case : sufficientcondition
Measurable selection assumption : There exists a Lipschitzcontinuous mapping u : O → U such that, for all(t, x) ∈ [0,T ]×O, the solution Xt,x of
X (s) = x +
∫ s
tb(X (r), u(X (r))
)dr +
∫ s
tσ(X (r), u(X (r))
)dWr
satisfies Xt,x(s) ∈ O for all s ∈ [t,T ], P− a.s.
Technical assumption : Either f is bounded or the coefficientsb(x , u) and σ(x , u) have linear growth in x , uniformly in u.
Proposition : Under the above assumption,v(t, x , y , 0+) = v(t, x , y , 0).
![Page 82: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/82.jpg)
DPP in the state constraint case
Lemma : Let B be an open neighborhood of (t, x) ∈ [0,T ]×Osuch that v(t, x) is finite.(i) Let ϕ : clB → R be a continuous function such that v ≤ ϕ onclB . For all ε > 0, there exists ν ∈ U(t, x) such that
v(t, x) ≤ E[ϕ(τ,X ν
t,x(τ))]
+ ε,
where τ is the first exit time of (s,X νt,x(s))s≥t from B .
(ii) For any ν ∈ Ut and any continuous function ϕ s.t. v ≥ ϕ on clB
v(t, x) ≥ E[ϕ(τ,X ν
t,x(τ))],
where τ is the first exit time of (s,X νt,x(s))s≥t from B .
![Page 83: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/83.jpg)
DPP in the state constraint case
Lemma : Let B be an open neighborhood of (t, x) ∈ [0,T ]×Osuch that v(t, x) is finite.(i) Let ϕ : clB → R be a continuous function such that v ≤ ϕ onclB . For all ε > 0, there exists ν ∈ U(t, x) such that
v(t, x) ≤ E[ϕ(τ,X ν
t,x(τ))]
+ ε,
where τ is the first exit time of (s,X νt,x(s))s≥t from B .
(ii) For any ν ∈ Ut and any continuous function ϕ s.t. v ≥ ϕ on clB
v(t, x) ≥ E[ϕ(τ,X ν
t,x(τ))],
where τ is the first exit time of (s,X νt,x(s))s≥t from B .
![Page 84: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/84.jpg)
The case of closed domain
•What appends if we replace O by clO ?
a. the easy part of the DPP still holds,b. our proof does not work for the difficult one.
• If a comparison principle holds for the associated PDE : our resultis in fact enough !
a. the sub-solution property still holds for v∗clO,b. by comparison vO∗ ≥ v∗clO,c. but vclO ≥ vO by definition.
![Page 85: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/85.jpg)
The case of closed domain
•What appends if we replace O by clO ?a. the easy part of the DPP still holds,
b. our proof does not work for the difficult one.
• If a comparison principle holds for the associated PDE : our resultis in fact enough !
a. the sub-solution property still holds for v∗clO,b. by comparison vO∗ ≥ v∗clO,c. but vclO ≥ vO by definition.
![Page 86: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/86.jpg)
The case of closed domain
•What appends if we replace O by clO ?a. the easy part of the DPP still holds,b. our proof does not work for the difficult one.
• If a comparison principle holds for the associated PDE : our resultis in fact enough !
a. the sub-solution property still holds for v∗clO,b. by comparison vO∗ ≥ v∗clO,c. but vclO ≥ vO by definition.
![Page 87: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/87.jpg)
The case of closed domain
•What appends if we replace O by clO ?a. the easy part of the DPP still holds,b. our proof does not work for the difficult one.
• If a comparison principle holds for the associated PDE : our resultis in fact enough !
a. the sub-solution property still holds for v∗clO,b. by comparison vO∗ ≥ v∗clO,c. but vclO ≥ vO by definition.
![Page 88: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/88.jpg)
The case of closed domain
•What appends if we replace O by clO ?a. the easy part of the DPP still holds,b. our proof does not work for the difficult one.
• If a comparison principle holds for the associated PDE : our resultis in fact enough !
a. the sub-solution property still holds for v∗clO,
b. by comparison vO∗ ≥ v∗clO,c. but vclO ≥ vO by definition.
![Page 89: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/89.jpg)
The case of closed domain
•What appends if we replace O by clO ?a. the easy part of the DPP still holds,b. our proof does not work for the difficult one.
• If a comparison principle holds for the associated PDE : our resultis in fact enough !
a. the sub-solution property still holds for v∗clO,b. by comparison vO∗ ≥ v∗clO,
c. but vclO ≥ vO by definition.
![Page 90: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/90.jpg)
The case of closed domain
•What appends if we replace O by clO ?a. the easy part of the DPP still holds,b. our proof does not work for the difficult one.
• If a comparison principle holds for the associated PDE : our resultis in fact enough !
a. the sub-solution property still holds for v∗clO,b. by comparison vO∗ ≥ v∗clO,c. but vclO ≥ vO by definition.
![Page 91: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/91.jpg)
Sufficient condition for comparison
Sufficient condition : Comparison holds if the super-solution is ofclass R(O) (for functions with polynomial growth).
Definition : w is of class R(O) if1. ∃ r > 0, an open neighborhood B of x in Rd and a function` : R+ → Rd such that
lim infε→0 ε−1|`(ε)| <∞ and
y + `(ε) + o(ε) ∈ O for all y ∈ clO ∩ B and ε ∈ (0, r).
2. ∃ λ : R+ → R+ such that
limε→0 λ(ε) = 0 andlimε→0 w
(t + λ(ε), x + `(ε)
)= w(t, x).
![Page 92: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/92.jpg)
Sufficient condition for comparison
Sufficient condition : Comparison holds if the super-solution is ofclass R(O) (for functions with polynomial growth).
Definition : w is of class R(O) if1. ∃ r > 0, an open neighborhood B of x in Rd and a function` : R+ → Rd such that
lim infε→0 ε−1|`(ε)| <∞ and
y + `(ε) + o(ε) ∈ O for all y ∈ clO ∩ B and ε ∈ (0, r).
2. ∃ λ : R+ → R+ such that
limε→0 λ(ε) = 0 andlimε→0 w
(t + λ(ε), x + `(ε)
)= w(t, x).
![Page 93: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/93.jpg)
Sufficient condition for comparison
Example : - There exists a C 1-function δ such that Dδ is locallyLipschitz continuous and
δ > 0 on O, δ = 0 on ∂O, δ < 0 outside clO.
- There exists a locally Lipschitz continuous mapping u : Rd → Us.t. for all x ∈ clO ∃ open neighborhood B of x and ι > 0 satisfying
µ(z , u(z))>Dδ(y) ≥ ι and σ(y , u(y)) = 0 ∀ y ∈ B ∩ clO , z ∈ B.
Similar conditions from the literature : Soner (1986),Katsoulakis (1994), Ishii and Loreti (2002).
![Page 94: Weak dynamic programming principle toward viscosity solutions · 2013-04-16 · Weak dynamic programming principle toward viscosity solutions Bruno Bouchard Ceremade and Crest University](https://reader034.vdocuments.site/reader034/viewer/2022042620/5f3c731c8d349c200e3fb1cd/html5/thumbnails/94.jpg)
Sufficient condition for comparison
Example : - There exists a C 1-function δ such that Dδ is locallyLipschitz continuous and
δ > 0 on O, δ = 0 on ∂O, δ < 0 outside clO.
- There exists a locally Lipschitz continuous mapping u : Rd → Us.t. for all x ∈ clO ∃ open neighborhood B of x and ι > 0 satisfying
µ(z , u(z))>Dδ(y) ≥ ι and σ(y , u(y)) = 0 ∀ y ∈ B ∩ clO , z ∈ B.
Similar conditions from the literature : Soner (1986),Katsoulakis (1994), Ishii and Loreti (2002).