![Page 1: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/1.jpg)
Stochastic particle methods in Bayesian statistical learning
P. Del Moral (INRIA team ALEA)
INRIA & Bordeaux Mathematical Institute & X CMAP
INRIA Workshop on Statistical Learning IHP Paris, Dec. 5-6, 2011
Some hyper-refs
I Feynman-Kac formulae, Genealogical & Interacting Particle Systems with appl., Springer (2004)
I Sequential Monte Carlo Samplers JRSS B. (2006). (joint work with Doucet & Jasra)
I On the concentration of interacting processes Hal-inria (2011). (joint work with Hu & Wu) [+ Refs]
I More references on the website http://www.math.u-bordeaux1.fr/∼delmoral/index.html [+ Links]
![Page 2: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/2.jpg)
Stochastic particle sampling methodsInteracting jumps modelsGenetic type interacting particle modelsParticle Feynman-Kac modelsThe 4 particle estimatesIsland particle models (⊂ Parallel Computing)
Bayesian statistical learningNonlinear filtering modelsFixed parameter estimation in HMM modelsParticle stochastic gradient modelsApproximate Bayesian ComputationInteracting Kalman-FiltersUncertainty propagations in numerical codes
Concentration inequalitiesCurrent population modelsParticle free energyGenealogical tree modelsBackward particle models
![Page 3: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/3.jpg)
Stochastic particle sampling methodsInteracting jumps modelsGenetic type interacting particle modelsParticle Feynman-Kac modelsThe 4 particle estimatesIsland particle models (⊂ Parallel Computing)
Bayesian statistical learning
Concentration inequalities
![Page 4: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/4.jpg)
Introduction
Stochastic particle methods=
Universal adaptive sampling technique
2 types of stochastic interacting particle models:
I Diffusive particle models with mean field drifts
[McKean-Vlasov style]
I Interacting jump particle models
[Boltzmann & Feynman-Kac style]
![Page 5: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/5.jpg)
Lectures ⊂ Interacting jumps models
I Interacting jumps = Recycling transitions =
I Discrete time models (⇔ geometric rejection/jump times)
![Page 6: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/6.jpg)
Genetic type interacting particle models
I Mutation-Proposals w.r.t. Markov transitions Xn−1 Xn ∈ En.
I Selection-Rejection-Recycling w.r.t. potential/fitness function Gn.
![Page 7: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/7.jpg)
Equivalent particle algorithms
Sequential Monte Carlo Sampling ResamplingParticle Filters Prediction Updating
Genetic Algorithms Mutation SelectionEvolutionary Population Exploration Branching-selectionDiffusion Monte Carlo Free evolutions AbsorptionQuantum Monte Carlo Walkers motions ReconfigurationSampling Algorithms Transition proposals Accept-reject-recycle
More botanical names:bootstrapping, spawning, cloning, pruning, replenish, multi-level splitting,enrichment, go with the winner, . . .
1950 ≤ Meta-Heuristic style stochastic algorithms ≤ 1996
![Page 8: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/8.jpg)
Equivalent particle algorithms
Sequential Monte Carlo Sampling ResamplingParticle Filters Prediction Updating
Genetic Algorithms Mutation SelectionEvolutionary Population Exploration Branching-selectionDiffusion Monte Carlo Free evolutions AbsorptionQuantum Monte Carlo Walkers motions ReconfigurationSampling Algorithms Transition proposals Accept-reject-recycle
More botanical names:bootstrapping, spawning, cloning, pruning, replenish, multi-level splitting,enrichment, go with the winner, . . .
1950 ≤ Meta-Heuristic style stochastic algorithms ≤ 1996
![Page 9: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/9.jpg)
A single stochastic model
Particle interpretation of Feynman-Kac path integrals
![Page 10: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/10.jpg)
Genealogical tree evolution(Population size, Time horizon)=(N , n) = (3, 3)
• - • • - • = •
• - • -
-
• • = •
• - • • -
--
• = •
Meta-heuristics ”Meta-Theorem” :Ancestral lines ' i.i.d. samples w.r.t. Feynman-Kac measure
Qn :=1
Zn
∏0≤p<n
Gp(Xp)
Pn with Pn := Law (X0, . . . ,Xn)
Example
Gn = 1An → Qn = Law((X0, . . . ,Xn) | Xp ∈ Ap, p < n)
![Page 11: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/11.jpg)
Genealogical tree evolution(Population size, Time horizon)=(N , n) = (3, 3)
• - • • - • = •
• - • -
-
• • = •
• - • • -
--
• = •
Meta-heuristics ”Meta-Theorem” :Ancestral lines ' i.i.d. samples w.r.t. Feynman-Kac measure
Qn :=1
Zn
∏0≤p<n
Gp(Xp)
Pn with Pn := Law (X0, . . . ,Xn)
Example
Gn = 1An → Qn = Law((X0, . . . ,Xn) | Xp ∈ Ap, p < n)
![Page 12: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/12.jpg)
Genealogical tree evolution(Population size, Time horizon)=(N , n) = (3, 3)
• - • • - • = •
• - • -
-
• • = •
• - • • -
--
• = •
Meta-heuristics ”Meta-Theorem” :Ancestral lines ' i.i.d. samples w.r.t. Feynman-Kac measure
Qn :=1
Zn
∏0≤p<n
Gp(Xp)
Pn with Pn := Law (X0, . . . ,Xn)
Example
Gn = 1An → Qn = Law((X0, . . . ,Xn) | Xp ∈ Ap, p < n)
![Page 13: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/13.jpg)
Particle estimatesMore formally(ξi0,n, ξ
i1,n, . . . , ξ
in,n
):= i-th ancetral line of the i-th current individual = ξin
⇓1
N
∑1≤i≤N
δ(ξi0,n,ξi1,n,...,ξin,n)−→N→∞ Qn
⊕ Current population models
ηNn :=1
N
∑1≤i≤N
δξin −→N→∞ ηn = n-th time marginal of Qn
⊕ Unbiased particle approximation
ZNn =
∏0≤p<n
ηNp (Gp) −→N→∞ Zn = E
∏0≤p<n
Gp(Xp)
=∏
0≤p<n
ηp(Gp)
Ex.: Gn = 1An ZNn =
∏proportion of success −→ P(Xp ∈ Ap, p < n)
![Page 14: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/14.jpg)
Particle estimatesMore formally(ξi0,n, ξ
i1,n, . . . , ξ
in,n
):= i-th ancetral line of the i-th current individual = ξin
⇓1
N
∑1≤i≤N
δ(ξi0,n,ξi1,n,...,ξin,n)−→N→∞ Qn
⊕ Current population models
ηNn :=1
N
∑1≤i≤N
δξin −→N→∞ ηn = n-th time marginal of Qn
⊕ Unbiased particle approximation
ZNn =
∏0≤p<n
ηNp (Gp) −→N→∞ Zn = E
∏0≤p<n
Gp(Xp)
=∏
0≤p<n
ηp(Gp)
Ex.: Gn = 1An ZNn =
∏proportion of success −→ P(Xp ∈ Ap, p < n)
![Page 15: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/15.jpg)
Particle estimatesMore formally(ξi0,n, ξ
i1,n, . . . , ξ
in,n
):= i-th ancetral line of the i-th current individual = ξin
⇓1
N
∑1≤i≤N
δ(ξi0,n,ξi1,n,...,ξin,n)−→N→∞ Qn
⊕ Current population models
ηNn :=1
N
∑1≤i≤N
δξin −→N→∞ ηn = n-th time marginal of Qn
⊕ Unbiased particle approximation
ZNn =
∏0≤p<n
ηNp (Gp) −→N→∞ Zn = E
∏0≤p<n
Gp(Xp)
=∏
0≤p<n
ηp(Gp)
Ex.: Gn = 1An ZNn =
∏proportion of success −→ P(Xp ∈ Ap, p < n)
![Page 16: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/16.jpg)
Particle estimatesMore formally(ξi0,n, ξ
i1,n, . . . , ξ
in,n
):= i-th ancetral line of the i-th current individual = ξin
⇓1
N
∑1≤i≤N
δ(ξi0,n,ξi1,n,...,ξin,n)−→N→∞ Qn
⊕ Current population models
ηNn :=1
N
∑1≤i≤N
δξin −→N→∞ ηn = n-th time marginal of Qn
⊕ Unbiased particle approximation
ZNn =
∏0≤p<n
ηNp (Gp) −→N→∞ Zn = E
∏0≤p<n
Gp(Xp)
=∏
0≤p<n
ηp(Gp)
Ex.: Gn = 1An ZNn =
∏proportion of success −→ P(Xp ∈ Ap, p < n)
![Page 17: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/17.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 18: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/18.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 19: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/19.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 20: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/20.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 21: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/21.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 22: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/22.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 23: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/23.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 24: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/24.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 25: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/25.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 26: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/26.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 27: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/27.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 28: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/28.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 29: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/29.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 30: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/30.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 31: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/31.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 32: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/32.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 33: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/33.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 34: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/34.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 35: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/35.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 36: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/36.jpg)
Graphical illustration : ηNn := 1N
∑1≤i≤N δξin ' ηn
![Page 37: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/37.jpg)
Complete ancestral tree when Mn(x , dy) = Hn(x , y) λ(dy)
Backward Markov chain model
QNn (d(x0, . . . , xn)) := ηNn (dxn) Mn,ηNn−1
(xn, dxn−1) . . .M1,ηN0(x1, dx0)
with the random particle matrices:
Mn+1,ηNn(xn+1, dxn) ∝ ηNn (dxn) Gn(xn) Hn+1(xn, xn+1)
Example: Normalized additive functionals
fn(x0, . . . , xn) =1
n + 1
∑0≤p≤n
fp(xp)
⇓
QNn (fn) :=
1
n + 1
∑0≤p≤n
ηNn Mn,ηNn−1. . .Mp+1,ηNp
(fp)︸ ︷︷ ︸matrix operations
![Page 38: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/38.jpg)
Complete ancestral tree when Mn(x , dy) = Hn(x , y) λ(dy)
Backward Markov chain model
QNn (d(x0, . . . , xn)) := ηNn (dxn) Mn,ηNn−1
(xn, dxn−1) . . .M1,ηN0(x1, dx0)
with the random particle matrices:
Mn+1,ηNn(xn+1, dxn) ∝ ηNn (dxn) Gn(xn) Hn+1(xn, xn+1)
Example: Normalized additive functionals
fn(x0, . . . , xn) =1
n + 1
∑0≤p≤n
fp(xp)
⇓
QNn (fn) :=
1
n + 1
∑0≤p≤n
ηNn Mn,ηNn−1. . .Mp+1,ηNp
(fp)︸ ︷︷ ︸matrix operations
![Page 39: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/39.jpg)
Island models (⊂ Parallel Computing)
Reminder : the unbiased property
E
fn(Xn)∏
0≤p<n
Gp(Xp)
= E
ηNn (fn)∏
0≤p<n
ηNp (Gp)
= E
Fn(Xn)∏
0≤p<n
Gp(Xp)
with the Island evolution Markov chain model
Xn := ηNn and Gn(Xn) = ηNn (Gn) = Xn (Gn)
⇓
particle model with (Xn,Gn(Xn)) = Interacting Island particle model
![Page 40: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/40.jpg)
Stochastic particle sampling methods
Bayesian statistical learningNonlinear filtering modelsFixed parameter estimation in HMM modelsParticle stochastic gradient modelsApproximate Bayesian ComputationInteracting Kalman-FiltersUncertainty propagations in numerical codes
Concentration inequalities
![Page 41: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/41.jpg)
Bayesian statistical learning
![Page 42: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/42.jpg)
Signal processing & filtering models
Law (Markov process X | Noisy & Partial observations Y )
I Signal X : target evolution (missile, plane, robot, vehicle, imagecontours), forecasting models, assets volatility, speech signals, ...
I Observation Y : Radar/Sonar/Gps sensors, financial assets prices,image processing, audio receivers, statistical data measurements, ...
⊂ Multiple objects tracking models (highly more complex pb)
I On the Stability and the Approximation of Branching Distribution Flows, with Applications
to Nonlinear Multiple Target Filtering. Francois Caron, Pierre Del Moral, Michele Pace, and B.-N.Vo (HAL-INRIA RR-7376) [50p]. Stoch. Analysis and Applications Volume 29, Issue 6, 2011.
I Comparison of implementations of Gaussian mixture PHD filters. M. Pace, P. Del Moral, Fr.Caron 13th International Conference on Information. FUSION, EICC, Edinburgh, UK, 26-29 July (2010)
Law
X =∑
1≤i≤NXt
δXit
∣∣∣∣∣∣∣ Y =∑
1≤i≤NYt
δY it
![Page 43: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/43.jpg)
Signal processing & filtering models
Law (Markov process X | Noisy & Partial observations Y )
I Signal X : target evolution (missile, plane, robot, vehicle, imagecontours), forecasting models, assets volatility, speech signals, ...
I Observation Y : Radar/Sonar/Gps sensors, financial assets prices,image processing, audio receivers, statistical data measurements, ...
⊂ Multiple objects tracking models (highly more complex pb)
I On the Stability and the Approximation of Branching Distribution Flows, with Applications
to Nonlinear Multiple Target Filtering. Francois Caron, Pierre Del Moral, Michele Pace, and B.-N.Vo (HAL-INRIA RR-7376) [50p]. Stoch. Analysis and Applications Volume 29, Issue 6, 2011.
I Comparison of implementations of Gaussian mixture PHD filters. M. Pace, P. Del Moral, Fr.Caron 13th International Conference on Information. FUSION, EICC, Edinburgh, UK, 26-29 July (2010)
Law
X =∑
1≤i≤NXt
δXit
∣∣∣∣∣∣∣ Y =∑
1≤i≤NYt
δY it
![Page 44: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/44.jpg)
Filtering (prediction ⊕ smoothing)
p((x0, . . . , xn) | (y0, . . . , yn)) & p(y0, . . . , yn) ?
Bayes’ rule
p((x0, . . . , xn) | (y0, . . . , yn)) ∝ p((y0, . . . , yn) | (x0, . . . , xn))︸ ︷︷ ︸∏0≤k≤n p(yk |xk )←likelihood functions Gk
×p(x0, . . . , xn)
⇓
Feynman-Kac models : Gn(xn) := p(yn|xn) & Pn := Law (X0, . . . ,Xn)
Law ((X0, . . . ,Xn) | Yp = yp, p < n) =1
Zn
∏0≤p<n
Gp(Xp)
Pn
Not unique stochastic model!
![Page 45: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/45.jpg)
Filtering (prediction ⊕ smoothing)
p((x0, . . . , xn) | (y0, . . . , yn)) & p(y0, . . . , yn) ?
Bayes’ rule
p((x0, . . . , xn) | (y0, . . . , yn)) ∝ p((y0, . . . , yn) | (x0, . . . , xn))︸ ︷︷ ︸∏0≤k≤n p(yk |xk )←likelihood functions Gk
×p(x0, . . . , xn)
⇓
Feynman-Kac models : Gn(xn) := p(yn|xn) & Pn := Law (X0, . . . ,Xn)
Law ((X0, . . . ,Xn) | Yp = yp, p < n) =1
Zn
∏0≤p<n
Gp(Xp)
Pn
Not unique stochastic model!
![Page 46: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/46.jpg)
Filtering (prediction ⊕ smoothing)
p((x0, . . . , xn) | (y0, . . . , yn)) & p(y0, . . . , yn) ?
Bayes’ rule
p((x0, . . . , xn) | (y0, . . . , yn)) ∝ p((y0, . . . , yn) | (x0, . . . , xn))︸ ︷︷ ︸∏0≤k≤n p(yk |xk )←likelihood functions Gk
×p(x0, . . . , xn)
⇓
Feynman-Kac models : Gn(xn) := p(yn|xn) & Pn := Law (X0, . . . ,Xn)
Law ((X0, . . . ,Xn) | Yp = yp, p < n) =1
Zn
∏0≤p<n
Gp(Xp)
Pn
Not unique stochastic model!
![Page 47: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/47.jpg)
Hidden Markov chains problems
Θ Signal XΘ observations Y Θ
Law(fixed parameter Θ
∣∣ Noisy & Partial observations Y Θ)
I Parameter Θ : kinetic model unknown parameters, statisticalparameters (signal/sensors), hypothesis testing, ..
I Signal XΘ : Single or multiple targets evolution, forecastingmodels, financial assets volatility, speech signals, video images, ...
I Observation Y Θ : Radar/Sonar/Gps sensors, financial assetsprices, image processing, statistical data measurements, ...
![Page 48: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/48.jpg)
Posterior density
p( θ | (y0, . . . , yn)) ∝ p((y0, . . . , yn) | θ)︸ ︷︷ ︸∏0≤k≤n p(yk |θ,(y0,...,yk−1))←likelihood functions
× p(θ)
⇓
Law (Θ | (y0, . . . , yn)) ∝
∏0≤p≤n
hp(θ)
λ(dθ)
withhn(θ) := p(yn|θ, (y0, . . . , yn−1)) & λ := Law (Θ)
![Page 49: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/49.jpg)
First key observation
p((y0, . . . , yn)|θ) =∏
0≤p≤n
hp(θ) = Zn(θ)
with the normalizing constant Zn(θ) of the conditional distribution
p((x0, . . . , xn)|(y0, . . . , yn), θ)
=1
p((y0, . . . , yn)|θ)p((y0, . . . , yn)|(x0, . . . , xn), θ) p((x0, . . . , xn)|θ)
Second key observation
hn(θ) and Zn(θ) easy to compute for linear/gaussian models
![Page 50: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/50.jpg)
Third key observation : Any target measure of the form
ηn(dθ) =1
Zn
∏0≤p≤n
hp(θ)
× λ(dθ)
is the n-th time marginal of the Feynman-Kac measure
Qn :=1
Zn
∏0≤p<n
Gp(Θp)
Pn
withGn = hn+1 and Pn := Law (Θ0, . . . ,Θn)
where
Θp−1 Θp as an MCMC move with target measure ηp
![Page 51: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/51.jpg)
Particle auxiliary variables θ ξθ ∼ P(θ, dξ)
ηn(dθ) ∝
∏0≤p≤n
hp(θ)
λ(dθ)︸ ︷︷ ︸=λ(dθ)×P(θ,dξ)
with θ = (θ, ξ) and
hn(θ) :=1
N
N∑i=1
p(yn | ξθ,in ) 'N↑∞ p(yn|θ, (y0, . . . , yn−1)) = hp(θ)
But by the unbiased property the θ-marginal of ηn coincides with
Law (Θ | (y0, . . . , yn)) ∝
∏0≤p≤n
hp(θ)
λ(dθ)
Feynman-Kac formulation :Ref. Markov chain Θk =
(Θk , ξ
(k))
MCMC with target ηn and Gn = hn+1
![Page 52: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/52.jpg)
Particle auxiliary variables θ ξθ ∼ P(θ, dξ)
ηn(dθ) ∝
∏0≤p≤n
hp(θ)
λ(dθ)︸ ︷︷ ︸=λ(dθ)×P(θ,dξ)
with θ = (θ, ξ) and
hn(θ) :=1
N
N∑i=1
p(yn | ξθ,in ) 'N↑∞ p(yn|θ, (y0, . . . , yn−1)) = hp(θ)
But by the unbiased property the θ-marginal of ηn coincides with
Law (Θ | (y0, . . . , yn)) ∝
∏0≤p≤n
hp(θ)
λ(dθ)
Feynman-Kac formulation :Ref. Markov chain Θk =
(Θk , ξ
(k))
MCMC with target ηn and Gn = hn+1
![Page 53: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/53.jpg)
Particle steepest descent gradient models
Zn(θ) = p((y0, . . . , yn−1) | θ) = E
∏0≤q<n
p(yq | θ, X θq )
⇓ (θ ∈ Rd)
∇ logZn(θ) = Q(θ)n (Λn)
with the Feynman-Kac measure Q(θ)n on path space associated with
(X θn ,G
θn (xn)) = (X θ
n , p(yq | θ, xn))
and with the additive functional
Λn(x0, . . . , xn) =∑
0≤p<n
∇ log (p(xq+1|θ, xq)p(yq | θ, xq))
Particle gradient algorithm
Θn = Θn−1 + τn ∇Q(θ)n (Λn) ' Θn−1 + τn ∇Q(θ),N
n (Λn)
![Page 54: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/54.jpg)
Particle steepest descent gradient models
Zn(θ) = p((y0, . . . , yn−1) | θ) = E
∏0≤q<n
p(yq | θ, X θq )
⇓ (θ ∈ Rd)
∇ logZn(θ) = Q(θ)n (Λn)
with the Feynman-Kac measure Q(θ)n on path space associated with
(X θn ,G
θn (xn)) = (X θ
n , p(yq | θ, xn))
and with the additive functional
Λn(x0, . . . , xn) =∑
0≤p<n
∇ log (p(xq+1|θ, xq)p(yq | θ, xq))
Particle gradient algorithm
Θn = Θn−1 + τn ∇Q(θ)n (Λn) ' Θn−1 + τn ∇Q(θ),N
n (Λn)
![Page 55: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/55.jpg)
Approximate Bayesian Computation
When p(yn|xn) is untractable or impossible to compute in reasonable time
{Xn = Fn(Xn−1,Wn)Yn = Hn(Xn,Vn)
Xn=(Xn,Yn)−−−−−−−−−−−−→
{Xn = Fn(Xn−1,Wn)Y εn = Yn + ε V ε
n
⇓
Law (X | Y ε = y ) 'ε↓0 Law (X | Y = y )
⇓
Feynman-Kac model with the Markov chain and the potentials :
Xn = (Xn,Yn) and Gn(Xn) = p(Y εn |Yn)
![Page 56: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/56.jpg)
Interacting Kalman-Filters
Xn = (X 1n ,X
2n ) with X 1
n Markov and (X 2n ,Yn)|X 1 linear-gaussian model{
X 2n = An(X 1
n ) X 2n−1 + Bn(X 1
n ) Wn
Yn = Cn(X 1n ) X 2
n + Dn(X 1n ) Vn
⇓
Law(X 2n
∣∣ X 1, Yp = yp, p < n)
= ηX 1,n = Kalman gaussian predictor
Integration over X 1 ⇒ Law((X 1,X 2) | Y
)= Feynman-Kac model
with the reference Markov chain and the gaussian potential
Xn =(X 1n , ηX 1,n
)& Gn(Xn) =
∫p(Yn | (x1
n , x2n )) ηX 1,n(dx2
n )
![Page 57: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/57.jpg)
Interacting Kalman-Filters
Xn = (X 1n ,X
2n ) with X 1
n Markov and (X 2n ,Yn)|X 1 linear-gaussian model{
X 2n = An(X 1
n ) X 2n−1 + Bn(X 1
n ) Wn
Yn = Cn(X 1n ) X 2
n + Dn(X 1n ) Vn
⇓
Law(X 2n
∣∣ X 1, Yp = yp, p < n)
= ηX 1,n = Kalman gaussian predictor
Integration over X 1 ⇒ Law((X 1,X 2) | Y
)= Feynman-Kac model
with the reference Markov chain and the gaussian potential
Xn =(X 1n , ηX 1,n
)& Gn(Xn) =
∫p(Yn | (x1
n , x2n )) ηX 1,n(dx2
n )
![Page 58: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/58.jpg)
I Uncertainty propagations in numerical codes
Law (Inputs I | Outputs O = C (I) ∈ Reference or Critical event )
m
µ = Law(I)A = {I : C (I) ∈ B}
}−→ P (I ∈ A) = µ(A) & Law (I | I ∈ A) = µA
Multi-level decomposition
hn = 1An with An ↓ =⇒ µAn(dx) ∝
∏0≤p≤n
hp(x)
µ(dx)
Feynman-Kac representation
(Xn−1 Xn) = an MCMC move with target µAn & Gn = 1An+1
![Page 59: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/59.jpg)
I Uncertainty propagations in numerical codes
Law (Inputs I | Outputs O = C (I) ∈ Reference or Critical event )
m
µ = Law(I)A = {I : C (I) ∈ B}
}−→ P (I ∈ A) = µ(A) & Law (I | I ∈ A) = µA
Multi-level decomposition
hn = 1An with An ↓ =⇒ µAn(dx) ∝
∏0≤p≤n
hp(x)
µ(dx)
Feynman-Kac representation
(Xn−1 Xn) = an MCMC move with target µAn & Gn = 1An+1
![Page 60: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/60.jpg)
Stochastic particle sampling methods
Bayesian statistical learning
Concentration inequalitiesCurrent population modelsParticle free energyGenealogical tree modelsBackward particle models
![Page 61: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/61.jpg)
Current population models
Constants (c1, c2) related to (bias,variance), c universal constantTest funct. ‖fn‖ ≤ 1
I ∀ (x ≥ 0, n ≥ 0,N ≥ 1), the probability of the event[ηNn − ηn
](f ) ≤ c1
N
(1 + x +
√x)
+c2√N
√x
is greater than 1− e−x .
I x = (xi )1≤i≤d (−∞, x ] =∏d
i=1(−∞, xi ] cells in En = Rd .
Fn(x) = ηn(1(−∞,x]
)and FN
n (x) = ηNn(1(−∞,x]
)∀ (y ≥ 0, n ≥ 0,N ≥ 1), the probability of the following event
√N∥∥FN
n − Fn
∥∥ ≤ c√d (y + 1)
is greater than 1− e−x .
![Page 62: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/62.jpg)
Particle free energy models
Constants (c1, c2) related to (bias,variance), c universal constant.
I ∀ (y ≥ 0, n ≥ 0,N ≥ 1, ε ∈ {+1,−1}), the probability of the event
ε
nlogZN
n
Zn≤ c1
N
(1 + x +
√x)
+c2√N
√x
is greater than 1− e−x .
Note : (0 ≤ ε ≤ 1⇒ (1− e−ε) ∨ (eε − 1) ≤ 2ε)
e−ε ≤ zN
z≤ eε ⇒
∣∣∣∣zNz − 1
∣∣∣∣ ≤ 2ε
![Page 63: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/63.jpg)
Genealogical tree models := ηNn (in path space)
Constants (c1, c2) related to (bias,variance), c universal constantfn test function ‖fn‖ ≤ 1.
I ∀ (y ≥ 0, n ≥ 0,N ≥ 1), the probability of the event
[ηNn −Qn
](f ) ≤ c1
n + 1
N
(1 + x +
√x)
+ c2
√(n + 1)
N
√x
is greater than 1− e−x .
I Fn = indicator fct. fn of cells in En =(Rd0 × . . . ,×Rdn
)∀ (y ≥ 0, n ≥ 0,N ≥ 1), the probability of the following event
supfn∈Fn
∣∣ηNn (fn)−Qn(fn)∣∣ ≤ c (n + 1)
√∑0≤p≤n dp
N(x + 1)
is greater than 1− e−x .
![Page 64: Stochastic particle methods in Bayesian statistical …people.bordeaux.inria.fr/pierre.delmoral/stat-learning.pdfStochastic particle methods in Bayesian statistical learning P. Del](https://reader034.vdocuments.site/reader034/viewer/2022042920/5f651ae3b666067fdf6f5e8e/html5/thumbnails/64.jpg)
Backward particle models
Constants (c1, c2) related to (bias,variance), c universal constant.fn normalized additive functional with ‖fp‖ ≤ 1.
I ∀ (x ≥ 0, n ≥ 0,N ≥ 1), the probability of the event
[QN
n −Qn
](fn) ≤ c1
1
N(1 + (x +
√x)) + c2
√x
N(n + 1)
is greater than 1− e−x .
I fa,n normalized additive functional w.r.t. fp = 1(−∞,a], a ∈ Rd = En
.
∀ (x ≥ 0, n ≥ 0,N ≥ 1), the probability of the following event
supa∈Rd
∣∣QNn (fa,n)−Qn(fa,n)
∣∣ ≤ c
√d
N(x + 1)
is greater than 1− e−x .