![Page 1: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/1.jpg)
Controlled Interacting Particle Systemsfor Estimation and Sampling
Amirhossein Taghvaei
Postdoctoral ScholarDepartment of Mechanical and Aerospace Engineering
University of California, Irvine
Presented at the Department of Aeronautics & AstronauticsUniversity of Washington, Seattle
Feb 25, 2021
![Page 2: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/2.jpg)
Uncertainty is everywhere
GPS location
weather forecast COVID-19
We have to deal withuncertainty
I. Stewart, Do Dice Play God? The Mathematics of Uncertainty. United States: Basic Books. 2019
Amirhossein Taghvaei 1 / 31 Amirhossein Taghvaei
![Page 3: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/3.jpg)
Uncertainty is everywhere
GPS locationweather forecast
COVID-19
We have to deal withuncertainty
I. Stewart, Do Dice Play God? The Mathematics of Uncertainty. United States: Basic Books. 2019
Amirhossein Taghvaei 1 / 31 Amirhossein Taghvaei
![Page 4: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/4.jpg)
Uncertainty is everywhere
GPS locationweather forecast
COVID-19
We have to deal withuncertainty
I. Stewart, Do Dice Play God? The Mathematics of Uncertainty. United States: Basic Books. 2019
Amirhossein Taghvaei 1 / 31 Amirhossein Taghvaei
![Page 5: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/5.jpg)
Uncertainty is everywhere
GPS locationweather forecast
COVID-19
uncertainty
imperfect models noise in measurements
stochastic algorithms
We have to deal withuncertainty
I. Stewart, Do Dice Play God? The Mathematics of Uncertainty. United States: Basic Books. 2019
Amirhossein Taghvaei 1 / 31 Amirhossein Taghvaei
![Page 6: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/6.jpg)
Uncertainty is everywhere
GPS locationweather forecast
COVID-19
uncertainty
imperfect models noise in measurements
stochastic algorithms
We have to deal withuncertainty
I. Stewart, Do Dice Play God? The Mathematics of Uncertainty. United States: Basic Books. 2019
Amirhossein Taghvaei 1 / 31 Amirhossein Taghvaei
![Page 7: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/7.jpg)
To be certain about uncertainty
Probability theory: (quantify uncertainty)
Optimal transport theory: (geometry for distributions)
Nobel prize (1975) Fields medal (2010)
Allows application of control and optimization techniques to distributions
Amirhossein Taghvaei 2 / 31 Amirhossein Taghvaei
![Page 8: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/8.jpg)
To be certain about uncertainty
Probability theory: (quantify uncertainty)
Optimal transport theory: (geometry for distributions)
Nobel prize (1975) Fields medal (2010)
Allows application of control and optimization techniques to distributions
Amirhossein Taghvaei 2 / 31 Amirhossein Taghvaei
![Page 9: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/9.jpg)
To be certain about uncertainty
Probability theory: (quantify uncertainty)
Optimal transport theory: (geometry for distributions)
Nobel prize (1975)Fields medal (2010)
Allows application of control and optimization techniques to distributions
Amirhossein Taghvaei 2 / 31 Amirhossein Taghvaei
![Page 10: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/10.jpg)
To be certain about uncertainty
Probability theory: (quantify uncertainty)
Optimal transport theory: (geometry for distributions)
Nobel prize (1975)Fields medal (2010)
Allows application of control and optimization techniques to distributions
Amirhossein Taghvaei 2 / 31 Amirhossein Taghvaei
![Page 11: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/11.jpg)
Research overview
Main theme: Control/Optimization for probability distributions
Optimal transporttheory
Nonlinear Estimation/filtering → this talk!
publications in IEEE TAC, SIAM UQ, ASME, IEEE CSM
Machine learning
publications in NeurIPS 17, ICML 19, ICML 20
Stochastic thermodynamics
publications in Automatica, Scientific reports
Amirhossein Taghvaei 3 / 31 Amirhossein Taghvaei
![Page 12: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/12.jpg)
Research overview
Main theme: Control/Optimization for probability distributions
Optimal transporttheory
Nonlinear Estimation/filtering → this talk!
publications in IEEE TAC, SIAM UQ, ASME, IEEE CSM
Machine learning
publications in NeurIPS 17, ICML 19, ICML 20
Stochastic thermodynamics
publications in Automatica, Scientific reports
Amirhossein Taghvaei 3 / 31 Amirhossein Taghvaei
![Page 13: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/13.jpg)
Research overview
Main theme: Control/Optimization for probability distributions
Optimal transporttheory
Nonlinear Estimation/filtering → this talk!
publications in IEEE TAC, SIAM UQ, ASME, IEEE CSM
Machine learning
publications in NeurIPS 17, ICML 19, ICML 20
Stochastic thermodynamics
publications in Automatica, Scientific reports
Amirhossein Taghvaei 3 / 31 Amirhossein Taghvaei
![Page 14: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/14.jpg)
Outline
Part I: background and motivation
Part II: mean-field control design
Part III: gain function approximation
Part IV: applications
Amirhossein Taghvaei 3 / 31 Amirhossein Taghvaei
![Page 15: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/15.jpg)
Outline
Part I: background and motivation
Part II: mean-field control design
Part III: gain function approximation
Part IV: applications
Amirhossein Taghvaei 3 / 31 Amirhossein Taghvaei
![Page 16: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/16.jpg)
Nonlinear filtering problem
state observationposterior distribution
Hidden state: physical activity
Observation: inertial motion sensors
Problem: estimate the state based on observation
Probabilistic approach: compute the conditional probability distribution (posterior)
Internship at university start-up company, Rithmio, 2014-2015
Amirhossein Taghvaei 4 / 31 Amirhossein Taghvaei
![Page 17: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/17.jpg)
Nonlinear filtering problem
state observationposterior distribution
Hidden state: physical activity
Observation: inertial motion sensors
Problem: estimate the state based on observation
Probabilistic approach: compute the conditional probability distribution (posterior)
Internship at university start-up company, Rithmio, 2014-2015
Amirhossein Taghvaei 4 / 31 Amirhossein Taghvaei
![Page 18: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/18.jpg)
Nonlinear filtering problemMathematical model
State process: Xk ∼ a(·|Xk−1), X0 ∼ π0(·)
Observation process: Zk ∼ l(·|Xk)
Objective: compute πk(·) := P(Xk ∈ ·|Z1:k)
In principle: recursive update for posterior
πkdynamics
−−−−→ πkcorrection
−−−−→Bayes rule
πk+1
In practice: no finite-dimensional solution
notable exception: linear Gaussian case
J. Xiong, An introduction to stochastic filtering theory, 2008
Amirhossein Taghvaei 5 / 31 Amirhossein Taghvaei
![Page 19: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/19.jpg)
Nonlinear filtering problemMathematical model
State process: Xk ∼ a(·|Xk−1), X0 ∼ π0(·)
Observation process: Zk ∼ l(·|Xk)
Objective: compute πk(·) := P(Xk ∈ ·|Z1:k)
In principle: recursive update for posterior
πkdynamics
−−−−→ πkcorrection
−−−−→Bayes rule
πk+1
In practice: no finite-dimensional solution
notable exception: linear Gaussian case
J. Xiong, An introduction to stochastic filtering theory, 2008
Amirhossein Taghvaei 5 / 31 Amirhossein Taghvaei
![Page 20: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/20.jpg)
Nonlinear filtering problemMathematical model
State process: Xk ∼ a(·|Xk−1), X0 ∼ π0(·)
Observation process: Zk ∼ l(·|Xk)
Objective: compute πk(·) := P(Xk ∈ ·|Z1:k)
In principle: recursive update for posterior
πkdynamics
−−−−→ πkcorrection
−−−−→Bayes rule
πk+1
In practice: no finite-dimensional solution
notable exception: linear Gaussian case
J. Xiong, An introduction to stochastic filtering theory, 2008
Amirhossein Taghvaei 5 / 31 Amirhossein Taghvaei
![Page 21: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/21.jpg)
Nonlinear filtering problemContinuous-time formulation
State process: dXt = (dynamic model) X0 ∼ π0(·)
Observation process: dZt = h(Xt)dt+ dWt
Objective: compute πt(·) := P(Xt ∈ ·|Z[0,t])
In principle: continuous update law for posterior
dπt = (dynamics) + (correction)
In practice: no finite-dimensional solution
notable exception: linear Gaussian case
J. Xiong, An introduction to stochastic filtering theory, 2008
Amirhossein Taghvaei 6 / 31 Amirhossein Taghvaei
![Page 22: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/22.jpg)
Nonlinear filtering problemContinuous-time formulation
State process: dXt = (dynamic model) X0 ∼ π0(·)
Observation process: dZt = h(Xt)dt+ dWt
Yt :=d
dtZt = h(Xt) + white noise
Objective: compute πt(·) := P(Xt ∈ ·|Z[0,t])
In principle: continuous update law for posterior
dπt = (dynamics) + (correction)
In practice: no finite-dimensional solution
notable exception: linear Gaussian case
J. Xiong, An introduction to stochastic filtering theory, 2008
Amirhossein Taghvaei 6 / 31 Amirhossein Taghvaei
![Page 23: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/23.jpg)
Nonlinear filtering problemContinuous-time formulation
State process: dXt = (dynamic model) X0 ∼ π0(·)
Observation process: dZt = h(Xt)dt+ dWt
Objective: compute πt(·) := P(Xt ∈ ·|Z[0,t])
In principle: continuous update law for posterior
dπt = (dynamics) + (correction)
In practice: no finite-dimensional solution
notable exception: linear Gaussian case
J. Xiong, An introduction to stochastic filtering theory, 2008
Amirhossein Taghvaei 6 / 31 Amirhossein Taghvaei
![Page 24: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/24.jpg)
Nonlinear filtering problemContinuous-time formulation
State process: dXt = (dynamic model) X0 ∼ π0(·)
Observation process: dZt = h(Xt)dt+ dWt
Objective: compute πt(·) := P(Xt ∈ ·|Z[0,t])
In principle: continuous update law for posterior
dπt = (dynamics) + (correction)
In practice: no finite-dimensional solution
notable exception: linear Gaussian case
J. Xiong, An introduction to stochastic filtering theory, 2008
Amirhossein Taghvaei 6 / 31 Amirhossein Taghvaei
![Page 25: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/25.jpg)
Kalman-Bucy filterLinear Gaussian setting
linear dynamics: AXtdt+ dBt
linear observation: h(x) = Hx
Kalman filter: posterior πt is Gaussian N(mt,Σt)
Update for mean: dmt = Amtdt︸ ︷︷ ︸dynamics
+ Kt(dZt −Hmtdt)︸ ︷︷ ︸correction
Update for variance: Σt = (Ricatti equation)
Kalman gain: Kt = ΣtHT
Properties:
strong theoretical properties
if state dimension is d, computational cost O(d2) → Ensemble Kalman filter
exact only in linear Gaussian setting → Particle filter
R. E. Kalman, A New Approach to Linear Filtering and Prediction Problems, 1960R. E. Kalman and R. S. Bucy. New results in linear filtering and prediction theory, 1961
Amirhossein Taghvaei 7 / 31 Amirhossein Taghvaei
![Page 26: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/26.jpg)
Kalman-Bucy filterLinear Gaussian setting
linear dynamics: AXtdt+ dBt
linear observation: h(x) = Hx
Kalman filter: posterior πt is Gaussian N(mt,Σt)
Update for mean: dmt = Amtdt︸ ︷︷ ︸dynamics
+ Kt(dZt −Hmtdt)︸ ︷︷ ︸correction
Update for variance: Σt = (Ricatti equation)
Kalman gain: Kt = ΣtHT
Properties:
strong theoretical properties
if state dimension is d, computational cost O(d2) → Ensemble Kalman filter
exact only in linear Gaussian setting → Particle filter
R. E. Kalman, A New Approach to Linear Filtering and Prediction Problems, 1960R. E. Kalman and R. S. Bucy. New results in linear filtering and prediction theory, 1961
Amirhossein Taghvaei 7 / 31 Amirhossein Taghvaei
![Page 27: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/27.jpg)
Kalman-Bucy filterLinear Gaussian setting
linear dynamics: AXtdt+ dBt
linear observation: h(x) = Hx
Kalman filter: posterior πt is Gaussian N(mt,Σt)
Update for mean: dmt = Amtdt︸ ︷︷ ︸dynamics
+ Kt(dZt −Hmtdt)︸ ︷︷ ︸correction
Update for variance: Σt = (Ricatti equation)
Kalman gain: Kt = ΣtHT
Properties:
strong theoretical properties
if state dimension is d, computational cost O(d2) → Ensemble Kalman filter
exact only in linear Gaussian setting → Particle filter
R. E. Kalman, A New Approach to Linear Filtering and Prediction Problems, 1960R. E. Kalman and R. S. Bucy. New results in linear filtering and prediction theory, 1961
Amirhossein Taghvaei 7 / 31 Amirhossein Taghvaei
![Page 28: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/28.jpg)
Ensemble Kalman filterMonte-Carlo approximation of Kalman filter
Posterior distribution: πt ≈1
N
N∑i=1
δXit
Update for particles: dXit = (dynamics) + K
(N)t (dZt −
HXit +Hm
(N)t
2dt)︸ ︷︷ ︸
correction
Kalman gain: K(N)t = Σ
(N)t HT
exact mean and variance in mean-field limit (N →∞)
computational cost O(Nd), efficient when d� N
not exact in nonlinear setting
G. Evensen. Sequential data assimilation with a nonlinear quasi-geostrophic model . . . 1994.K. Bergemann and S. Reich. An ensemble Kalman-Bucy filter for continuous data assimilation, 2012
Amirhossein Taghvaei 8 / 31 Amirhossein Taghvaei
![Page 29: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/29.jpg)
Ensemble Kalman filterMonte-Carlo approximation of Kalman filter
Posterior distribution: πt ≈1
N
N∑i=1
δXit
Update for particles: dXit = (dynamics) + K
(N)t (dZt −
HXit +Hm
(N)t
2dt)︸ ︷︷ ︸
correction
Kalman gain: K(N)t = Σ
(N)t HT
exact mean and variance in mean-field limit (N →∞)
computational cost O(Nd), efficient when d� N
not exact in nonlinear setting
G. Evensen. Sequential data assimilation with a nonlinear quasi-geostrophic model . . . 1994.K. Bergemann and S. Reich. An ensemble Kalman-Bucy filter for continuous data assimilation, 2012
Amirhossein Taghvaei 8 / 31 Amirhossein Taghvaei
![Page 30: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/30.jpg)
Particle filtersSequential importance sampling
Algorithm:
approximate πt with weighted empirical distributionN∑i=1
wiδXit
update the weights using observation and Bayes rule (importance sampling)
Properties:
exact in the limit as N →∞weight degeneracy → curse of dimensionality
no feedback control structure
N. Gordon, D. Salmond, and A. Smith. Novel approach to nonlinear/non-Gaussian Bayesian state estimation (1993).P. Del Moral, A.Guionnet. On the stability of interacting processes with applications to filtering and genetic algorithms. (2001)A. Doucet and A. Johansen, A Tutorial on Particle Filtering and Smoothing: Fifteen years later (2008).P. Bickel, B. Li, and T. Bengtsson, Sharp failure rates for the bootstrap particle filter in high dimensions (2008).
Amirhossein Taghvaei 9 / 31 Amirhossein Taghvaei
![Page 31: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/31.jpg)
Particle filtersSequential importance sampling
Algorithm:
approximate πt with weighted empirical distributionN∑i=1
wiδXit
update the weights using observation and Bayes rule (importance sampling)
Properties:
exact in the limit as N →∞weight degeneracy → curse of dimensionality
no feedback control structure
N. Gordon, D. Salmond, and A. Smith. Novel approach to nonlinear/non-Gaussian Bayesian state estimation (1993).P. Del Moral, A.Guionnet. On the stability of interacting processes with applications to filtering and genetic algorithms. (2001)A. Doucet and A. Johansen, A Tutorial on Particle Filtering and Smoothing: Fifteen years later (2008).P. Bickel, B. Li, and T. Bengtsson, Sharp failure rates for the bootstrap particle filter in high dimensions (2008).
Amirhossein Taghvaei 9 / 31 Amirhossein Taghvaei
![Page 32: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/32.jpg)
Curse of dimensionalityLinear Gaussian setting
number of particles to achieve error ε:
1 2 3 4 5 6 7 8 9 10
5
6
7
8
9
log(
N)
d
N2d
0.010.02
0.05
0.10
m.s.e (PF) EnKF
PF: N ≈ O(ed
ε) EnKF: N ≈ O(
d2
ε)
Question: Can we generalize EnKF to nonlinear setting?
S. C. Surace, A. Kutschireiter, J. Pfister, How to avoid the curse of dimensionality: scalability of particle filters . . . , SIAM review, 2019A. Taghvaei, P. G. Mehta, An optimal transport formulation of ensemble Kalman filter, (TAC) 2020
Amirhossein Taghvaei 10 / 31 Amirhossein Taghvaei
![Page 33: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/33.jpg)
Part I: summary
Kalman filter Ensemble Kalman filter Particle filter
KF: dmt = (dynamics) + Kt(dZt −Hmtdt)
EnKF: dXit = (dynamics) + K
(N)t (dZt −
HXik +Hm
(N)t
2dt)
PF: no feedback control structure
Question: Can we generalize EnKF to nonlinear and non-Gaussian setting?
Amirhossein Taghvaei 11 / 31 Amirhossein Taghvaei
![Page 34: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/34.jpg)
Outline
Part I: background and motivation
Part II: mean-field control design
Part III: gain function approximation
Part III: applications
References:
A. Taghvaei, P. G. Mehta, Optimal Transportation Methods in Nonlinear Filtering: Thefeedback particle filter, IEEE Control Systems Magazine (CSM), Accepted
A. Taghvaei, P. G. Mehta, An optimal transport formulation of the Ensemble Kalman filter,Transactions of Automatic Control (TAC), Accepted
Amirhossein Taghvaei 11 / 31 Amirhossein Taghvaei
![Page 35: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/35.jpg)
Outline
Part I: background and motivation
Part II: mean-field control design
Part III: gain function approximation
Part III: applications
References:
A. Taghvaei, P. G. Mehta, Optimal Transportation Methods in Nonlinear Filtering: Thefeedback particle filter, IEEE Control Systems Magazine (CSM), Accepted
A. Taghvaei, P. G. Mehta, An optimal transport formulation of the Ensemble Kalman filter,Transactions of Automatic Control (TAC), Accepted
Amirhossein Taghvaei 11 / 31 Amirhossein Taghvaei
![Page 36: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/36.jpg)
Mean-field design
State process: dXt = (dynamic model) X0 ∼ π0(·)
Observation process: dZt = h(Xt)dt+ dWt
Objective: compute πt(·) := P(Xt ∈ ·|Z[0,t])
Idea:
construct a controlled stochastic process
dXt = (dynamics) + ut(Xt)dt+ Kt(Xt)dZt︸ ︷︷ ︸control
design the control law such that
Xt ∼ πt, ∀t ≥ 0 (exactness)
realize Xt with N particles {X1t , . . . , X
Nt }
Amirhossein Taghvaei 12 / 31 Amirhossein Taghvaei
![Page 37: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/37.jpg)
Mean-field design
State process: dXt = (dynamic model) X0 ∼ π0(·)
Observation process: dZt = h(Xt)dt+ dWt
Objective: compute πt(·) := P(Xt ∈ ·|Z[0,t])
Idea:
construct a controlled stochastic process
dXt = (dynamics) + ut(Xt)dt+ Kt(Xt)dZt︸ ︷︷ ︸control
design the control law such that
Xt ∼ πt, ∀t ≥ 0 (exactness)
realize Xt with N particles {X1t , . . . , X
Nt }
Amirhossein Taghvaei 12 / 31 Amirhossein Taghvaei
![Page 38: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/38.jpg)
Mean-field design
State process: dXt = (dynamic model) X0 ∼ π0(·)
Observation process: dZt = h(Xt)dt+ dWt
Objective: compute πt(·) := P(Xt ∈ ·|Z[0,t])
Idea:
construct a controlled stochastic process
dXt = (dynamics) + ut(Xt)dt+ Kt(Xt)dZt︸ ︷︷ ︸control
design the control law such that
Xt ∼ πt, ∀t ≥ 0 (exactness)
realize Xt with N particles {X1t , . . . , X
Nt }
Amirhossein Taghvaei 12 / 31 Amirhossein Taghvaei
![Page 39: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/39.jpg)
Mean-field design
State process: dXt = (dynamic model) X0 ∼ π0(·)
Observation process: dZt = h(Xt)dt+ dWt
Objective: compute πt(·) := P(Xt ∈ ·|Z[0,t])
Idea:
construct a controlled stochastic process
dXt = (dynamics) + ut(Xt)dt+ Kt(Xt)dZt︸ ︷︷ ︸control
design the control law such that
Xt ∼ πt, ∀t ≥ 0 (exactness)
realize Xt with N particles {X1t , . . . , X
Nt }
Amirhossein Taghvaei 12 / 31 Amirhossein Taghvaei
![Page 40: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/40.jpg)
Mean-field design
State process: dXt = (dynamic model) X0 ∼ π0(·)
Observation process: dZt = h(Xt)dt+ dWt
Objective: compute πt(·) := P(Xt ∈ ·|Z[0,t])
Idea:
construct a controlled stochastic process
dXt = (dynamics) + ut(Xt)dt+ Kt(Xt)dZt︸ ︷︷ ︸control
design the control law such that
Xt ∼ πt, ∀t ≥ 0 (exactness)
realize Xt with N particles {X1t , . . . , X
Nt }
Amirhossein Taghvaei 12 / 31 Amirhossein Taghvaei
![Page 41: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/41.jpg)
Example
Objective: construct Xt ∼ πt = N(0, 1 + t)
Two solutions:
As N →∞, they have the same marginal distribution πt
However, different couplings between two time instants
Amirhossein Taghvaei 13 / 31 Amirhossein Taghvaei
![Page 42: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/42.jpg)
Example
Objective: construct Xt ∼ πt = N(0, 1 + t)
Two solutions:
(I) dXt = dBt, X0 ∼ N(0, 1) (II)d
dtXt =
Xt
2Σt, X0 ∼ N(0, 1)
As N →∞, they have the same marginal distribution πt
However, different couplings between two time instants
Amirhossein Taghvaei 13 / 31 Amirhossein Taghvaei
![Page 43: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/43.jpg)
Example
Objective: construct Xt ∼ πt = N(0, 1 + t)
Two solutions:
(I) dXit = dBit (II)
d
dtXit =
Xit
2N
∑Nj=1(Xj
t )2
As N →∞, they have the same marginal distribution πt
However, different couplings between two time instants
Amirhossein Taghvaei 13 / 31 Amirhossein Taghvaei
![Page 44: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/44.jpg)
Optimal transport construction
optimal transport coupling
mincoupling
E[|X0 −X1|2
]s.t. X0 ∼ π0, X1 ∼ π1
Brenier’s theorem: optimal coupling is deterministic and of gradient form
X1 = ∇Φ(X0)
adapt the procedure to continuous-time filtering setup → feedback particle filter
C. Villani, Topics in optimal transportation, American Mathematical Soc., 2003
Amirhossein Taghvaei 14 / 31 Amirhossein Taghvaei
![Page 45: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/45.jpg)
Optimal transport construction
optimal transport coupling
mincoupling
E[|X0 −X1|2
]s.t. X0 ∼ π0, X1 ∼ π1
Brenier’s theorem: optimal coupling is deterministic and of gradient form
X1 = ∇Φ(X0)
adapt the procedure to continuous-time filtering setup → feedback particle filter
C. Villani, Topics in optimal transportation, American Mathematical Soc., 2003
Amirhossein Taghvaei 14 / 31 Amirhossein Taghvaei
![Page 46: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/46.jpg)
Optimal transport construction
optimal transport coupling
mincoupling
E[|X0 −X1|2
]s.t. X0 ∼ π0, X1 ∼ π1
Brenier’s theorem: optimal coupling is deterministic and of gradient form
X1 = ∇Φ(X0)
adapt the procedure to continuous-time filtering setup → feedback particle filter
C. Villani, Topics in optimal transportation, American Mathematical Soc., 2003
Amirhossein Taghvaei 14 / 31 Amirhossein Taghvaei
![Page 47: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/47.jpg)
Feedback particle filter (FPF)
State process: dXt = (dynamic model) X0 ∼ π0(·)
Observation process: dZt = h(Xt)dt+ dWt
Objective: construct Xt ∼ πt(·) := P(Xt ∈ ·|Z[0,t])
Mean-field process:
dXt = (dynamic model) + Kt(Xt) ◦ (dZt −h(Xt) + ht
2dt)︸ ︷︷ ︸
correction
, X0i.i.d∼ π0
Gain function Kt(x) = ∇φt(x) where φt solves the Poisson eq.
1
ρt(x)∇ · (ρt(x)∇φt(x)) = h(x)− ht
ht = E[h(Xt)|Zt]
ρt is probability density for Xt
Amirhossein Taghvaei 15 / 31 Amirhossein Taghvaei
![Page 48: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/48.jpg)
Feedback particle filter (FPF)
State process: dXt = (dynamic model) X0 ∼ π0(·)
Observation process: dZt = h(Xt)dt+ dWt
Objective: construct Xt ∼ πt(·) := P(Xt ∈ ·|Z[0,t])
Mean-field process:
dXt = (dynamic model) + Kt(Xt) ◦ (dZt −h(Xt) + ht
2dt)︸ ︷︷ ︸
correction
, X0i.i.d∼ π0
Gain function Kt(x) = ∇φt(x) where φt solves the Poisson eq.
1
ρt(x)∇ · (ρt(x)∇φt(x)) = h(x)− ht
ht = E[h(Xt)|Zt]
ρt is probability density for Xt
Amirhossein Taghvaei 15 / 31 Amirhossein Taghvaei
![Page 49: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/49.jpg)
Feedback particle filter (FPF)
State process: dXt = (dynamic model) X0 ∼ π0(·)
Observation process: dZt = h(Xt)dt+ dWt
Objective: construct Xt ∼ πt(·) := P(Xt ∈ ·|Z[0,t])
Mean-field process:
dXt = (dynamic model) + Kt(Xt) ◦ (dZt −h(Xt) + ht
2dt)︸ ︷︷ ︸
correction
, X0i.i.d∼ π0
Gain function Kt(x) = ∇φt(x) where φt solves the Poisson eq.
1
ρt(x)∇ · (ρt(x)∇φt(x)) = h(x)− ht
ht = E[h(Xt)|Zt]
ρt is probability density for Xt
Amirhossein Taghvaei 15 / 31 Amirhossein Taghvaei
![Page 50: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/50.jpg)
Feedback particle filter (FPF)
State process: dXt = (dynamic model) X0 ∼ π0(·)
Observation process: dZt = h(Xt)dt+ dWt
Objective: construct Xt ∼ πt(·) := P(Xt ∈ ·|Z[0,t])
Mean-field process:
dXt = (dynamic model) + Kt(Xt) ◦ (dZt −h(Xt) + ht
2dt)︸ ︷︷ ︸
correction
, X0i.i.d∼ π0
Gain function Kt(x) = ∇φt(x) where φt solves the Poisson eq.
1
ρt(x)∇ · (ρt(x)∇φt(x)) = h(x)− ht
ht = E[h(Xt)|Zt]
ρt is probability density for Xt
Amirhossein Taghvaei 15 / 31 Amirhossein Taghvaei
![Page 51: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/51.jpg)
Finite-N approximation
Update formula:
dXit = (dynamic model) + K
(N)t (Xi
t) ◦ (dZt −h(Xi
t) + h(N)t
2dt)︸ ︷︷ ︸
correction
, X0i.i.d∼ π0
Gain function K(N)t (x) is approximated in terms of particles
K(N)t = Algorithm({X1
t , . . . , XNt }, h)
h(N)t =
1
N
N∑i=1
h(Xit)
Main challenge: Gain function approximation
Amirhossein Taghvaei 16 / 31 Amirhossein Taghvaei
![Page 52: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/52.jpg)
Finite-N approximation
Update formula:
dXit = (dynamic model) + K
(N)t (Xi
t) ◦ (dZt −h(Xi
t) + h(N)t
2dt)︸ ︷︷ ︸
correction
, X0i.i.d∼ π0
Gain function K(N)t (x) is approximated in terms of particles
K(N)t = Algorithm({X1
t , . . . , XNt }, h)
h(N)t =
1
N
N∑i=1
h(Xit)
Main challenge: Gain function approximation
Amirhossein Taghvaei 16 / 31 Amirhossein Taghvaei
![Page 53: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/53.jpg)
Finite-N approximation
Update formula:
dXit = (dynamic model) + K
(N)t (Xi
t) ◦ (dZt −h(Xi
t) + h(N)t
2dt)︸ ︷︷ ︸
correction
, X0i.i.d∼ π0
Gain function K(N)t (x) is approximated in terms of particles
K(N)t = Algorithm({X1
t , . . . , XNt }, h)
h(N)t =
1
N
N∑i=1
h(Xit)
Main challenge: Gain function approximation
Amirhossein Taghvaei 16 / 31 Amirhossein Taghvaei
![Page 54: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/54.jpg)
Part II: summary
feedback particle filter
prior posterior
Poisson equation
KF: dmt = (dynamical model) + Kt(dZt −Hmtdt)
EnKF: dXit = (dynamical model) + K
(N)t (dZt −
HXit +Hm
(N)t
2dt)
FPF: dXit = (dynamical model) + K
(N)t (Xi
t) ◦ (dZt −h(Xi
t) + h(N)t
2dt)
Difficulty of filtering = gain function approximating
Amirhossein Taghvaei 17 / 31 Amirhossein Taghvaei
![Page 55: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/55.jpg)
Outline
Part I: background and motivation
Part II: mean-field control design
Part III: gain function approximation
Part III: applications
References:
A. Taghvaei, P. G. Mehta, S. P. Meyn, Diffusion map-based algorithm for gain functionapproximation in the feedback particle filter, SIAM Journal on Uncertainty Quantification,8(3):1090–1117, 2020
Amirhossein Taghvaei 17 / 31 Amirhossein Taghvaei
![Page 56: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/56.jpg)
Outline
Part I: background and motivation
Part II: mean-field control design
Part III: gain function approximation
Part III: applications
References:
A. Taghvaei, P. G. Mehta, S. P. Meyn, Diffusion map-based algorithm for gain functionapproximation in the feedback particle filter, SIAM Journal on Uncertainty Quantification,8(3):1090–1117, 2020
Amirhossein Taghvaei 17 / 31 Amirhossein Taghvaei
![Page 57: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/57.jpg)
Gain function approximation (K(x) = ∇φ(x))Problem formulation
Poisson equation:−∆ρφ(x) = h(x)− h
where ∆ρφ :=1
ρ∇ · (ρ∇φ)
Computational problem:
Given: {X1, . . . , XN} i.i.d∼ ρ
Approximate: {∇φ(X1), . . . ,∇φ(XN )}
Amirhossein Taghvaei 18 / 31 Amirhossein Taghvaei
![Page 58: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/58.jpg)
Gain function approximation (K(x) = ∇φ(x))Problem formulation
Poisson equation:−∆ρφ(x) = h(x)− h
where ∆ρφ := ∆φ+∇ log(ρ) · ∇φ
Computational problem:
Given: {X1, . . . , XN} i.i.d∼ ρ
Approximate: {∇φ(X1), . . . ,∇φ(XN )}
Amirhossein Taghvaei 18 / 31 Amirhossein Taghvaei
![Page 59: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/59.jpg)
Gain function approximation (K(x) = ∇φ(x))Problem formulation
Poisson equation:−∆ρφ(x) = h(x)− h
where ∆ρφ :=1
ρ∇ · (ρ∇φ)
Computational problem:
Given: {X1, . . . , XN} i.i.d∼ ρ
Approximate: {∇φ(X1), . . . ,∇φ(XN )}
Amirhossein Taghvaei 18 / 31 Amirhossein Taghvaei
![Page 60: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/60.jpg)
Gain function approximationConstant gain approximation
Kconst := arg minK∈Rd
∫|∇φ(x)− K|2ρ(x)dx
A closed-form formula:
Kconst =
∫(h(x)− h)xρ(x)dx ≈ 1
N
∑i=1
(h(Xi)− h(N))Xi
FPFconst. gain approx.−−−−−−−−−−−→ Ensemble Kalman filter (EnKF)
Can we improve this approximation?
A. Taghvaei, J de Wiljes, P. G. Mehta, and S. Reich. Kalman filter and its modern extensions... ASME, 2017
Amirhossein Taghvaei 19 / 31 Amirhossein Taghvaei
![Page 61: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/61.jpg)
Gain function approximationConstant gain approximation
Kconst := arg minK∈Rd
∫|∇φ(x)− K|2ρ(x)dx
A closed-form formula:
Kconst =
∫(h(x)− h)xρ(x)dx ≈ 1
N
∑i=1
(h(Xi)− h(N))Xi
FPFconst. gain approx.−−−−−−−−−−−→ Ensemble Kalman filter (EnKF)
Can we improve this approximation?
A. Taghvaei, J de Wiljes, P. G. Mehta, and S. Reich. Kalman filter and its modern extensions... ASME, 2017
Amirhossein Taghvaei 19 / 31 Amirhossein Taghvaei
![Page 62: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/62.jpg)
Gain function approximationConstant gain approximation
Kconst := arg minK∈Rd
∫|∇φ(x)− K|2ρ(x)dx
A closed-form formula:
Kconst =
∫(h(x)− h)xρ(x)dx ≈ 1
N
∑i=1
(h(Xi)− h(N))Xi
FPFconst. gain approx.−−−−−−−−−−−→ Ensemble Kalman filter (EnKF)
Can we improve this approximation?
A. Taghvaei, J de Wiljes, P. G. Mehta, and S. Reich. Kalman filter and its modern extensions... ASME, 2017
Amirhossein Taghvaei 19 / 31 Amirhossein Taghvaei
![Page 63: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/63.jpg)
Gain function approximationThree formulations, Three algorithms
−∆ρφ = h− h
(I) Weak formulation: (Galerkin algorithm)⟨∇φ,∇ψ
⟩=⟨h− h, ψ
⟩, ∀ψ ∈ H1(ρ)
where⟨f, g⟩
:=
∫f(x)g(x)ρ(x)dx
(II) Semigroup formulation: (Diffusion-map algorithm)
φ = Ptφ+
∫ t
0
Ps(h− h)ds
where Pt := et∆ρ is the semigroup
(III) Variational formulation: (Neural net. approximation)
minφ∈H1
0 (ρ)
∫ [1
2|∇φ(x)|2 − φ(x)(h(x)− h)
]ρ(x)dx
Amirhossein Taghvaei 20 / 31 Amirhossein Taghvaei
![Page 64: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/64.jpg)
Gain function approximationThree formulations, Three algorithms
−∆ρφ = h− h
(I) Weak formulation: (Galerkin algorithm)⟨∇φ,∇ψ
⟩=⟨h− h, ψ
⟩, ∀ψ ∈ H1(ρ)
where⟨f, g⟩
:=
∫f(x)g(x)ρ(x)dx
(II) Semigroup formulation: (Diffusion-map algorithm)
φ = Ptφ+
∫ t
0
Ps(h− h)ds
where Pt := et∆ρ is the semigroup
(III) Variational formulation: (Neural net. approximation)
minφ∈H1
0 (ρ)
∫ [1
2|∇φ(x)|2 − φ(x)(h(x)− h)
]ρ(x)dx
Amirhossein Taghvaei 20 / 31 Amirhossein Taghvaei
![Page 65: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/65.jpg)
Gain function approximationThree formulations, Three algorithms
−∆ρφ = h− h
(I) Weak formulation: (Galerkin algorithm)⟨∇φ,∇ψ
⟩=⟨h− h, ψ
⟩, ∀ψ ∈ H1(ρ)
where⟨f, g⟩
:=
∫f(x)g(x)ρ(x)dx
(II) Semigroup formulation: (Diffusion-map algorithm)
φ = Ptφ+
∫ t
0
Ps(h− h)ds
where Pt := et∆ρ is the semigroup
(III) Variational formulation: (Neural net. approximation)
minφ∈H1
0 (ρ)
∫ [1
2|∇φ(x)|2 − φ(x)(h(x)− h)
]ρ(x)dx
Amirhossein Taghvaei 20 / 31 Amirhossein Taghvaei
![Page 66: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/66.jpg)
Gain function approximationThree formulations, Three algorithms
−∆ρφ = h− h
(I) Weak formulation: (Galerkin algorithm)⟨∇φ,∇ψ
⟩=⟨h− h, ψ
⟩, ∀ψ ∈ H1(ρ)
where⟨f, g⟩
:=
∫f(x)g(x)ρ(x)dx
(II) Semigroup formulation: (Diffusion-map algorithm)
φ = Ptφ+
∫ t
0
Ps(h− h)ds
where Pt := et∆ρ is the semigroup
(III) Variational formulation: (Neural net. approximation)
minφ∈H1
0 (ρ)
∫ [1
2|∇φ(x)|2 − φ(x)(h(x)− h)
]ρ(x)dx
Amirhossein Taghvaei 20 / 31 Amirhossein Taghvaei
![Page 67: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/67.jpg)
Gain function approximationThree formulations, Three algorithms
−∆ρφ = h− h
(I) Weak formulation: (Galerkin algorithm)⟨∇φ,∇ψ
⟩=⟨h− h, ψ
⟩, ∀ψ ∈ H1(ρ)
where⟨f, g⟩
:=
∫f(x)g(x)ρ(x)dx
(II) Semigroup formulation: (Diffusion-map algorithm)
φ = Ptφ+
∫ t
0
Ps(h− h)ds
where Pt := et∆ρ is the semigroup
(III) Variational formulation: (Neural net. approximation)
minφ∈H1
0 (ρ)
∫ [1
2|∇φ(x)|2 − φ(x)(h(x)− h)
]ρ(x)dx
Amirhossein Taghvaei 20 / 31 Amirhossein Taghvaei
![Page 68: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/68.jpg)
Gain function approximationThree formulations, Three algorithms
−∆ρφ = h− h
(I) Weak formulation: (Galerkin algorithm)⟨∇φ,∇ψ
⟩=⟨h− h, ψ
⟩, ∀ψ ∈ H1(ρ)
where⟨f, g⟩
:=
∫f(x)g(x)ρ(x)dx
(II) Semigroup formulation: (Diffusion-map algorithm)
φ = Ptφ+
∫ t
0
Ps(h− h)ds
where Pt := et∆ρ is the semigroup
(III) Variational formulation: (Neural net. approximation)
minφ∈H1
0 (ρ)
∫ [1
2|∇φ(x)|2 − φ(x)(h(x)− h)
]ρ(x)dx
Amirhossein Taghvaei 20 / 31 Amirhossein Taghvaei
![Page 69: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/69.jpg)
Gain function approximationThree formulations, Three algorithms
−∆ρφ = h− h
(I) Weak formulation: (Galerkin algorithm)⟨∇φ,∇ψ
⟩=⟨h− h, ψ
⟩, ∀ψ ∈ H1(ρ)
where⟨f, g⟩
:=
∫f(x)g(x)ρ(x)dx
(II) Semigroup formulation: (Diffusion-map algorithm)
φ = Ptφ+
∫ t
0
Ps(h− h)ds
where Pt := et∆ρ is the semigroup
(III) Variational formulation: (Neural net. approximation)
minφ∈H1
0 (ρ)
∫ [1
2|∇φ(x)|2 − φ(x)(h(x)− h)
]ρ(x)dx
Amirhossein Taghvaei 20 / 31 Amirhossein Taghvaei
![Page 70: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/70.jpg)
Diffusion map-based algorithmOverview
(I) Semigroup formulation:
φ = Pεφ+
∫ ε
0
Ps(h− h)ds
where Pε := eε∆ρ is the semigroup
(II) Diffusion map approximation: (Coifman, et. al. 2006)
Pεf(x)ε↓0≈ Tεf(x) :=
1
nε(x)
∫gε(x, y)
f(y)ρ(y)√(gε ∗ ρ)(y)
dy
(III) Empirical approximation :
Tεf(x)N↑∞≈ T (N)
ε f(x) :=1
n(N)ε (x)
N∑i=1
gε(x,Xi)
f(Xi)√∑Nj=1 gε(X
i, Xj)
A. Taghvaei, P. G. Mehta, S. P. Meyn Diffusion map-based algorithm for gain function approximation in the feedback particle filter, SIAM UQ 2020
Amirhossein Taghvaei 21 / 31 Amirhossein Taghvaei
![Page 71: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/71.jpg)
Diffusion map-based algorithmOverview
(I) Semigroup formulation:
φ = Pεφ+
∫ ε
0
Ps(h− h)ds
where Pε := eε∆ρ is the semigroup
(II) Diffusion map approximation: (Coifman, et. al. 2006)
Pεf(x)ε↓0≈ Tεf(x) :=
1
nε(x)
∫gε(x, y)
f(y)ρ(y)√(gε ∗ ρ)(y)
dy
(III) Empirical approximation :
Tεf(x)N↑∞≈ T (N)
ε f(x) :=1
n(N)ε (x)
N∑i=1
gε(x,Xi)
f(Xi)√∑Nj=1 gε(X
i, Xj)
A. Taghvaei, P. G. Mehta, S. P. Meyn Diffusion map-based algorithm for gain function approximation in the feedback particle filter, SIAM UQ 2020
Amirhossein Taghvaei 21 / 31 Amirhossein Taghvaei
![Page 72: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/72.jpg)
Diffusion map-based algorithmOverview
(I) Semigroup formulation:
φ = Pεφ+
∫ ε
0
Ps(h− h)ds
where Pε := eε∆ρ is the semigroup
(II) Diffusion map approximation: (Coifman, et. al. 2006)
Pεf(x)ε↓0≈ Tεf(x) :=
1
nε(x)
∫gε(x, y)
f(y)ρ(y)√(gε ∗ ρ)(y)
dy
(III) Empirical approximation :
Tεf(x)N↑∞≈ T (N)
ε f(x) :=1
n(N)ε (x)
N∑i=1
gε(x,Xi)
f(Xi)√∑Nj=1 gε(X
i, Xj)
A. Taghvaei, P. G. Mehta, S. P. Meyn Diffusion map-based algorithm for gain function approximation in the feedback particle filter, SIAM UQ 2020
Amirhossein Taghvaei 21 / 31 Amirhossein Taghvaei
![Page 73: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/73.jpg)
Diffusion map-based algorithmNumerical analysis
2 1 0 1 2
x
0
1
2
3
4
5
6
7
K
const. gain
exact= 102
Objective: error analysis of bias and variance
Amirhossein Taghvaei 22 / 31 Amirhossein Taghvaei
![Page 74: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/74.jpg)
Diffusion map-based algorithmNumerical analysis
2 1 0 1 2
x
0
1
2
3
4
5
6
7
K
const. gain
exact= 101
Objective: error analysis of bias and variance
Amirhossein Taghvaei 22 / 31 Amirhossein Taghvaei
![Page 75: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/75.jpg)
Diffusion map-based algorithmNumerical analysis
2 1 0 1 2
x
0
1
2
3
4
5
6
7
K
const. gain
exact= 100
Objective: error analysis of bias and variance
Amirhossein Taghvaei 22 / 31 Amirhossein Taghvaei
![Page 76: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/76.jpg)
Diffusion map-based algorithmNumerical analysis
2 1 0 1 2
x
0
1
2
3
4
5
6
7
K
const. gain
exact= 10 1
Objective: error analysis of bias and variance
Amirhossein Taghvaei 22 / 31 Amirhossein Taghvaei
![Page 77: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/77.jpg)
Diffusion map-based algorithmNumerical analysis
2 1 0 1 2
x
0
1
2
3
4
5
6
7
K
const. gain
exact= 10 2
Objective: error analysis of bias and variance
Amirhossein Taghvaei 22 / 31 Amirhossein Taghvaei
![Page 78: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/78.jpg)
Diffusion map-based algorithmNumerical analysis
2 1 0 1 2
x
0
2
4
6
8
10
K
const. gain
exact= 10 3
Objective: error analysis of bias and variance
Amirhossein Taghvaei 22 / 31 Amirhossein Taghvaei
![Page 79: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/79.jpg)
Diffusion map-based algorithmNumerical analysis
2 1 0 1 2
x
0
2
4
6
8
10
K
const. gain
exact= 10 3
variance dominates
bias dominates
diffusion mapconstant gain
Objective: error analysis of bias and variance
Amirhossein Taghvaei 22 / 31 Amirhossein Taghvaei
![Page 80: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/80.jpg)
Diffusion map-based algorithmNumerical analysis
2 1 0 1 2
x
0
2
4
6
8
10
K
const. gain
exact= 10 3
variance dominates
bias dominates
diffusion mapconstant gain
Objective: error analysis of bias and variance
Amirhossein Taghvaei 22 / 31 Amirhossein Taghvaei
![Page 81: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/81.jpg)
Diffusion map-based algorithmError analysis
Approximation steps:
φDM approx.
−−−−−→bias
φεempirical approx.
−−−−−→variance
φ(N)ε
Proposition
Under technical assumptions, with high probability
‖φ(N)ε − φ‖L2(ρ) ≤ O(ε)︸︷︷︸
bias
+O(1
ε1+ d2N
12
)︸ ︷︷ ︸variance
Tools for proof:
1 stochastic stability theory (Meyn & Tweedie, 2012)
2 numerical analysis of integral equations (Anselone, 1971, Atkinson, 1976)
A. Taghvaei, P. G. Mehta, S. P. Meyn Diffusion map-based algorithm for gain function approximation in the feedback particle filter, SIAM UQ 2020
Amirhossein Taghvaei 23 / 31 Amirhossein Taghvaei
![Page 82: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/82.jpg)
Diffusion map-based algorithmError analysis
Approximation steps:
φDM approx.
−−−−−→bias
φεempirical approx.
−−−−−→variance
φ(N)ε
Proposition
Under technical assumptions, with high probability
‖φ(N)ε − φ‖L2(ρ) ≤ O(ε)︸︷︷︸
bias
+O(1
ε1+ d2N
12
)︸ ︷︷ ︸variance
Tools for proof:
1 stochastic stability theory (Meyn & Tweedie, 2012)
2 numerical analysis of integral equations (Anselone, 1971, Atkinson, 1976)
A. Taghvaei, P. G. Mehta, S. P. Meyn Diffusion map-based algorithm for gain function approximation in the feedback particle filter, SIAM UQ 2020
Amirhossein Taghvaei 23 / 31 Amirhossein Taghvaei
![Page 83: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/83.jpg)
Diffusion map-based algorithmError analysis
Approximation steps:
φDM approx.
−−−−−→bias
φεempirical approx.
−−−−−→variance
φ(N)ε
Proposition
Under technical assumptions, with high probability
‖φ(N)ε − φ‖L2(ρ) ≤ O(ε)︸︷︷︸
bias
+O(1
ε1+ d2N
12
)︸ ︷︷ ︸variance
Tools for proof:
1 stochastic stability theory (Meyn & Tweedie, 2012)
2 numerical analysis of integral equations (Anselone, 1971, Atkinson, 1976)
A. Taghvaei, P. G. Mehta, S. P. Meyn Diffusion map-based algorithm for gain function approximation in the feedback particle filter, SIAM UQ 2020
Amirhossein Taghvaei 23 / 31 Amirhossein Taghvaei
![Page 84: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/84.jpg)
Diffusion map-based algorithmNumerical analysis
Setup: ρ(x) = ρbimodal(x1)
d∏n=2
ρGaussian(xn) and h(x) = x1.
Convergence to const. gain approx.:
limε→∞
K(N)ε = const. gain approximatoin
Amirhossein Taghvaei 24 / 31 Amirhossein Taghvaei
![Page 85: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/85.jpg)
Diffusion map-based algorithmNumerical analysis
Setup: ρ(x) = ρbimodal(x1)
d∏n=2
ρGaussian(xn) and h(x) = x1.
const. gain
m.s
.e
O( )
m.s
.eConvergence to const. gain approx.:
limε→∞
K(N)ε = const. gain approximatoin
Amirhossein Taghvaei 24 / 31 Amirhossein Taghvaei
![Page 86: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/86.jpg)
Diffusion map-based algorithmNumerical analysis
Setup: ρ(x) = ρbimodal(x1)
d∏n=2
ρGaussian(xn) and h(x) = x1.
const. gain
m.s
.e
O( )
m.s
.eConvergence to const. gain approx.:
limε→∞
K(N)ε = const. gain approximatoin
Amirhossein Taghvaei 24 / 31 Amirhossein Taghvaei
![Page 87: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/87.jpg)
Outline
Part I: background and motivation
Part II: mean-field control design
Part III: gain function approximation
Part III: applications
References:
C. Zhang, A. Taghvaei, P. G. Mehta, Feedback Particle Filter on Riemannian Manifolds andMatrix Lie groups, IEEE Transactions on Automatic Control (TAC), 63(8):2465–2480, 2017.
C. Zhang, A. Taghvaei, P. G. Mehta, Attitude Estimation of a Wearable Motion SensorIEEE American Control Conference (ACC), 2017
A. Taghvaei, S. A. Hutchinson, P. G. Mehta, A Coupled Oscillator-based ControlArchitecture for Locomotory Gaits, IEEE Conference on Decision and Control (CDC), 2014
Amirhossein Taghvaei 24 / 31 Amirhossein Taghvaei
![Page 88: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/88.jpg)
Outline
Part I: background and motivation
Part II: mean-field control design
Part III: gain function approximation
Part III: applications
References:
C. Zhang, A. Taghvaei, P. G. Mehta, Feedback Particle Filter on Riemannian Manifolds andMatrix Lie groups, IEEE Transactions on Automatic Control (TAC), 63(8):2465–2480, 2017.
C. Zhang, A. Taghvaei, P. G. Mehta, Attitude Estimation of a Wearable Motion SensorIEEE American Control Conference (ACC), 2017
A. Taghvaei, S. A. Hutchinson, P. G. Mehta, A Coupled Oscillator-based ControlArchitecture for Locomotory Gaits, IEEE Conference on Decision and Control (CDC), 2014
Amirhossein Taghvaei 24 / 31 Amirhossein Taghvaei
![Page 89: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/89.jpg)
Attitude estimation problemProblem formulation
Model:
State dRt = Rt[ωt ]×dt+Rt ◦ [σBdBt ]× R0 ∼ π0
Observation dZt = h(Rt)dt+ σWdWt
FPF:
dRit = (dynamaics) +Rit [ Kt(Rit) ◦ (dZt −
1
2(h(Rit) + ht) dt) ]×︸ ︷︷ ︸
correction
gain function Kt(R) = grad(φ)(R)
φ satisfies the Poisson equation on SO(3)
diffusion map algorithm is extended to SO(3)
C. Zhang, A. Taghvaei, P. G. Mehta Feedback Particle Filter on Riemannian Manifolds and Matrix Lie groups, IEEE TAC, 2017
Amirhossein Taghvaei 25 / 31 Amirhossein Taghvaei
![Page 90: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/90.jpg)
Attitude estimation problemProblem formulation
Model:
State dRt = Rt[ωt ]×dt+Rt ◦ [σBdBt ]× R0 ∼ π0
Observation dZt = h(Rt)dt+ σWdWt
Notation Rt ∈ SO(3), Zt ∈ Rm
FPF:
dRit = (dynamaics) +Rit [ Kt(Rit) ◦ (dZt −
1
2(h(Rit) + ht) dt) ]×︸ ︷︷ ︸
correction
gain function Kt(R) = grad(φ)(R)
φ satisfies the Poisson equation on SO(3)
diffusion map algorithm is extended to SO(3)
C. Zhang, A. Taghvaei, P. G. Mehta Feedback Particle Filter on Riemannian Manifolds and Matrix Lie groups, IEEE TAC, 2017
Amirhossein Taghvaei 25 / 31 Amirhossein Taghvaei
![Page 91: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/91.jpg)
Attitude estimation problemProblem formulation
Model:
State dRt = Rt[ωt ]×dt+Rt ◦ [σBdBt ]× R0 ∼ π0
Observation dZt = h(Rt)dt+ σWdWt
Notation Rt ∈ SO(3), Zt ∈ Rm
ωt ∈ R3 : angular velocity in body frame
FPF:
dRit = (dynamaics) +Rit [ Kt(Rit) ◦ (dZt −
1
2(h(Rit) + ht) dt) ]×︸ ︷︷ ︸
correction
gain function Kt(R) = grad(φ)(R)
φ satisfies the Poisson equation on SO(3)
diffusion map algorithm is extended to SO(3)
C. Zhang, A. Taghvaei, P. G. Mehta Feedback Particle Filter on Riemannian Manifolds and Matrix Lie groups, IEEE TAC, 2017
Amirhossein Taghvaei 25 / 31 Amirhossein Taghvaei
![Page 92: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/92.jpg)
Attitude estimation problemProblem formulation
Model:
State dRt = Rt[ωt ]×dt+Rt ◦ [σBdBt ]× R0 ∼ π0
Observation dZt = h(Rt)dt+ σWdWt
Notation Rt ∈ SO(3), Zt ∈ Rm
ωt ∈ R3 : angular velocity in body frame
Applications:
aircraft navigation ( Crassidis et. al 03’ [JGCD], Hua et. al 14’ [IEEE TCST] )
robot localization ( Hesch et. al 13’ [IJRR], Kelly et. al 11’ [IJRR] )
visual tracking ( Choi et. al 11’ [ICRA], Kwon et. al 13’ [IEEE TPAMI] )
FPF:
dRit = (dynamaics) +Rit [ Kt(Rit) ◦ (dZt −
1
2(h(Rit) + ht) dt) ]×︸ ︷︷ ︸
correction
gain function Kt(R) = grad(φ)(R)
φ satisfies the Poisson equation on SO(3)
diffusion map algorithm is extended to SO(3)
C. Zhang, A. Taghvaei, P. G. Mehta Feedback Particle Filter on Riemannian Manifolds and Matrix Lie groups, IEEE TAC, 2017
Amirhossein Taghvaei 25 / 31 Amirhossein Taghvaei
![Page 93: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/93.jpg)
Attitude estimation problemProblem formulation
Model:
State dRt = Rt[ωt ]×dt+Rt ◦ [σBdBt ]× R0 ∼ π0
Observation dZt = h(Rt)dt+ σWdWt
FPF:
dRit = (dynamaics) +Rit [ Kt(Rit) ◦ (dZt −
1
2(h(Rit) + ht) dt) ]×︸ ︷︷ ︸
correction
gain function Kt(R) = grad(φ)(R)
φ satisfies the Poisson equation on SO(3)
diffusion map algorithm is extended to SO(3)
C. Zhang, A. Taghvaei, P. G. Mehta Feedback Particle Filter on Riemannian Manifolds and Matrix Lie groups, IEEE TAC, 2017
Amirhossein Taghvaei 25 / 31 Amirhossein Taghvaei
![Page 94: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/94.jpg)
Attitude estimation problemNumerical experiment
Comparison with:
MEKF: Multiplicative EKF (Markley 03’)
USQUE: Unscented Quaternion Estimator (Crassidis et. al 03’)
LIEKF: Left invariant EKF (Bonnabel et. al 09’)
Performance metric: RMSE of rotation angle RMSEt =
√1
100
∑100
j=1(δθjt )
2
Amirhossein Taghvaei 26 / 31 Amirhossein Taghvaei
![Page 95: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/95.jpg)
Arm motion tracking
State: (Rt, ϕt) ∈ SO(3)× SO(2)
Observation: dZt = RTt h(ϕt) + σWdWt
C. Zhang, A. Taghvaei, P. G. Mehta, Attitude Estimation of a Wearable Motion Sensor, ACC 2017
Amirhossein Taghvaei 27 / 31 Amirhossein Taghvaei
![Page 96: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/96.jpg)
Arm motion tracking
State: (Rt, ϕt) ∈ SO(3)× SO(2)
Observation: dZt = RTt h(ϕt) + σWdWt
C. Zhang, A. Taghvaei, P. G. Mehta, Attitude Estimation of a Wearable Motion Sensor, ACC 2017
Amirhossein Taghvaei 27 / 31 Amirhossein Taghvaei
![Page 97: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/97.jpg)
Arm motion tracking
State: (Rt, ϕt) ∈ SO(3)× SO(2)
Observation: dZt = RTt h(ϕt) + σWdWt
C. Zhang, A. Taghvaei, P. G. Mehta, Attitude Estimation of a Wearable Motion Sensor, ACC 2017
Amirhossein Taghvaei 27 / 31 Amirhossein Taghvaei
![Page 98: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/98.jpg)
Arm motion tracking
State: (Rt, ϕt) ∈ SO(3)× SO(2)
Observation: dZt = RTt h(ϕt) + σWdWt
C. Zhang, A. Taghvaei, P. G. Mehta, Attitude Estimation of a Wearable Motion Sensor, ACC 2017
Amirhossein Taghvaei 27 / 31 Amirhossein Taghvaei
![Page 99: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/99.jpg)
Optimal control of locomotion gaits2-body System
[Click to play the movie]
A. Taghvaei, S. Hutchinson and P. G. Mehta. A coupled-oscillators-based control architecture for locomotory gaits. IEEE CDC (2014).
Amirhossein Taghvaei 28 / 31 Amirhossein Taghvaei
![Page 100: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/100.jpg)
Summary
Controlled interacting particle system:
feedback particle filter
prior posterior
Poisson equation
3 2 1 0 1 2 3x
0
2
4
6
8
10
K
const. gain
exact= 0.02= 0.10= 0.50= 2.00
mean-field control design gain function approximation
Question: can the design and approximation be addressed in a single framework?
some directions for future research
Amirhossein Taghvaei 29 / 31 Amirhossein Taghvaei
![Page 101: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/101.jpg)
Summary
Controlled interacting particle system:
feedback particle filter
prior posterior
Poisson equation
3 2 1 0 1 2 3x
0
2
4
6
8
10
K
const. gain
exact= 0.02= 0.10= 0.50= 2.00
mean-field control design gain function approximation
Question: can the design and approximation be addressed in a single framework?
some directions for future research
Amirhossein Taghvaei 29 / 31 Amirhossein Taghvaei
![Page 102: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/102.jpg)
Summary
Controlled interacting particle system:
feedback particle filter
prior posterior
Poisson equation
3 2 1 0 1 2 3x
0
2
4
6
8
10
K
const. gain
exact= 0.02= 0.10= 0.50= 2.00
mean-field control design gain function approximation
Question: can the design and approximation be addressed in a single framework?
some directions for future research
Amirhossein Taghvaei 29 / 31 Amirhossein Taghvaei
![Page 103: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/103.jpg)
Reinforcement learning with partial observation
Minimize Bellman error
sensory data
rewardcontrol input
estimate
TD-learning
coupled oscillator FPF
Idea: particles represent the belief state
Challenge: filtering is based on known dynamic and observation model
Amirhossein Taghvaei 30 / 31 Amirhossein Taghvaei
![Page 104: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/104.jpg)
Interplay between optimization and sampling
MCMC
dXit = −∇V (Xi
t)dt+√
2dBit
Controlled interacting particle system
d
dtXit = −∇V (Xi
t) +∇ log(pt(Xit))︸ ︷︷ ︸
interaction
Example: Stein variational gradient descent (Liu & Wang, 2016)
Questions:
Principled method to approximate the interaction term
What are the fundamental differences and trade-offs between two approaches?
A. Taghvaei, P. G. Mehta. Accelerated flow for probability distributions, ICML 2019
Amirhossein Taghvaei 31 / 31 Amirhossein Taghvaei
![Page 105: Controlled Interacting Particle Systems for Estimation and](https://reader031.vdocuments.site/reader031/viewer/2022022023/6211717a83fab67f157e14d5/html5/thumbnails/105.jpg)
The End
Optimal transporttheory
feedback particle filter
prior posterior
Thank you for your attention!
Questions?
Amirhossein Taghvaei 31 / 31 Amirhossein Taghvaei