goal-oriented optimal experimental design · 2016-11-18 · ahmed attia statistical and applied...

GOal-Oriented Optimal Experimental DesignGO-OED for PDE-based Bayesian Linear Inverse Problems

Ahmed Attia

Statistical and Applied Mathematical Science Institute (SAMSI)19 TW Alexander Dr, Durham, NC 27703

attia@ samsi.info || vt.edu

Alen Alexanderian, and Arvind Krishna Saibaba

Department of MathematicsNorth Carolina State University

SAMSINovember 16, 2016

GO-OED for PDE-based Bayesian Linear Inverse Problems [1/28]

November 16, 2016: Goal-Oriented Optimal Experimental Design, Ahmed Attia. (http://samsi.info)

Outline

Motivation

Inverse Problems and Data Assimilation

Optimal Experimental Design (OED)Optimal experimental design (OED)Sensor placement as OED problem

Goal-Oriented OEDGO-OED: problem formulationGO-OED: preliminary results

GO-OED for PDE-based Bayesian Linear Inverse Problems [2/28]


Motivation




GO-OED for PDE-based Bayesian Linear Inverse Problems Motivation [3/28]


Motivation

I How do we efficiently forecast the local/global state of the atmosphere?

I How is it feasible to predict a volcanic eruption, or simulate the inside of avolcano given external observations?

I Is it possible to infer the initial distribution of a contaminant givenobservations taken after a diffusive transport is applied?

I If we manage to make predictions, how certain are they?

I How are the measurement sensors efficiently distributed?

I What are the computational costs and challenges in these problems?

I Inverse Problems, Data Assimilation, andOptimal Design of Experiments

GO-OED for PDE-based Bayesian Linear Inverse Problems Motivation [4/28]


Motivation




GO-OED for PDE-based Bayesian Linear Inverse Problems Inverse Problems and Data Assimilation [5/28]


Inverse Problems and Data AssimilationI Inverse Problems: the process by which the unknown true state of (e.g. a

physical) system is inferred from measurements produced by this system.I Data Assimilation:

Model + Prior + Observations︸︷︷︸with associated uncertainties

→ Best description of the state︸︷︷︸Variational+Ensemble

I Applications include: atmospheric forecasting, power flow, oil reservoir,volcano simulation, etc.



Example: Initial Condition in Advection-Diffusive transport

I Consider an open and bounded domain Ω ∈ Rn. Here, we choose n = 2.

I Consider the concentration of a contaminant u in the domain Ω.

I Assume, we are interested in inferring the initial distribution of thecontaminant, from measurements b taken after the contaminant has beensubjected to diffusive transport. For example, consider measuring u on someparts Γm of (or all) the domain boundary.

I Consider the domain:




I Let u be the solution of:

ut − κ∆u + v · ∇u = 0 in Ω× [0,T ]

u(0, x) = u0 in Ω

κ∇u · n = 0 on ∂Ω× [0,T ]

where κ is the diffusivity, and v is the velocity field:




I The Goal: given measurements b over time interval [T1,T ], find the initialcondition u0 such that the evolution of the contaminant distribution isgoverned by the PDE, and is consistent with the collected measurements.

I A Continuous Formulation: solve the optimization problem:

minu0

J(u0) :=1

2

∫ T

T1

∫Γm

(B(u)− b)2dxdt +γ

2

∫Ω

u20dx

where u is constrained by the advection diffusion PDEs.

I A Discrete Formulation: solve the optimization problem:

minu0

J(u0) :=1

2

m∑i=1

(B(ui )− bi )TΓ−1

noise(B(ui )− bi ) +1

2uT

0 R−1u0

where u is a spatial discretization of u, and is constrained by the discretizedadvection diffusion PDEs, and bii=1,2,...,m are discrete-time observations.



Example: Initial Condition in Advection-Diffusive transportA Discrete Bayesian Formulation:

I Let u0 be a random vector with prior distribution N (0, Cprior).

I Assume a likelihood function:

P(b1, . . . ,bm|u0) ∝ exp

(−1

2

m∑i=1

(bi − B(ui ))TΓ−1noise(bi − B(ui ))

)

I The posterior (Bayes’ theorem):

P(u0|b1, . . . ,bm) ∝ exp(− 1

2

∑mi=1(B(ui )− bi )

TΓ−1noise(bi − B(ui ))

)+ 1

2 uT0 C−1

prioru0

where Γnoise is an observation error covariance matrix, and Cprior is aparameter prior covariance matrix.

I Solution strategy: Given observation(s), and a prior (or only ensemble fromit), describe the posterior PDF, and use it to make predictions.



Challenges in practical settings

I Solving the inverse problem for high dimensional model state spaces, e.g.O(109) for realistic atmospheric models; (it is what it is!).

I Nonlinearity of model, or observation operators (linearization or sampling).We consider the linear case here.

I Optimal sensor placement or measurements collection strategy (Design ofExperiments).

I Construction of full-rank Cprior based on limited-size ensemble of modelstates (Spatial Localization).



Design of Experiments and Sensor Placement

I Design of experiments (DOE) involves applying engineering principles andtechniques for data collection so as to ensure the conclusions made based onthe experiment are valid and reliable.

I Suppose we have Nobs candidate observation locations. Where do we putsensors, to actually collect measurements?

I Alternatively, find the measurement locations such that the uncertainty in theQoI is minimized.



Motivation




GO-OED for PDE-based Bayesian Linear Inverse Problems Optimal Experimental Design (OED) [13/28]


Problem Formulation:I Consider an additive noise model:

b = F(m) + η . (1)

where F is the “Forward Operator” e.g. (B S).

I In a Bayesian formulation, we obtain the posterior law for m through,

dµbpost

dµpr∝ πlike(b|m) , (2)

where the data likelihood is given by, πlike(b|m) = ρnoise(d −F(m)).

I An experimental design, ξ, will specify the way data is collected (e.g. sensorplacement). In OED problem, we seek ξopt that minimizes the uncertainty inthe inferred parameter m, i.e. we consider:

dµb|ξpost

dµpr∝ πlike(b|m; ξ) . (3)



Problem Formulation: OED

I The overall goal then is to find ξ for which the “posterior uncertainty” of mis minimized.

I The way one chooses to quantify “posterior uncertainty” leads to the choiceof the design criterion.

I Alphabetical criteria:

1. minimize the trace of the posterior covariance (A-Optimality)

2. minimize the determinant of the posterior covariance (D-Optimality)

3. minimize the largest eigenvalue of the posterior covariance (E-Optimality)

4. etc.



Problem Formulation: OED

I Consider an additive Gaussian noise model:

b = F(m) + η, η ∼ N (0,Γnoise) ,

where, F is defined by solving an equation of the form

ut +Au = 0 ,

u(x , 0) = m + boundary conditions .(4)

I The likelihood: πlike(b|m) ∝ exp¶− 1

2

(F(m)− b

)TΓ−1

noise

(F(m)− b

)©.

I In the context of optimal sensor placement, the design ξ is a vector w ofweights corresponding to a set of candidate locations for sensors.



Problem Formulation: OEDI w enters the Bayesian inverse problem through the data likelihood,

amounting to a weighted data likelihood:

πlike(b|m; w) ∝ exp

ß−1

2

(F(m)− b

)TW1/2Γ−1

noiseW1/2(F(m)− b

)™,

where W = diag(w1, . . . ,wns ).

I Posterior (Gaussian Linear case) N (0, Cpost) with

Cpost(w) = (F∗WσF + C−1prior)

−1 ≡ H−1 ,

where Wσ = W1/2Γ−1noiseW1/2.

I OED: find w that minimizes (Cpost trace, determinant, etc.)

I what if we are interested in a prediction quantity p = P(m) rather than theparameter itself? e.g. the average contaminant concentration within aspecific distance from the buildings’ walls. (GO-OED)



Motivation




GO-OED for PDE-based Bayesian Linear Inverse Problems Goal-Oriented OED [18/28]


GO-OED: the objective

I we consider a linear, scalar Θpred of the form,

Θpred(m) = 〈v ,Pm〉 .

I The variance of the prediction Θpred:

σ2pred(w) = 〈v ,PCpost(w)P∗v〉 .

I This will be used to define the goal-oriented A-optimal OED (equivalent toC-Optimality here) objective function.

I The goal would be to find w that minimizes σ2pred.

I We need the gradient ∇w (σ2pred)!



GO-OED: the gradient

I One version:

∇wσ2pred(w) = −

(Γ− 1

2noiseFH−1P∗v

)(

Γ− 1

2noiseFH−1P∗v

)

I A simpler version:

∇wσ2pred(w) = −

¨v ,PCpr (∇w [Hmisfit])

T Cpr P∗v∂

+¨v ,PCpr (∇w [Hmisfit])

T CprF∗ [W−1

σ + FCprF∗]−1

FCpr P∗v∂

+¨v ,PCprF

∗ [W−1σ + FCprF

∗]−1FCpr (∇w [Hmisfit])

T Cpr P∗v∂

−¨v ,PCprF

∗ [W−1σ + FCprF

∗]−1FCpr (∇w [Hmisfit])

T CprF∗ [W−1

σ + FCprF∗]−1

FCpr P∗v∂

where Hmisfit = F∗W1/2Γ−1noiseW1/2F.



GO-OED: preliminary results I

I Consider the Advection diffusion problem.

I Solution of the inverse problem:

200204060

80

100

True Parameter

(a) True m

4020020406080100

Prior sample 1

(b) A prior sample

20

0

20

40

60

MAP Estimator

(c) The MAP

Figure : Inverse problem components and solution



GO-OED: preliminary results II

I An OED solution:

0

10

20

30

40

22.5

25.0

27.5

30.0

32.5

35.0

37.5

40.0

Prior standard deviation

(a) Pointwise prior standard deviations

0

10

20

30

40

4

6

8

10

12

14

16

18

posterior std dev (dots == sensors)

(b) Posterior standard deviations

Figure : Sensor Placement (GO-OED)



GO-OED: preliminary results IIII GO-OED with Θpred = σ2

pred and 10 candidate measurement locations in thevicinity of the two buildings:

1 2 3 4 5 6 7 8 9 10Candidate locations

0

50

100

150

200

250

w

1 1 1 1 1 1 1 1 1 1

222

17

0 0

31

161

133

0

48

0

Initial Design; cost=0.1283Optimal Design; cost=0.079

Figure : Sensor Placement (OED)



Next:

I Develop the GO-OED interesting alphabetical criteria for non-scalarpredictive QoIs.

I Gradient validation and more numerical results.



Later: involve KL-divergence:

I instead of considering average variance, consider the expectedKullback-Leibler divergence from the prior and posterior predictivedistributions of Θpred. That is consider

DKL(νprior‖νy|wpost) ,

whereνprior = N (Θpred(mpr), 〈v ,PCpriorP∗v〉) ,

νy|wpost = N (Θpred(mMAP(w)), 〈v ,PCpost(w)P∗v〉) .

I One can use the expected information gain defined as

Ψ(w) :=

∫ ∫DKL(νprior‖νy|w

post)πlike(y|m)dy µpr(dm).



Future: GO-OED for Adaptive Spatial Localization in DA

I Put the design on the covariance matrix Cprior, or more practically on it’sprojection in the observational space.

I For example:

Cpost(w) = (F∗Γ−1noiseF + W C−1

prior)−1 ,

where W = (wij)i,j=1,...,Nstate.



Main Tools used:

I FEniCS/DOLFIN (Finite Element Computational Software):https://fenicsproject.org/

I hIPPYlib (Inverse Problems Python Library):https://hippylib.github.io/

I DATeS (Data Assimilation Testing Suite):https://bitbucket.org/a_attia/dates



https://fenicsproject.org/

https://hippylib.github.io/

https://bitbucket.org/a_attia/dates

References

I Alexanderian, Alen, et al. ”A-Optimal Design of Experiments forInfinite-Dimensional Bayesian Linear Inverse Problems with Regularized`0-Sparsification.” SIAM Journal on Scientific Computing 36.5 (2014):A2122-A2148.

I Chaloner, Kathryn, and Isabella Verdinelli. ”Bayesian experimental design: Areview.” Statistical Science (1995): 273-304.

I Evensen, Geir. ”The ensemble Kalman filter: Theoretical formulation andpractical implementation.” Ocean dynamics 53.4 (2003): 343-367.

I Haber, Eldad, Lior Horesh, and Luis Tenorio. ”Numerical methods forexperimental design of large-scale linear ill-posed inverse problems.” InverseProblems 24.5 (2008): 055012.

I Kalnay, Eugenia. Atmospheric modeling, data assimilation and predictability.Cambridge university press, 2003.

Thank You!GO-OED for PDE-based Bayesian Linear Inverse Problems Goal-Oriented OED [28/28]


goal-oriented optimal experimental design · 2016-11-18 · ahmed attia statistical and applied...

Documents