Sparse Grid Quadrature and Interpolation

Note: We have encountered potentially high dimensional quadrature at several points:!

⇡(q|�obs

) =⇡(�

obs

|q)⇡0(q)RRp ⇡(�obs|q)⇡0(q)dq

Bayes’ Relation:!

QoI:! y(t, x) = E[uK(t, x,Q)] =

Z

�u

K(t, x, q)⇢Q(q)dq

Discrete Projection:! uk(t, x) =1

�khu, ki⇢ =

1

�k

Z

�u(t, x, q) k(q)⇢Q(q)dq

Stochastic Quadrature Methods: Monte Carlo!

hu, ki⇢ =1

R

RX

r=1

u(t, x, qr) k(qr)⇢Q(q

r) + "R

Notes:!•  Errors satisfy!•  Optimal for sufficiently large p.!

E["R] = 0 and "R = O‘ ( 1pR) for large R

Deterministic Quadrature Methods

1-D Quadrature Relations:!

I(1)f =

Z

�1

f(q)⇢Q(q)dq ⇡RX

r=1

f(qr)wr = Q(1)f

Gaussian Quadrature:!

I(1)f =1

2

Z 1

�1f(q)dq ⇡ 1

2

RX

r=1

f(qr)wr,

I(1)f =1p2⇡

Z

Rf(q)e�q2/2dq ⇡

RX

r=1

f(qr)wr

r Nodes qr Weights wr

1 0 2

2 ± 1p3

1

3 0 89

±q

35

59

4 ±p

15+2p30p

3549

6(18+p30)

±p

15�2p30p

3549

6(18�p30)

1-D Quadrature Relations

Nested Quadrature Techniques: Consider on [0,1]!

Q(1)` f =

RX̀

r=1

f (qr` )wr`

Trapezoid Rule:!

Q(1)` f =

1

2h`

"f(0) + f(1) + 2

R`�1X

r=1

f(qr` )

#

where!

h` =1

2`�1, R` = 2`�1+1 , qr` = rh` =

r

2`�1, wr =

1

2h`,1

h`, · · · , 1

h`,

1

2h`

�

0 0.25 0.5 0.75 10

1

2

3

4

5

6

Level

Clenshaw--Curtis:!

qr` =

1

2

1� cos

⇡(r � 1)

R` � 1

�, r = 1, · · · , R`

0 0.25 0.5 0.75 10

1

2

3

4

5

6

Level

Tensor Product Formulation

l1

Levell2

0

0.5

1

0

0.5

1

0

0.5

1

0 0.5 10

0.5

1

0 0.5 1 0 0.5 1 0 0.5 1

Level

Integrals:!

I(p)f =

Z

�f(q)⇢Q(q)dq

Tensor Product Quadrature:!

Q(p)` f =

⇣Q(1)

`1⌦ · · ·⌦Q(1)

`p

⌘f

⌘R`1X

r1=1

· · ·R`pX

rp=1

f�q r11 , · · · , q rp

p

�wr1

`1· · ·wrp

`p

where!

R =pY

i=1

R`i

Errors:!���I(p)f �Q(p)` f

��� = O‘ (R�↵/p` )

for functions f in the space!

C↵([0, 1]p) =

(f : [0, 1]p ! R

����� max

|k0|↵

�����@|k0|f

@q k11 · · · @q kp

p

�����1

< 1)

Sparse Grid Construction Motivation:!

R

0 1

1 x y

2 x

2xy y

2

3 x

3x

2y xy

2y

3

4 x

4x

3y x

2y

2xy

3y

4

Difference Relations: Define!

�(1)` f =

⇣Q(1)

` �Q(1)`�1

⌘f

where!

Q(1)` f =

RX̀

r=1

f(qr` )wr`

1-D Nodal Points!

⇥(1)` =

n

q 1` , · · · , q

R``

o

Example: Trapezoid rule!

` = 2: ⇥(1)2 = {0, 1

2 , 1} and w = [ 14 ,12 ,

14 ]

` = 1 : ⇥(1)1 = {0, 1} and w = [ 12 ,

12 ]

Thus!

�(1)2 f = �1

4f(0) +

1

2f(1/2)� 1

4f(1)

Note: Weights can be negative!

Sparse Grid Construction Sparse Grid Quadrature Rule:!

Q(p)` f =

X

|`0|`+p�1

⇣�(1)

`1⌦ · · ·⌦�(1)

`p

⌘f

where `0 = (`1, · · · , `p) 2 Npis a multi-index with |`0| =

Ppi=1 `i.

Note: Tensor product formula can be expressed as!

Q(p)` f =

X

max `0`

⇣�(1)

`1⌦ · · ·⌦�(1)

`p

⌘f

where max `0 ⌘ max{`1, · · · , `p}.

Sparse Grid Nodal Set:!

⇥(p)` =

[

|`0|`+p�1

⇥(1)`1

⇥ · · ·⇥⇥(1)`p

.

Sparse Grid Construction Example:!Consider Clenshaw-Curtis with ⇥(1)

1 = { 12} and ⇥(1)

2 = {0, 12 , 1}. For p = 2,

`0 = (`1, `2) so |`0| = `1 + `2. For ` = 4, the sparse grid nodal set is

⇥(2)4 =

⇣⇥(1)

1 ⇥⇥(1)1

⌘, (`1 = 1, `2 = 1)

[⇣⇥(1)

1 ⇥⇥(1)2

⌘[⇣⇥(1)

2 ⇥⇥(1)1

⌘

[⇣⇥(1)

1 ⇥⇥(1)3

⌘[⇣⇥(1)

2 ⇥⇥(1)2

⌘[⇣⇥(1)

1 ⇥⇥(1)3

⌘

[⇣⇥(1)

1 ⇥⇥(1)4

⌘[⇣⇥(1)

2 ⇥⇥(1)3

⌘[⇣⇥(1)

3 ⇥⇥(1)2

⌘[⇣⇥(1)

4 ⇥⇥(1)1

⌘

l1

Levell2

0

0.5

1

0

0.5

1

0

0.5

1

0 0.5 10

0.5

1

0 0.5 1 0 0.5 1 0 0.5 1

Level

X

2

l1

X

X

X

X

X

X X X

X

1 2 3 4

3

4

2

1

l

0 0.25 0.5 0.75 10

0.25

0.5

0.75

1

Sparse Grid: 29 points!Tensored Grid: 81 points!

Sparse Grid Construction Error Analysis:!

kIf �A(q, p)fk = O‘⇣R�↵

log(R)

(p�1)(↵+1)⌘

Grid Sizes:!

p R` Sparse Grid R Tensored Grid R = (R`)p

2 5 13 25

9 29 81

5 5 61 3125

9 241 59,049

10 5 221 9,765,625

9 1581 > 3⇥ 109

50 5 5101 > 8⇥ 1034

9 171,901 > 5⇥ 1047

100 5 20,201 > 7⇥ 1069

9 1,353,801 > 2⇥ 1095

Anisotropic Sparse Grids Multi-Index Set:!

I(`) =(`0 2 Np

��� `0 · a =pX

i=1

ai`i `+ p� 1

)

where a 2 Rp+ is a vector of weights.

7

2

l1

X

X

X

X

X

X X X

X

X

X

X

X

X

X

X

7

6

5

4

3

2

1

1 2 3 4 5 6

l

0 0.25 0.5 0.75 10

0.25

0.5

0.75

1

Interpolating Polynomials 1-D Interpolation: Seek polynomials that satisfy!

uM1(qm) = um = u(qm) , m = 1, · · · ,M1

Lagrange Interpolation: Employ representation!

uM1(q) =M1X

m=1

umLm(q)

where Lm(q) are Lagrange interpolating polynomials defined by

Lm(q) =M1Y

j=0j 6=m

q � q j

qm � q j

=(q � q 1) · · · (q � qm�1)(q � qm+1) · · · (q � qM1)

(qm � q 1) · · · (qm � qm�1)(qm � qm+1) · · · (qm � qM1)

Note: By construction, the Lagrange polynomials satisfy!

Lm(q n) = �mn , 1 m,n M1

1-D Interpolating Polynomials Example: Consider the Runge function!f(q) =

1

1 + 25q2with points!

qj = �1 + (j � 1)2

M1, j = 1, · · · ,M1 + 1 qj = � cos

⇡(j � 1)

M1 � 1

, j = 1, · · · ,M1

−1 −0.5 0 0.5 1−0.2

0

0.2

0.4

0.6

0.8

1

1.2

q

f(q)

−1 −0.5 0 0.5 10

0.2

0.4

0.6

0.8

1

q

f(q)

−1 −0.5 0 0.5 1−80

−60

−40

−20

0

20

q

f(q)

−1 −0.5 0 0.5 10

0.2

0.4

0.6

0.8

1

q

f(q)

Multi-Dimensional Interpolation Tensor Products and Sparse Grid: Interpolation formulae analogous to quadrature rules.!

Sparse Grid Software: !•  MATLAB: Sparse Grid Interpolation Toolbox – Be careful of Clenshaw-Curtis,

which are actually Newton-Cotes points – versus Chebychev!•  Dakota!

Uses: !•  Spectral collocation methods to propagate input uncertainties.!

•  Construction of surrogate models.!