siam cse 2017 talk

Adaptive Methods for Cubature

Fred J. HickernellDepartment of Applied Mathematics, Illinois Institute of Technology

[email protected] mypages.iit.edu/~hickernell

Thanks to Lan Jiang, Tony Jiménez Rugama, Jagadees Rathinavel,and the rest of the the Guaranteed Automatic Integration Library (GAIL) team

Supported by NSF-DMS-1522687

For more details see H. (2017+), H. et al. (2017+), and Choi et al. (2013–2015)

mailto:[email protected]

[email protected]

http://mypages.iit.edu/~hickernell

mypages.iit.edu/~hickernell

Problem IID Monte Carlo Low Discrepancy Sampling Bayesian Cubature Numerical Examples References

Problemanswer(µ) =?, µ =

ż

Rdf(x)ν(dx) P Rp, pµn =

nÿ

i=1wif(xi)

Given εa, εr, choose n and {answer dependent on pµn to guarantee∣∣answer(µ)´ {answer∣∣ ď max(εa, εr |answer(µ)|) (with high probability)

adaptively and automatically

If µ P [pµn ´ errn, pµn + errn] (with high probability), then the optimal andsuccessful choice is

{answer =ans´max(εa, εr |ans+|) + ans+ max(εa, εr |ans´|)

max(εa, εr |ans+|) + max(εa, εr |ans´|)

where ans˘ :=

"

supinf

*

µ P [pµn ´ errn, pµn + errn]answer(µ)

provided|ans+ ´ ans´|

max(εa, εr |ans+|) + max(εa, εr |ans´|)ď 1 (H. et al., 2017+)

2/10



ż


nÿ

i=1wif(xi)



E.g.,

option price =

ż

Rdpayoff(x)

e´xTΣ´1x/2

(2π)d/2 |Σ|1/2 dx, Σ =

(min(i, j)T/d

)d

i,j=1

Gaussian probability =

ż

[a,b]

e´xTΣ´1x/2

(2π)d/2 |Σ|1/2 dx Genz (1993)

=

ż

[0,1]d´1f (x)dx

Sobol’ indexj =

ş

[0,1]2d

[output(x)´ output(xj : x

1´j)]output(x 1)dxdx 1

ş

[0,1]d output(x)2 dx´[ş

[0,1]d output(x)dx]2

Bayesian estimatej =

ş

Rd βj prob(data|β) probprior(β)dβş

Rd prob(data|β) probprior(β)dβ




where ans˘ :=

"

supinf

*




2/10



ż


nÿ

i=1wif(xi)






where ans˘ :=

"

supinf

*




2/10


Berry-Esseen Stopping Rule for IID Monte Carlo

µ =

ż

Rdf (x)ν(dx)

pµn =1n

nÿ

i=1f (xi), xi

IID„ ν

Need µ P [µ̂n ´ errn, µ̂n + errn]with high probability

P[|µ´ µ̂n| ď errn] « 99%for Φ

(´?

n errn /(1.2σ̂))= 0.005

by the Central Limit Theorem

where σ̂2 is the sample variation3/10


Berry-Esseen Stopping Rule for IID Monte Carlo

µ =

ż

Rdf (x)ν(dx)

pµn =1n

nÿ

i=1f (xi), xi

IID„ ν


P[|µ´ µ̂n| ď errn] ě 99%for Φ

(´?

n errn /(1.2σ̂nσ))

+ ∆n(´?

n errn /(1.2σ̂nσ), κmax) = 0.0025by the Berry-Esseen Inequality

where σ̂2nσ is the sample variation using an independent sample, and provided

that kurt(f (X)) ď κmax(nσ) (H. et al., 2013; Jiang, 2016)

3/10


Adaptive Low Discrepancy Sampling Cubature

µ =

ż

[0,1]df (x)dx

pµn =1n

nÿ

i=1f (xi), xi Sobol’ or lattice

Need µ P [µ̂n ´ errn, µ̂n + errn]

Express µ´ µ̂n in terms of the Fourier coefficients of f . Assuming that thesecoefficients do not decay erratically, the discrete transform,

rfn,κ(n´1κ=0, may be

used to bound the error reliably (H. and Jiménez Rugama, 2016; Jiménez Rugama andH., 2016; H. et al., 2017+):

|µ´ µ̂n| ď errn := C(n, `)2`´1ÿ

κ=2`´1

∣∣∣rfn,κ∣∣∣

4/10


Adaptive Low Discrepancy Sampling Cubature

µ =

ż

[0,1]df (x)dx

pµn =1n

nÿ

i=1f (xi), xi Sobol’ or lattice

Need µ P [µ̂n ´ errn, µ̂n + errn]

Express µ´ µ̂n in terms of the Fourier coefficients of f . Assuming that thesecoefficients do not decay erratically, the discrete transform,

rfn,κ(n´1κ=0, may be

used to bound the error reliably (H. and Jiménez Rugama, 2016; Jiménez Rugama andH., 2016; H. et al., 2017+):

|µ´ µ̂n| ď errn := C(n, `)2`´1ÿ

κ=2`´1

∣∣∣rfn,κ∣∣∣4/10


Bayesian Cubature—f Is Random

µ =

ż

Rdf (x)ν(dx)

pµn =nÿ

i=1wi f (xi)


Assume f „ GP(0,C). Choose the wi to integrate the best estimate of f given thedata txi, f (xi)u

ni=1 (Diaconis, 1988; O’Hagan, 1991; Ritter, 2000; Rasmussen and

Ghahramani, 2003)

P[|µ´ µ̂n| ď errn] = 99% for errn = an expression involving C and txi, f (xi)uni=1

A de-randomized interpretation exists (H., 2017+)5/10


Gaussian Probability

µ =

ż

[a,b]

exp(´ 1

2tTΣ´1t

)a

(2π)d det(Σ)dt Genz (1993)

=

ż

[0,1]d´1f (x)dx

For some typical choice of a, b, Σ, d = 3, εa = 0; µ « 0.6763

Worst 10% Worst 10%εr Method % Accuracy n Time (s)

IID Monte Carlo 100% 8.1E4 1.8E´21E´2 Sobol’ Sampling 100% 1.0E3 5.1E´3

Bayesian Lattice 100% 1.0E3 2.8E´3

IID Monte Carlo 100% 2.0E6 3.8E´11E´3 Sobol’ Sampling 100% 2.0E3 7.7E´3

Bayesian Lattice 100% 1.0E3 2.8E´3

1E´4 Sobol’ Sampling 100% 1.6E4 1.8E´2Bayesian Lattice 100% 8.2E3 1.4E´2

6/10


Sobol’ IndicesY = output(X), where X „ U[0, 1]d; Sobol’ Indexj(µ) describes how muchcoordinate j of input X influences output Y (Sobol’, 1990; 2001):

Sobol’ Indexj(µ) :=µ1

µ2 ´ µ23, j = 1, . . . , d

µ1 :=

ż

[0,1)2d[output(x)´ output(xj : x

1´j)]output(x 1)dxdx 1

µ2 :=

ż

[0,1)doutput(x)2 dx, µ3 :=

ż

[0,1)doutput(x)dx.

output(x) = ´x1 + x1x2 ´ x1x2x3 + ¨ ¨ ¨+ x1x2x3x4x5x6 (Bratley et al., 1992)

εa = 1E´3, εr = 0 j 1 2 3 4 5 6n 65 536 32 768 16 384 16 384 2 048 2 048

Sobol’ Indexj 0.6529 0.1791 0.0370 0.0133 0.0015 0.0015{Sobol’ Indexj 0.6528 0.1792 0.0363 0.0126 0.0010 0.0012

Sobol’ Indexj(pµn) 0.6492 0.1758 0.0308 0.0083 0.0018 0.00397/10

Thank you

Slides available atwww.slideshare.net/fjhickernell/siam-cse-2017-talk

http://www.slideshare.net/fjhickernell/siam-cse-2017-talk


References I

Bratley, P., B. L. Fox, and H. Niederreiter. 1992. Implementation and tests of low-discrepancysequences, ACM Trans. Model. Comput. Simul. 2, 195–213.

Choi, S.-C. T., Y. Ding, F. J. H., L. Jiang, Ll. A. Jiménez Rugama, X. Tong, Y. Zhang, and X. Zhou.2013–2015. GAIL: Guaranteed Automatic Integration Library (versions 1.0–2.1).

Cools, R. and D. Nuyens (eds.) 2016. Monte Carlo and quasi-Monte Carlo methods: MCQMC,Leuven, Belgium, April 2014, Springer Proceedings in Mathematics and Statistics, vol. 163,Springer-Verlag, Berlin.

Diaconis, P. 1988. Bayesian numerical analysis, Statistical decision theory and related topics IV,Papers from the 4th Purdue symp., West Lafayette, Indiana 1986, pp. 163–175.

Genz, A. 1993. Comparison of methods for the computation of multivariate normal probabilities,Computing Science and Statistics 25, 400–405.

H., F. J. 2017+. Error analysis of quasi-Monte Carlo methods. submitted for publication,arXiv:1702.01487.

H., F. J., L. Jiang, Y. Liu, and A. B. Owen. 2013. Guaranteed conservative fixed width confidenceintervals via Monte Carlo sampling, Monte Carlo and quasi-Monte Carlo methods 2012, pp. 105–128.

H., F. J. and Ll. A. Jiménez Rugama. 2016. Reliable adaptive cubature using digital sequences,Monte Carlo and quasi-Monte Carlo methods: MCQMC, Leuven, Belgium, April 2014, pp. 367–383.arXiv:1410.8615 [math.NA].

9/10


References II

H., F. J., Ll. A. Jiménez Rugama, and D. Li. 2017+. Adaptive quasi-Monte Carlo methods. submittedfor publication, arXiv:1702.01491 [math.NA].

Jiang, L. 2016. Guaranteed adaptive Monte Carlo methods for estimating means of randomvariables, Ph.D. Thesis.

Jiménez Rugama, Ll. A. and F. J. H. 2016. Adaptive multidimensional integration based on rank-1lattices, Monte Carlo and quasi-Monte Carlo methods: MCQMC, Leuven, Belgium, April 2014,pp. 407–422. arXiv:1411.1966.

O’Hagan, A. 1991. Bayes-Hermite quadrature, J. Statist. Plann. Inference 29, 245–260.

Rasmussen, C. E. and Z. Ghahramani. 2003. Bayesian Monte Carlo, Advances in Neural InformationProcessing Systems, pp. 489–496.

Ritter, K. 2000. Average-case analysis of numerical problems, Lecture Notes in Mathematics,vol. 1733, Springer-Verlag, Berlin.

Sobol’, I. M. 1990. On sensitivity estimation for nonlinear mathematical models, Matem. Mod. 2,no. 1, 112–118.

. 2001. Global sensitivity indices for nonlinear mathematical models and their monte carloestimates, Math. Comput. Simul. 55, no. 1-3, 271–280.

10/10