some recent advances on approximate bayesian … · thanks numerous colleagues participated to...
TRANSCRIPT
![Page 1: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/1.jpg)
Some recent advances on ApproximateBayesian Computation techniques
Jean-Michel Marin
University of Montpellier, CNRSAlexander Grothendieck Montpellier Institute
9 December 2017
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 1 / 41
![Page 2: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/2.jpg)
Thanks
Numerous colleagues participated to parts of this work
I Pierre Pudlo (Marseille)I Louis Raynal (PhD student Montpellier)I Arnaud Estoup (molecular ecologist, Montpellier)I Christian Robert (Paris and Warwick)I Judith, Natesh, ...
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 2 / 41
![Page 3: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/3.jpg)
Thanks
Numerous colleagues participated to parts of this work
I Pierre Pudlo (Marseille)
I Louis Raynal (PhD student Montpellier)I Arnaud Estoup (molecular ecologist, Montpellier)I Christian Robert (Paris and Warwick)I Judith, Natesh, ...
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 2 / 41
![Page 4: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/4.jpg)
Thanks
Numerous colleagues participated to parts of this work
I Pierre Pudlo (Marseille)I Louis Raynal (PhD student Montpellier)
I Arnaud Estoup (molecular ecologist, Montpellier)I Christian Robert (Paris and Warwick)I Judith, Natesh, ...
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 2 / 41
![Page 5: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/5.jpg)
Thanks
Numerous colleagues participated to parts of this work
I Pierre Pudlo (Marseille)I Louis Raynal (PhD student Montpellier)I Arnaud Estoup (molecular ecologist, Montpellier)
I Christian Robert (Paris and Warwick)I Judith, Natesh, ...
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 2 / 41
![Page 6: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/6.jpg)
Thanks
Numerous colleagues participated to parts of this work
I Pierre Pudlo (Marseille)I Louis Raynal (PhD student Montpellier)I Arnaud Estoup (molecular ecologist, Montpellier)I Christian Robert (Paris and Warwick)
I Judith, Natesh, ...
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 2 / 41
![Page 7: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/7.jpg)
Thanks
Numerous colleagues participated to parts of this work
I Pierre Pudlo (Marseille)I Louis Raynal (PhD student Montpellier)I Arnaud Estoup (molecular ecologist, Montpellier)I Christian Robert (Paris and Warwick)I Judith, Natesh, ...
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 2 / 41
![Page 8: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/8.jpg)
Introduction
Bayesian parametric paradigm
Likelihood function f(y|θ) expensive or impossible to calculate
Extremely difficult to sample from the posterior distribution
π(θ|y) ∝ π(θ)f(y|θ)
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 3 / 41
![Page 9: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/9.jpg)
Introduction
Bayesian parametric paradigm
Likelihood function f(y|θ) expensive or impossible to calculate
Extremely difficult to sample from the posterior distribution
π(θ|y) ∝ π(θ)f(y|θ)
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 3 / 41
![Page 10: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/10.jpg)
Introduction
Bayesian parametric paradigm
Likelihood function f(y|θ) expensive or impossible to calculate
Extremely difficult to sample from the posterior distribution
π(θ|y) ∝ π(θ)f(y|θ)
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 3 / 41
![Page 11: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/11.jpg)
Introduction
Two typical situations
I f(y|θ) =∫f(y, u|θ)µ(du) intractable
population genetics models, coalescent process
EM algorithms, Gibbs sampling, pseudo-marginalMCMC methods, variational approximations
I f(y|θ) = g(y,θ)/Z(θ) and Z(θ) intractableMarkov random field
pseudo-marginal MCMC methods, variationalapproximations
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 4 / 41
![Page 12: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/12.jpg)
Introduction
Two typical situations
I f(y|θ) =∫f(y, u|θ)µ(du) intractable
population genetics models, coalescent process
EM algorithms, Gibbs sampling, pseudo-marginalMCMC methods, variational approximations
I f(y|θ) = g(y,θ)/Z(θ) and Z(θ) intractableMarkov random field
pseudo-marginal MCMC methods, variationalapproximations
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 4 / 41
![Page 13: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/13.jpg)
Introduction
Two typical situations
I f(y|θ) =∫f(y, u|θ)µ(du) intractable
population genetics models, coalescent process
EM algorithms, Gibbs sampling, pseudo-marginalMCMC methods, variational approximations
I f(y|θ) = g(y,θ)/Z(θ) and Z(θ) intractableMarkov random field
pseudo-marginal MCMC methods, variationalapproximations
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 4 / 41
![Page 14: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/14.jpg)
Introduction
Two typical situations
I f(y|θ) =∫f(y, u|θ)µ(du) intractable
population genetics models, coalescent process
EM algorithms, Gibbs sampling, pseudo-marginalMCMC methods, variational approximations
I f(y|θ) = g(y,θ)/Z(θ) and Z(θ) intractableMarkov random field
pseudo-marginal MCMC methods, variationalapproximations
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 4 / 41
![Page 15: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/15.jpg)
Introduction
Two typical situations
I f(y|θ) =∫f(y, u|θ)µ(du) intractable
population genetics models, coalescent process
EM algorithms, Gibbs sampling, pseudo-marginalMCMC methods, variational approximations
I f(y|θ) = g(y,θ)/Z(θ) and Z(θ) intractableMarkov random field
pseudo-marginal MCMC methods, variationalapproximations
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 4 / 41
![Page 16: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/16.jpg)
Introduction
ABC is a technique that only requires being able to samplefrom the likelihood f(·|θ)
This technique stemmed from population genetics models,about 15 years ago, and population geneticists still significantlycontribute to methodological developments of ABC
If, with Christian, we work on ABC methods, we can be verygrateful to our biologist colleagues!
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 5 / 41
![Page 17: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/17.jpg)
Introduction
ABC is a technique that only requires being able to samplefrom the likelihood f(·|θ)
This technique stemmed from population genetics models,about 15 years ago, and population geneticists still significantlycontribute to methodological developments of ABC
If, with Christian, we work on ABC methods, we can be verygrateful to our biologist colleagues!
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 5 / 41
![Page 18: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/18.jpg)
Introduction
ABC is a technique that only requires being able to samplefrom the likelihood f(·|θ)
This technique stemmed from population genetics models,about 15 years ago, and population geneticists still significantlycontribute to methodological developments of ABC
If, with Christian, we work on ABC methods, we can be verygrateful to our biologist colleagues!
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 5 / 41
![Page 19: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/19.jpg)
Introduction
I some methodological aspects of ABC
I our ABC random forests proposalI ABC and PAC-Bayes
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 6 / 41
![Page 20: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/20.jpg)
Introduction
I some methodological aspects of ABCI our ABC random forests proposal
I ABC and PAC-Bayes
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 6 / 41
![Page 21: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/21.jpg)
Introduction
I some methodological aspects of ABCI our ABC random forests proposalI ABC and PAC-Bayes
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 6 / 41
![Page 22: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/22.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
Rubin (1984) The Annals of StatisticsTavare et al. (1997) GeneticsPritchard et al. (1999) Mol. Biol. Evol.
1) Set i = 12) Generate θ ′ from the prior distribution π(·)3) Generate z from the likelihood f(·|θ ′)4) If d(η(z),η(y)) 6 ε, set θi = θ ′ and i = i+ 15) If i 6 N, return to 2)
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 7 / 41
![Page 23: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/23.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
Rubin (1984) The Annals of StatisticsTavare et al. (1997) GeneticsPritchard et al. (1999) Mol. Biol. Evol.
1) Set i = 1
2) Generate θ ′ from the prior distribution π(·)3) Generate z from the likelihood f(·|θ ′)4) If d(η(z),η(y)) 6 ε, set θi = θ ′ and i = i+ 15) If i 6 N, return to 2)
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 7 / 41
![Page 24: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/24.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
Rubin (1984) The Annals of StatisticsTavare et al. (1997) GeneticsPritchard et al. (1999) Mol. Biol. Evol.
1) Set i = 12) Generate θ ′ from the prior distribution π(·)
3) Generate z from the likelihood f(·|θ ′)4) If d(η(z),η(y)) 6 ε, set θi = θ ′ and i = i+ 15) If i 6 N, return to 2)
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 7 / 41
![Page 25: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/25.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
Rubin (1984) The Annals of StatisticsTavare et al. (1997) GeneticsPritchard et al. (1999) Mol. Biol. Evol.
1) Set i = 12) Generate θ ′ from the prior distribution π(·)3) Generate z from the likelihood f(·|θ ′)
4) If d(η(z),η(y)) 6 ε, set θi = θ ′ and i = i+ 15) If i 6 N, return to 2)
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 7 / 41
![Page 26: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/26.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
Rubin (1984) The Annals of StatisticsTavare et al. (1997) GeneticsPritchard et al. (1999) Mol. Biol. Evol.
1) Set i = 12) Generate θ ′ from the prior distribution π(·)3) Generate z from the likelihood f(·|θ ′)4) If d(η(z),η(y)) 6 ε, set θi = θ ′ and i = i+ 1
5) If i 6 N, return to 2)
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 7 / 41
![Page 27: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/27.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
Rubin (1984) The Annals of StatisticsTavare et al. (1997) GeneticsPritchard et al. (1999) Mol. Biol. Evol.
1) Set i = 12) Generate θ ′ from the prior distribution π(·)3) Generate z from the likelihood f(·|θ ′)4) If d(η(z),η(y)) 6 ε, set θi = θ ′ and i = i+ 15) If i 6 N, return to 2)
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 7 / 41
![Page 28: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/28.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
ε reflects the tension between computability and accuracy
I if ε→∞, we get simulations from the prior
I if ε→ 0, we get simulations from the posterior
ABC target
πε(θ|y) =
∫π(θ)f(z|θ)I(z ∈ Aε,y)dz∫Aε,y×Θ π(θ)f(z|θ)dzdθ
Aε,y = {z|d(η(z),η(y)) 6 ε} the acceptance set
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 8 / 41
![Page 29: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/29.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
ε reflects the tension between computability and accuracy
I if ε→∞, we get simulations from the prior
I if ε→ 0, we get simulations from the posterior
ABC target
πε(θ|y) =
∫π(θ)f(z|θ)I(z ∈ Aε,y)dz∫Aε,y×Θ π(θ)f(z|θ)dzdθ
Aε,y = {z|d(η(z),η(y)) 6 ε} the acceptance set
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 8 / 41
![Page 30: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/30.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
ε reflects the tension between computability and accuracy
I if ε→∞, we get simulations from the prior
I if ε→ 0, we get simulations from the posterior
ABC target
πε(θ|y) =
∫π(θ)f(z|θ)I(z ∈ Aε,y)dz∫Aε,y×Θ π(θ)f(z|θ)dzdθ
Aε,y = {z|d(η(z),η(y)) 6 ε} the acceptance set
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 8 / 41
![Page 31: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/31.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
ε reflects the tension between computability and accuracy
I if ε→∞, we get simulations from the prior
I if ε→ 0, we get simulations from the posterior
ABC target
πε(θ|y) =
∫π(θ)f(z|θ)I(z ∈ Aε,y)dz∫Aε,y×Θ π(θ)f(z|θ)dzdθ
Aε,y = {z|d(η(z),η(y)) 6 ε} the acceptance set
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 8 / 41
![Page 32: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/32.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
A toy example from Richard Wilkinson (Tutorial on ABC,NIPS 2013)
y|θ ∼ N1(2(θ+ 2)θ(θ− 2), 0.1 + θ2
)θ ∼ U[−10,10]
y = 2
d(z,y) = |z− y|
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 9 / 41
![Page 33: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/33.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
A toy example from Richard Wilkinson (Tutorial on ABC,NIPS 2013)
y|θ ∼ N1(2(θ+ 2)θ(θ− 2), 0.1 + θ2
)
θ ∼ U[−10,10]
y = 2
d(z,y) = |z− y|
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 9 / 41
![Page 34: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/34.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
A toy example from Richard Wilkinson (Tutorial on ABC,NIPS 2013)
y|θ ∼ N1(2(θ+ 2)θ(θ− 2), 0.1 + θ2
)θ ∼ U[−10,10]
y = 2
d(z,y) = |z− y|
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 9 / 41
![Page 35: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/35.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
A toy example from Richard Wilkinson (Tutorial on ABC,NIPS 2013)
y|θ ∼ N1(2(θ+ 2)θ(θ− 2), 0.1 + θ2
)θ ∼ U[−10,10]
y = 2
d(z,y) = |z− y|
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 9 / 41
![Page 36: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/36.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
A toy example from Richard Wilkinson (Tutorial on ABC,NIPS 2013)
y|θ ∼ N1(2(θ+ 2)θ(θ− 2), 0.1 + θ2
)θ ∼ U[−10,10]
y = 2
d(z,y) = |z− y|
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 9 / 41
![Page 37: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/37.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
✏ = 7.5
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●● ●
●
●
● ●●
●
●
●
●
●
●
● ●
●
● ●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●●●
●
●
●●
●
●
●●●
●
●●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
−3 −2 −1 0 1 2 3
−10
010
20
theta vs D
theta
D●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●●● ●
●●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●● ●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●● ●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●●
●●
●
●
●
●
● ●
●●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
● ●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●●
●
●
●●
●●
●●
●● ●
● ●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
−ε
+ ε
D
−3 −2 −1 0 1 2 3
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Density
theta
Density
ABCTrue
ε = 7.5
✏ = 5
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
● ●●
●
●
●
●
●● ●
● ●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
● ●●●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
● ●
● ●
●
●
●●
●
●●● ●
●●
●
●
●
●
●
●
●●
●
●●●
●
●
●
● ●●
●
●
●
●
●●
●
●●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
−3 −2 −1 0 1 2 3
−10
010
20
theta vs D
theta
D
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●●● ●
●●
●
● ●
●
● ● ●●
●
●●
●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●●●
●● ●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●● ●
●●
●
●
●
●●
●● ●
●
●●
●●●●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●●
●
●●
●
●
●●
●
●
●
● ●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
● ●● ●
●
●●
●●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
● ●
●
●●●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●●
●
●● ●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
−ε
+ ε
D
−3 −2 −1 0 1 2 3
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Density
theta
Density
ABCTrue
ε = 5
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 10 / 41
![Page 38: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/38.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
✏ = 7.5
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●● ●
●
●
● ●●
●
●
●
●
●
●
● ●
●
● ●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●●●
●
●
●●
●
●
●●●
●
●●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
−3 −2 −1 0 1 2 3
−10
010
20
theta vs D
theta
D●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●●● ●
●●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●● ●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●● ●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●●
●●
●
●
●
●
● ●
●●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
● ●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●●
●
●
●●
●●
●●
●● ●
● ●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
−ε
+ ε
D
−3 −2 −1 0 1 2 3
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Density
theta
Density
ABCTrue
ε = 7.5
✏ = 5
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
● ●●
●
●
●
●
●● ●
● ●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
● ●●●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
● ●
● ●
●
●
●●
●
●●● ●
●●
●
●
●
●
●
●
●●
●
●●●
●
●
●
● ●●
●
●
●
●
●●
●
●●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
−3 −2 −1 0 1 2 3
−10
010
20
theta vs D
theta
D
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●●● ●
●●
●
● ●
●
● ● ●●
●
●●
●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●●●
●● ●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●● ●
●●
●
●
●
●●
●● ●
●
●●
●●●●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●●
●
●●
●
●
●●
●
●
●
● ●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
● ●● ●
●
●●
●●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
● ●
●
●●●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●●
●
●● ●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
−ε
+ ε
D
−3 −2 −1 0 1 2 3
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Density
theta
Density
ABCTrue
ε = 5
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 10 / 41
![Page 39: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/39.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
✏ = 2.5
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
● ● ●●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
● ●●
●
●
●●
●●
●
●
●
●
●
●
●
●
● ●
●●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
● ●
●
●
●
●
● ●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●●●
●
●
●●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
−3 −2 −1 0 1 2 3
−10
010
20
theta vs D
theta
D
●
●●
●
●●
●
●
●
●
●●
●●●●
●
● ●●
●
●●
●●
●●
●●●●
●
●●●
● ●●
●●
●●
●
● ●
●
●
●●
●
●●●
●
●
●●
●
●●
●●
●●
●
●●
●●●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●● ●
●
●
●
●
●
●●●
●
●●
●
●
●●
●●●
●
●
●
●
●●
●
●
●●
●
●
●
●●●
●●
●●
●
●
●●
●
●
●
●●●
●
●●●
●
●
●●●
● ●●
●
●●
●●●
●●
●●
●●
●●
●
●●●●●
●
●
●
●●
●●
●
●
●
●
●●●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
●
●●
●
●●
●● ●
●●
●●
●
●●
●
●
●●
●●
●
●
●
●●
● ●
●
●
●
●
●
●
●●
●●
●
●● ●
●
●●
●
●
−ε
+ ε
D
−3 −2 −1 0 1 2 3
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Density
theta
Density
ABCTrue
ε = 2.5
✏ = 1
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●● ●
●
●
● ●
●●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
−3 −2 −1 0 1 2 3
−10
010
20
theta vs D
theta
D ●●● ●●●
● ●●●●●●● ●
●●
●●● ●●● ●
●●
●●
●●
●●
●●●●●
●
●●●●●
●
●●●
●●●● ●●
●●
● ●●●●●
●
●●
●●
●●●●●
●●
●●
●●
●● ●● ●
● ●● ●●●
●
● ●● ● ●
●●●●●
−ε
+ ε
−3 −2 −1 0 1 2 3
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Density
theta
Density
ABCTrue
ε = 1
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 11 / 41
![Page 40: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/40.jpg)
Methodological aspects of ABCLikelihood-free rejection sampler
✏ = 2.5
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
● ● ●●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
● ●●
●
●
●●
●●
●
●
●
●
●
●
●
●
● ●
●●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
● ●
●
●
●
●
● ●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●●●
●
●
●●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
−3 −2 −1 0 1 2 3
−10
010
20
theta vs D
theta
D
●
●●
●
●●
●
●
●
●
●●
●●●●
●
● ●●
●
●●
●●
●●
●●●●
●
●●●
● ●●
●●
●●
●
● ●
●
●
●●
●
●●●
●
●
●●
●
●●
●●
●●
●
●●
●●●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●● ●
●
●
●
●
●
●●●
●
●●
●
●
●●
●●●
●
●
●
●
●●
●
●
●●
●
●
●
●●●
●●
●●
●
●
●●
●
●
●
●●●
●
●●●
●
●
●●●
● ●●
●
●●
●●●
●●
●●
●●
●●
●
●●●●●
●
●
●
●●
●●
●
●
●
●
●●●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
●
●●
●
●●
●● ●
●●
●●
●
●●
●
●
●●
●●
●
●
●
●●
● ●
●
●
●
●
●
●
●●
●●
●
●● ●
●
●●
●
●
−ε
+ ε
D
−3 −2 −1 0 1 2 3
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Density
theta
Density
ABCTrue
ε = 2.5
✏ = 1
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●● ●
●
●
● ●
●●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
−3 −2 −1 0 1 2 3
−10
010
20
theta vs D
theta
D ●●● ●●●
● ●●●●●●● ●
●●
●●● ●●● ●
●●
●●
●●
●●
●●●●●
●
●●●●●
●
●●●
●●●● ●●
●●
● ●●●●●
●
●●
●●
●●●●●
●●
●●
●●
●● ●● ●
● ●● ●●●
●
● ●● ● ●
●●●●●
−ε
+ ε
−3 −2 −1 0 1 2 3
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Density
theta
Density
ABCTrue
ε = 1
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 11 / 41
![Page 41: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/41.jpg)
Methodological aspects of ABCA k-NN approximation
Practitioners really use
1) For i = 1, . . . ,M
a) Generate θi from the prior π(·)b) Generate z from the model f(·|θi)c) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the θi’s that correspond to the N-smallest distances
N = bαMc
ε corresponds to a quantile of the distances
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 12 / 41
![Page 42: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/42.jpg)
Methodological aspects of ABCA k-NN approximation
Practitioners really use
1) For i = 1, . . . ,M
a) Generate θi from the prior π(·)b) Generate z from the model f(·|θi)c) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the θi’s that correspond to the N-smallest distances
N = bαMc
ε corresponds to a quantile of the distances
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 12 / 41
![Page 43: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/43.jpg)
Methodological aspects of ABCA k-NN approximation
Practitioners really use
1) For i = 1, . . . ,Ma) Generate θi from the prior π(·)
b) Generate z from the model f(·|θi)c) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the θi’s that correspond to the N-smallest distances
N = bαMc
ε corresponds to a quantile of the distances
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 12 / 41
![Page 44: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/44.jpg)
Methodological aspects of ABCA k-NN approximation
Practitioners really use
1) For i = 1, . . . ,Ma) Generate θi from the prior π(·)b) Generate z from the model f(·|θi)
c) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the θi’s that correspond to the N-smallest distances
N = bαMc
ε corresponds to a quantile of the distances
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 12 / 41
![Page 45: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/45.jpg)
Methodological aspects of ABCA k-NN approximation
Practitioners really use
1) For i = 1, . . . ,Ma) Generate θi from the prior π(·)b) Generate z from the model f(·|θi)c) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the θi’s that correspond to the N-smallest distances
N = bαMc
ε corresponds to a quantile of the distances
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 12 / 41
![Page 46: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/46.jpg)
Methodological aspects of ABCA k-NN approximation
Practitioners really use
1) For i = 1, . . . ,Ma) Generate θi from the prior π(·)b) Generate z from the model f(·|θi)c) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the θi’s that correspond to the N-smallest distances
N = bαMc
ε corresponds to a quantile of the distances
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 12 / 41
![Page 47: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/47.jpg)
Methodological aspects of ABCA k-NN approximation
Practitioners really use
1) For i = 1, . . . ,Ma) Generate θi from the prior π(·)b) Generate z from the model f(·|θi)c) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the θi’s that correspond to the N-smallest distances
N = bαMc
ε corresponds to a quantile of the distances
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 12 / 41
![Page 48: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/48.jpg)
Methodological aspects of ABCA k-NN approximation
Practitioners really use
1) For i = 1, . . . ,Ma) Generate θi from the prior π(·)b) Generate z from the model f(·|θi)c) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the θi’s that correspond to the N-smallest distances
N = bαMc
ε corresponds to a quantile of the distances
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 12 / 41
![Page 49: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/49.jpg)
Methodological aspects of ABCA k-NN approximation
Practitioners really use
1) For i = 1, . . . ,Ma) Generate θi from the prior π(·)b) Generate z from the model f(·|θi)c) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the θi’s that correspond to the N-smallest distances
N = bαMc
ε corresponds to a quantile of the distances
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 12 / 41
![Page 50: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/50.jpg)
Methodological aspects of ABCA k-NN approximation
New insights into Approximate Bayesian ComputationBiau, Cerou, Guyader (2015) Annales de l’IHP
I intuitiveI simple to implementI embarrassingly parallelisableI BUT curse of dimensionality: most of the simulations are at
the boundary of the space as the number of summarystatistics increases
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 13 / 41
![Page 51: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/51.jpg)
Methodological aspects of ABCA k-NN approximation
New insights into Approximate Bayesian ComputationBiau, Cerou, Guyader (2015) Annales de l’IHP
I intuitiveI simple to implementI embarrassingly parallelisableI BUT curse of dimensionality: most of the simulations are at
the boundary of the space as the number of summarystatistics increases
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 13 / 41
![Page 52: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/52.jpg)
Methodological aspects of ABCA k-NN approximation
New insights into Approximate Bayesian ComputationBiau, Cerou, Guyader (2015) Annales de l’IHP
I intuitive
I simple to implementI embarrassingly parallelisableI BUT curse of dimensionality: most of the simulations are at
the boundary of the space as the number of summarystatistics increases
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 13 / 41
![Page 53: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/53.jpg)
Methodological aspects of ABCA k-NN approximation
New insights into Approximate Bayesian ComputationBiau, Cerou, Guyader (2015) Annales de l’IHP
I intuitiveI simple to implement
I embarrassingly parallelisableI BUT curse of dimensionality: most of the simulations are at
the boundary of the space as the number of summarystatistics increases
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 13 / 41
![Page 54: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/54.jpg)
Methodological aspects of ABCA k-NN approximation
New insights into Approximate Bayesian ComputationBiau, Cerou, Guyader (2015) Annales de l’IHP
I intuitiveI simple to implementI embarrassingly parallelisable
I BUT curse of dimensionality: most of the simulations are atthe boundary of the space as the number of summarystatistics increases
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 13 / 41
![Page 55: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/55.jpg)
Methodological aspects of ABCA k-NN approximation
New insights into Approximate Bayesian ComputationBiau, Cerou, Guyader (2015) Annales de l’IHP
I intuitiveI simple to implementI embarrassingly parallelisableI BUT curse of dimensionality: most of the simulations are at
the boundary of the space as the number of summarystatistics increases
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 13 / 41
![Page 56: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/56.jpg)
Methodological aspects of ABCTwo views of the ABC approximation
=⇒ Wilkinson (2013) SAGMB shows that ABC is exact but fora different model to that intended
=⇒ Blum (2010) JASA emphasizes that ABC is a kernelsmoothing approximation of the likelihood function
πε(θ|y) =
∫π(θ)f(z|θ)I(z ∈ Aε,y)dz∫Aε,y×Θ π(θ)f(z|θ)dzdθ
=π(θ)
∫f(z|θ)K(d(η(z),η(y)))dz∫
π(θ)f(z|θ)K(d(η(z),η(y)))dzdθ
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 14 / 41
![Page 57: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/57.jpg)
Methodological aspects of ABCTwo views of the ABC approximation
=⇒ Wilkinson (2013) SAGMB shows that ABC is exact but fora different model to that intended
=⇒ Blum (2010) JASA emphasizes that ABC is a kernelsmoothing approximation of the likelihood function
πε(θ|y) =
∫π(θ)f(z|θ)I(z ∈ Aε,y)dz∫Aε,y×Θ π(θ)f(z|θ)dzdθ
=π(θ)
∫f(z|θ)K(d(η(z),η(y)))dz∫
π(θ)f(z|θ)K(d(η(z),η(y)))dzdθ
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 14 / 41
![Page 58: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/58.jpg)
Methodological aspects of ABCTwo views of the ABC approximation
=⇒ Wilkinson (2013) SAGMB shows that ABC is exact but fora different model to that intended
=⇒ Blum (2010) JASA emphasizes that ABC is a kernelsmoothing approximation of the likelihood function
πε(θ|y) =
∫π(θ)f(z|θ)I(z ∈ Aε,y)dz∫Aε,y×Θ π(θ)f(z|θ)dzdθ
=π(θ)
∫f(z|θ)K(d(η(z),η(y)))dz∫
π(θ)f(z|θ)K(d(η(z),η(y)))dzdθ
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 14 / 41
![Page 59: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/59.jpg)
Methodological aspects of ABCMore efficient algorithms
Simulate all the θ’s particles using the prior distribution
=⇒ very inefficient
various sequential Monte Carlo algorithms have been con-structed as an alternative
Sisson et al. (2007) PNASBeaumont, Cornuet, Marin and Robert (2009) BiometrikaDel Moral et al. (2012) Statistics and ComputingMarin, Pudlo and Sedki (2012) IEEE Proceedings of WSCFilippi et al. (2013) SAGMB
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 15 / 41
![Page 60: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/60.jpg)
Methodological aspects of ABCMore efficient algorithms
Simulate all the θ’s particles using the prior distribution
=⇒ very inefficient
various sequential Monte Carlo algorithms have been con-structed as an alternative
Sisson et al. (2007) PNASBeaumont, Cornuet, Marin and Robert (2009) BiometrikaDel Moral et al. (2012) Statistics and ComputingMarin, Pudlo and Sedki (2012) IEEE Proceedings of WSCFilippi et al. (2013) SAGMB
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 15 / 41
![Page 61: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/61.jpg)
Methodological aspects of ABCMore efficient algorithms
Simulate all the θ’s particles using the prior distribution
=⇒ very inefficient
various sequential Monte Carlo algorithms have been con-structed as an alternative
Sisson et al. (2007) PNASBeaumont, Cornuet, Marin and Robert (2009) BiometrikaDel Moral et al. (2012) Statistics and ComputingMarin, Pudlo and Sedki (2012) IEEE Proceedings of WSCFilippi et al. (2013) SAGMB
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 15 / 41
![Page 62: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/62.jpg)
Methodological aspects of ABCMore efficient algorithms
Simulate all the θ’s particles using the prior distribution
=⇒ very inefficient
various sequential Monte Carlo algorithms have been con-structed as an alternative
Sisson et al. (2007) PNASBeaumont, Cornuet, Marin and Robert (2009) BiometrikaDel Moral et al. (2012) Statistics and ComputingMarin, Pudlo and Sedki (2012) IEEE Proceedings of WSCFilippi et al. (2013) SAGMB
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 15 / 41
![Page 63: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/63.jpg)
Methodological aspects of ABCMore efficient algorithms
Simulate all the θ’s particles using the prior distribution
=⇒ very inefficient
various sequential Monte Carlo algorithms have been con-structed as an alternative
Sisson et al. (2007) PNASBeaumont, Cornuet, Marin and Robert (2009) BiometrikaDel Moral et al. (2012) Statistics and ComputingMarin, Pudlo and Sedki (2012) IEEE Proceedings of WSCFilippi et al. (2013) SAGMB
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 15 / 41
![Page 64: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/64.jpg)
Methodological aspects of ABCMore efficient algorithms
The key idea is to decompose the difficult problem of sam-pling from πε(θ, z|y) into a series of simpler subproblems
Time 0 sampling from πε0(θ, z|y) with large ε0Then simulating from an increasing difficult sequence of targetdistribution πεt(θ, z|y) that is εt < εt−1
Likelihood free MCMC sampler Majoram et al. (2003) PNAS
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 16 / 41
![Page 65: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/65.jpg)
Methodological aspects of ABCMore efficient algorithms
The key idea is to decompose the difficult problem of sam-pling from πε(θ, z|y) into a series of simpler subproblems
Time 0 sampling from πε0(θ, z|y) with large ε0
Then simulating from an increasing difficult sequence of targetdistribution πεt(θ, z|y) that is εt < εt−1
Likelihood free MCMC sampler Majoram et al. (2003) PNAS
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 16 / 41
![Page 66: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/66.jpg)
Methodological aspects of ABCMore efficient algorithms
The key idea is to decompose the difficult problem of sam-pling from πε(θ, z|y) into a series of simpler subproblems
Time 0 sampling from πε0(θ, z|y) with large ε0Then simulating from an increasing difficult sequence of targetdistribution πεt(θ, z|y) that is εt < εt−1
Likelihood free MCMC sampler Majoram et al. (2003) PNAS
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 16 / 41
![Page 67: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/67.jpg)
Methodological aspects of ABCMore efficient algorithms
The key idea is to decompose the difficult problem of sam-pling from πε(θ, z|y) into a series of simpler subproblems
Time 0 sampling from πε0(θ, z|y) with large ε0Then simulating from an increasing difficult sequence of targetdistribution πεt(θ, z|y) that is εt < εt−1
Likelihood free MCMC sampler Majoram et al. (2003) PNAS
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 16 / 41
![Page 68: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/68.jpg)
Methodological aspects of ABCRegression adjustments
Beaumont et al. (2002) Geneticslocal linear regression adjustment of the parameter values
Blum and Francois (2010) Statistics and Computingheteroscedastic models, feed-forward neural networks
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 17 / 41
![Page 69: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/69.jpg)
Methodological aspects of ABCRegression adjustments
Beaumont et al. (2002) Geneticslocal linear regression adjustment of the parameter values
Blum and Francois (2010) Statistics and Computingheteroscedastic models, feed-forward neural networks
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 17 / 41
![Page 70: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/70.jpg)
Methodological aspects of ABCRegression adjustments
Beaumont et al. (2002) Geneticslocal linear regression adjustment of the parameter values
Blum and Francois (2010) Statistics and Computingheteroscedastic models, feed-forward neural networks
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 17 / 41
![Page 71: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/71.jpg)
Methodological aspects of ABCSummary statistics
Best subset selectionI Joyce and Marjoram (2008) SAGMB, τ-sufficiencyI Nunes and Balding (2010) SAGMB, entropy
ProjectionI Fearnhead and Prangle (2012) JRSS B introduce
semi-automatic ABC
Regularization techniquesI Blum, Nunes, Prangle and Fearnhead (2013) Statistical
Science use ridge regressionI Saulnier, Gascuel, Alizon (2017) Plos Computational
Biology use LASSO
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 18 / 41
![Page 72: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/72.jpg)
Methodological aspects of ABCSummary statistics
Best subset selectionI Joyce and Marjoram (2008) SAGMB, τ-sufficiencyI Nunes and Balding (2010) SAGMB, entropy
ProjectionI Fearnhead and Prangle (2012) JRSS B introduce
semi-automatic ABC
Regularization techniquesI Blum, Nunes, Prangle and Fearnhead (2013) Statistical
Science use ridge regressionI Saulnier, Gascuel, Alizon (2017) Plos Computational
Biology use LASSO
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 18 / 41
![Page 73: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/73.jpg)
Methodological aspects of ABCSummary statistics
Best subset selectionI Joyce and Marjoram (2008) SAGMB, τ-sufficiencyI Nunes and Balding (2010) SAGMB, entropy
ProjectionI Fearnhead and Prangle (2012) JRSS B introduce
semi-automatic ABC
Regularization techniquesI Blum, Nunes, Prangle and Fearnhead (2013) Statistical
Science use ridge regressionI Saulnier, Gascuel, Alizon (2017) Plos Computational
Biology use LASSO
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 18 / 41
![Page 74: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/74.jpg)
Methodological aspects of ABCSummary statistics
Best subset selectionI Joyce and Marjoram (2008) SAGMB, τ-sufficiencyI Nunes and Balding (2010) SAGMB, entropy
ProjectionI Fearnhead and Prangle (2012) JRSS B introduce
semi-automatic ABC
Regularization techniquesI Blum, Nunes, Prangle and Fearnhead (2013) Statistical
Science use ridge regressionI Saulnier, Gascuel, Alizon (2017) Plos Computational
Biology use LASSO
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 18 / 41
![Page 75: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/75.jpg)
Methodological aspects of ABCABC model choice procedure
1) For i = 1, . . . ,M
a) Generate mi from the prior π(M = m)b) Generate θ ′mi
from the prior πmi(·)
c) Generate z from the model fmi(·|θ ′mi
)d) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the mi’s that correspond to the N-smallestdistances
N = bαMc
A k-NN approximation of the posterior probabilities
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 19 / 41
![Page 76: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/76.jpg)
Methodological aspects of ABCABC model choice procedure
1) For i = 1, . . . ,M
a) Generate mi from the prior π(M = m)b) Generate θ ′mi
from the prior πmi(·)
c) Generate z from the model fmi(·|θ ′mi
)d) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the mi’s that correspond to the N-smallestdistances
N = bαMc
A k-NN approximation of the posterior probabilities
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 19 / 41
![Page 77: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/77.jpg)
Methodological aspects of ABCABC model choice procedure
1) For i = 1, . . . ,Ma) Generate mi from the prior π(M = m)
b) Generate θ ′mifrom the prior πmi
(·)c) Generate z from the model fmi
(·|θ ′mi)
d) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the mi’s that correspond to the N-smallestdistances
N = bαMc
A k-NN approximation of the posterior probabilities
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 19 / 41
![Page 78: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/78.jpg)
Methodological aspects of ABCABC model choice procedure
1) For i = 1, . . . ,Ma) Generate mi from the prior π(M = m)b) Generate θ ′mi
from the prior πmi(·)
c) Generate z from the model fmi(·|θ ′mi
)d) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the mi’s that correspond to the N-smallestdistances
N = bαMc
A k-NN approximation of the posterior probabilities
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 19 / 41
![Page 79: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/79.jpg)
Methodological aspects of ABCABC model choice procedure
1) For i = 1, . . . ,Ma) Generate mi from the prior π(M = m)b) Generate θ ′mi
from the prior πmi(·)
c) Generate z from the model fmi(·|θ ′mi
)
d) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the mi’s that correspond to the N-smallestdistances
N = bαMc
A k-NN approximation of the posterior probabilities
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 19 / 41
![Page 80: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/80.jpg)
Methodological aspects of ABCABC model choice procedure
1) For i = 1, . . . ,Ma) Generate mi from the prior π(M = m)b) Generate θ ′mi
from the prior πmi(·)
c) Generate z from the model fmi(·|θ ′mi
)d) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the mi’s that correspond to the N-smallestdistances
N = bαMc
A k-NN approximation of the posterior probabilities
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 19 / 41
![Page 81: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/81.jpg)
Methodological aspects of ABCABC model choice procedure
1) For i = 1, . . . ,Ma) Generate mi from the prior π(M = m)b) Generate θ ′mi
from the prior πmi(·)
c) Generate z from the model fmi(·|θ ′mi
)d) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the mi’s that correspond to the N-smallestdistances
N = bαMc
A k-NN approximation of the posterior probabilities
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 19 / 41
![Page 82: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/82.jpg)
Methodological aspects of ABCABC model choice procedure
1) For i = 1, . . . ,Ma) Generate mi from the prior π(M = m)b) Generate θ ′mi
from the prior πmi(·)
c) Generate z from the model fmi(·|θ ′mi
)d) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the mi’s that correspond to the N-smallestdistances
N = bαMc
A k-NN approximation of the posterior probabilities
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 19 / 41
![Page 83: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/83.jpg)
Methodological aspects of ABCABC model choice procedure
1) For i = 1, . . . ,Ma) Generate mi from the prior π(M = m)b) Generate θ ′mi
from the prior πmi(·)
c) Generate z from the model fmi(·|θ ′mi
)d) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the mi’s that correspond to the N-smallestdistances
N = bαMc
A k-NN approximation of the posterior probabilities
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 19 / 41
![Page 84: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/84.jpg)
Methodological aspects of ABCABC model choice procedure
1) For i = 1, . . . ,Ma) Generate mi from the prior π(M = m)b) Generate θ ′mi
from the prior πmi(·)
c) Generate z from the model fmi(·|θ ′mi
)d) Calculate di = d(η(z),η(y))
2) Order the distances d(1), . . . ,d(M)
3) Return the mi’s that correspond to the N-smallestdistances
N = bαMc
A k-NN approximation of the posterior probabilities
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 19 / 41
![Page 85: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/85.jpg)
Methodological aspects of ABCABC model choice procedure
If η(y) is a sufficient statistics for the model choice problem, thiscan work pretty well
ABC likelihood-free methods for model choice in Gibbsrandom fields Grelaud, Robert, Marin, Rodolphe and Taly(2009) Bayesian Analysis
If not...
Lack of confidence in approximate Bayesian computationmodel choice Robert, Cornuet, Marin, Pillai (2011) PNAS
Relevant statistics for Bayesian model choice Marin, Pillai,Robert, Rousseau (2014) JRSS B
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 20 / 41
![Page 86: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/86.jpg)
Methodological aspects of ABCABC model choice procedure
If η(y) is a sufficient statistics for the model choice problem, thiscan work pretty well
ABC likelihood-free methods for model choice in Gibbsrandom fields Grelaud, Robert, Marin, Rodolphe and Taly(2009) Bayesian Analysis
If not...
Lack of confidence in approximate Bayesian computationmodel choice Robert, Cornuet, Marin, Pillai (2011) PNAS
Relevant statistics for Bayesian model choice Marin, Pillai,Robert, Rousseau (2014) JRSS B
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 20 / 41
![Page 87: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/87.jpg)
Methodological aspects of ABCABC model choice procedure
If η(y) is a sufficient statistics for the model choice problem, thiscan work pretty well
ABC likelihood-free methods for model choice in Gibbsrandom fields Grelaud, Robert, Marin, Rodolphe and Taly(2009) Bayesian Analysis
If not...
Lack of confidence in approximate Bayesian computationmodel choice Robert, Cornuet, Marin, Pillai (2011) PNAS
Relevant statistics for Bayesian model choice Marin, Pillai,Robert, Rousseau (2014) JRSS B
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 20 / 41
![Page 88: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/88.jpg)
Methodological aspects of ABCABC model choice procedure
If η(y) is a sufficient statistics for the model choice problem, thiscan work pretty well
ABC likelihood-free methods for model choice in Gibbsrandom fields Grelaud, Robert, Marin, Rodolphe and Taly(2009) Bayesian Analysis
If not...
Lack of confidence in approximate Bayesian computationmodel choice Robert, Cornuet, Marin, Pillai (2011) PNAS
Relevant statistics for Bayesian model choice Marin, Pillai,Robert, Rousseau (2014) JRSS B
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 20 / 41
![Page 89: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/89.jpg)
Methodological aspects of ABCABC model choice procedure
If η(y) is a sufficient statistics for the model choice problem, thiscan work pretty well
ABC likelihood-free methods for model choice in Gibbsrandom fields Grelaud, Robert, Marin, Rodolphe and Taly(2009) Bayesian Analysis
If not...
Lack of confidence in approximate Bayesian computationmodel choice Robert, Cornuet, Marin, Pillai (2011) PNAS
Relevant statistics for Bayesian model choice Marin, Pillai,Robert, Rousseau (2014) JRSS B
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 20 / 41
![Page 90: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/90.jpg)
Methodological aspects of ABCABC model choice procedure
We investigate some ABC model choice techniques that useothers machine learning procedures
Estimation of demo-genetic model probabilities with Ap-proximate Bayesian Computation using linear discriminantanalysis on summary statistics Estoup, Lombaert, Marin,Guillemaud, Pudlo, Robert, Cornuet (2012) Molecular Ecol-ogy
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 21 / 41
![Page 91: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/91.jpg)
Methodological aspects of ABCABC model choice procedure
We investigate some ABC model choice techniques that useothers machine learning procedures
Estimation of demo-genetic model probabilities with Ap-proximate Bayesian Computation using linear discriminantanalysis on summary statistics Estoup, Lombaert, Marin,Guillemaud, Pudlo, Robert, Cornuet (2012) Molecular Ecol-ogy
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 21 / 41
![Page 92: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/92.jpg)
Methodological aspects of ABCSofwares
abc R package several ABC algorithms for performing parame-ter estimation and model selection
abctools R package tuning ABC analyseshttps://journal.r-project.org/archive/2015-2/nunes-prangle.pdf
abcrf R package ABC via random forests
EasyABC R package several algorithms for performing effi-cient ABC sampling schemes, including 4 sequential samplingschemes and 3 MCMC schemes
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 22 / 41
![Page 93: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/93.jpg)
Methodological aspects of ABCSofwares
DIY-ABC software performs parameter estimation and modelselection for population genetics models
ABC-SysBio python package parameter inference and modelselection for dynamical systems
ABCtoolbox programs various ABC algorithms including rejec-tion sampling, MCMC without likelihood, a particle-based sam-pler, and ABC-GLM
PopABC software package for inference of the pattern of de-mographic divergence, coalescent simulation, bayesian modelchoice
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 23 / 41
![Page 94: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/94.jpg)
Methodological aspects of ABCSofwares
Infering population history with DIY ABC: a user-friedly approach Ap-proximate Bayesian Computation Cornuet, Santos, Beaumont, Robert,Marin, Balding, Guillemaud, Estoup (2008) Bioinformatics
DIYABC v2.0: a software to make Approximate Bayesian Computationinferences about population history using Single Nucleotide Polymor-phism, DNA sequence and microsatellite data Cornuet, Pudlo, Veyssier,Dehne-Garcia, Gautier, Leblois, Marin, Estoup (2014) Bioinformatics
Asian ladybugEuropean honey beedrosophila suzukiiPigmies populationsFour human populations, to studythe out-of-Africa colonization
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 24 / 41
![Page 95: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/95.jpg)
Methodological aspects of ABCSofwares
Infering population history with DIY ABC: a user-friedly approach Ap-proximate Bayesian Computation Cornuet, Santos, Beaumont, Robert,Marin, Balding, Guillemaud, Estoup (2008) Bioinformatics
DIYABC v2.0: a software to make Approximate Bayesian Computationinferences about population history using Single Nucleotide Polymor-phism, DNA sequence and microsatellite data Cornuet, Pudlo, Veyssier,Dehne-Garcia, Gautier, Leblois, Marin, Estoup (2014) Bioinformatics
Asian ladybugEuropean honey beedrosophila suzukiiPigmies populationsFour human populations, to studythe out-of-Africa colonization
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 24 / 41
![Page 96: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/96.jpg)
Methodological aspects of ABCSofwares
Infering population history with DIY ABC: a user-friedly approach Ap-proximate Bayesian Computation Cornuet, Santos, Beaumont, Robert,Marin, Balding, Guillemaud, Estoup (2008) Bioinformatics
DIYABC v2.0: a software to make Approximate Bayesian Computationinferences about population history using Single Nucleotide Polymor-phism, DNA sequence and microsatellite data Cornuet, Pudlo, Veyssier,Dehne-Garcia, Gautier, Leblois, Marin, Estoup (2014) Bioinformatics
Asian ladybugEuropean honey beedrosophila suzukiiPigmies populationsFour human populations, to studythe out-of-Africa colonization
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 24 / 41
![Page 97: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/97.jpg)
Methodological aspects of ABCSofwares
Infering population history with DIY ABC: a user-friedly approach Ap-proximate Bayesian Computation Cornuet, Santos, Beaumont, Robert,Marin, Balding, Guillemaud, Estoup (2008) Bioinformatics
DIYABC v2.0: a software to make Approximate Bayesian Computationinferences about population history using Single Nucleotide Polymor-phism, DNA sequence and microsatellite data Cornuet, Pudlo, Veyssier,Dehne-Garcia, Gautier, Leblois, Marin, Estoup (2014) Bioinformatics
Asian ladybug
European honey beedrosophila suzukiiPigmies populationsFour human populations, to studythe out-of-Africa colonization
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 24 / 41
![Page 98: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/98.jpg)
Methodological aspects of ABCSofwares
Infering population history with DIY ABC: a user-friedly approach Ap-proximate Bayesian Computation Cornuet, Santos, Beaumont, Robert,Marin, Balding, Guillemaud, Estoup (2008) Bioinformatics
DIYABC v2.0: a software to make Approximate Bayesian Computationinferences about population history using Single Nucleotide Polymor-phism, DNA sequence and microsatellite data Cornuet, Pudlo, Veyssier,Dehne-Garcia, Gautier, Leblois, Marin, Estoup (2014) Bioinformatics
Asian ladybugEuropean honey bee
drosophila suzukiiPigmies populationsFour human populations, to studythe out-of-Africa colonization
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 24 / 41
![Page 99: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/99.jpg)
Methodological aspects of ABCSofwares
Infering population history with DIY ABC: a user-friedly approach Ap-proximate Bayesian Computation Cornuet, Santos, Beaumont, Robert,Marin, Balding, Guillemaud, Estoup (2008) Bioinformatics
DIYABC v2.0: a software to make Approximate Bayesian Computationinferences about population history using Single Nucleotide Polymor-phism, DNA sequence and microsatellite data Cornuet, Pudlo, Veyssier,Dehne-Garcia, Gautier, Leblois, Marin, Estoup (2014) Bioinformatics
Asian ladybugEuropean honey beedrosophila suzukii
Pigmies populationsFour human populations, to studythe out-of-Africa colonization
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 24 / 41
![Page 100: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/100.jpg)
Methodological aspects of ABCSofwares
Infering population history with DIY ABC: a user-friedly approach Ap-proximate Bayesian Computation Cornuet, Santos, Beaumont, Robert,Marin, Balding, Guillemaud, Estoup (2008) Bioinformatics
DIYABC v2.0: a software to make Approximate Bayesian Computationinferences about population history using Single Nucleotide Polymor-phism, DNA sequence and microsatellite data Cornuet, Pudlo, Veyssier,Dehne-Garcia, Gautier, Leblois, Marin, Estoup (2014) Bioinformatics
Asian ladybugEuropean honey beedrosophila suzukiiPigmies populations
Four human populations, to studythe out-of-Africa colonization
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 24 / 41
![Page 101: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/101.jpg)
Methodological aspects of ABCSofwares
Infering population history with DIY ABC: a user-friedly approach Ap-proximate Bayesian Computation Cornuet, Santos, Beaumont, Robert,Marin, Balding, Guillemaud, Estoup (2008) Bioinformatics
DIYABC v2.0: a software to make Approximate Bayesian Computationinferences about population history using Single Nucleotide Polymor-phism, DNA sequence and microsatellite data Cornuet, Pudlo, Veyssier,Dehne-Garcia, Gautier, Leblois, Marin, Estoup (2014) Bioinformatics
Asian ladybugEuropean honey beedrosophila suzukiiPigmies populationsFour human populations, to studythe out-of-Africa colonization
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 24 / 41
![Page 102: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/102.jpg)
Methodological aspects of ABCFrontline news from population geneticists country
DIYABC (2014) paper has now around 300 citations
I simulate from the model can be very computationallyintensive, parallelizable algorithms are necessary
I likelihoods are intractable due to the strong and complexdependence structure of the model
I sequential methods are difficult to calibrate and do not givereproducible results
I post hoc adjustments are crucial but they underestimatethe amount of uncertainty
I available techniques to select the summary statistics donot give reproducible results
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 25 / 41
![Page 103: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/103.jpg)
Methodological aspects of ABCFrontline news from population geneticists country
DIYABC (2014) paper has now around 300 citations
I simulate from the model can be very computationallyintensive, parallelizable algorithms are necessary
I likelihoods are intractable due to the strong and complexdependence structure of the model
I sequential methods are difficult to calibrate and do not givereproducible results
I post hoc adjustments are crucial but they underestimatethe amount of uncertainty
I available techniques to select the summary statistics donot give reproducible results
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 25 / 41
![Page 104: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/104.jpg)
Methodological aspects of ABCFrontline news from population geneticists country
DIYABC (2014) paper has now around 300 citations
I simulate from the model can be very computationallyintensive, parallelizable algorithms are necessary
I likelihoods are intractable due to the strong and complexdependence structure of the model
I sequential methods are difficult to calibrate and do not givereproducible results
I post hoc adjustments are crucial but they underestimatethe amount of uncertainty
I available techniques to select the summary statistics donot give reproducible results
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 25 / 41
![Page 105: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/105.jpg)
Methodological aspects of ABCFrontline news from population geneticists country
DIYABC (2014) paper has now around 300 citations
I simulate from the model can be very computationallyintensive, parallelizable algorithms are necessary
I likelihoods are intractable due to the strong and complexdependence structure of the model
I sequential methods are difficult to calibrate and do not givereproducible results
I post hoc adjustments are crucial but they underestimatethe amount of uncertainty
I available techniques to select the summary statistics donot give reproducible results
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 25 / 41
![Page 106: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/106.jpg)
Methodological aspects of ABCFrontline news from population geneticists country
DIYABC (2014) paper has now around 300 citations
I simulate from the model can be very computationallyintensive, parallelizable algorithms are necessary
I likelihoods are intractable due to the strong and complexdependence structure of the model
I sequential methods are difficult to calibrate and do not givereproducible results
I post hoc adjustments are crucial but they underestimatethe amount of uncertainty
I available techniques to select the summary statistics donot give reproducible results
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 25 / 41
![Page 107: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/107.jpg)
Methodological aspects of ABCFrontline news from population geneticists country
DIYABC (2014) paper has now around 300 citations
I simulate from the model can be very computationallyintensive, parallelizable algorithms are necessary
I likelihoods are intractable due to the strong and complexdependence structure of the model
I sequential methods are difficult to calibrate and do not givereproducible results
I post hoc adjustments are crucial but they underestimatethe amount of uncertainty
I available techniques to select the summary statistics donot give reproducible results
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 25 / 41
![Page 108: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/108.jpg)
Methodological aspects of ABCFrontline news from population geneticists country
DIYABC (2014) paper has now around 300 citations
I simulate from the model can be very computationallyintensive, parallelizable algorithms are necessary
I likelihoods are intractable due to the strong and complexdependence structure of the model
I sequential methods are difficult to calibrate and do not givereproducible results
I post hoc adjustments are crucial but they underestimatethe amount of uncertainty
I available techniques to select the summary statistics donot give reproducible results
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 25 / 41
![Page 109: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/109.jpg)
Methodological aspects of ABCFrontline news from population geneticists country
Despite all these works, two major difficulties
I to ensure reliability of the method, the number ofsimulations should be large
I choice of the summaries statistics is still a problem
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 26 / 41
![Page 110: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/110.jpg)
Methodological aspects of ABCFrontline news from population geneticists country
Despite all these works, two major difficulties
I to ensure reliability of the method, the number ofsimulations should be large
I choice of the summaries statistics is still a problem
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 26 / 41
![Page 111: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/111.jpg)
Methodological aspects of ABCFrontline news from population geneticists country
Despite all these works, two major difficulties
I to ensure reliability of the method, the number ofsimulations should be large
I choice of the summaries statistics is still a problem
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 26 / 41
![Page 112: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/112.jpg)
Methodological aspects of ABCUse modern machine learning tools
Exploiting a large number of summary statistics is not an issuefor some machine learning methods
Idea: learn on a huge reference table using random forests
Some theoretical guarantees for sparse problems
Analysis of a random forest modelBiau (2012) JMLR
Consistency of random forestsScornet, Biau, Vert (2015) The Annals of Statistics
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 27 / 41
![Page 113: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/113.jpg)
Methodological aspects of ABCUse modern machine learning tools
Exploiting a large number of summary statistics is not an issuefor some machine learning methods
Idea: learn on a huge reference table using random forests
Some theoretical guarantees for sparse problems
Analysis of a random forest modelBiau (2012) JMLR
Consistency of random forestsScornet, Biau, Vert (2015) The Annals of Statistics
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 27 / 41
![Page 114: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/114.jpg)
Methodological aspects of ABCUse modern machine learning tools
Exploiting a large number of summary statistics is not an issuefor some machine learning methods
Idea: learn on a huge reference table using random forests
Some theoretical guarantees for sparse problems
Analysis of a random forest modelBiau (2012) JMLR
Consistency of random forestsScornet, Biau, Vert (2015) The Annals of Statistics
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 27 / 41
![Page 115: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/115.jpg)
Methodological aspects of ABCUse modern machine learning tools
Exploiting a large number of summary statistics is not an issuefor some machine learning methods
Idea: learn on a huge reference table using random forests
Some theoretical guarantees for sparse problems
Analysis of a random forest modelBiau (2012) JMLR
Consistency of random forestsScornet, Biau, Vert (2015) The Annals of Statistics
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 27 / 41
![Page 116: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/116.jpg)
Methodological aspects of ABCUse modern machine learning tools
Exploiting a large number of summary statistics is not an issuefor some machine learning methods
Idea: learn on a huge reference table using random forests
Some theoretical guarantees for sparse problems
Analysis of a random forest modelBiau (2012) JMLR
Consistency of random forestsScornet, Biau, Vert (2015) The Annals of Statistics
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 27 / 41
![Page 117: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/117.jpg)
Methodological aspects of ABCUse modern machine learning tools
Exploiting a large number of summary statistics is not an issuefor some machine learning methods
Idea: learn on a huge reference table using random forests
Some theoretical guarantees for sparse problems
Analysis of a random forest modelBiau (2012) JMLR
Consistency of random forestsScornet, Biau, Vert (2015) The Annals of Statistics
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 27 / 41
![Page 118: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/118.jpg)
Methodological aspects of ABCUse modern machine learning tools
This work stands at the interface between Bayesian inferenceand machine learning techniques
As an alternative, Papamakarios and Murray (2016) propose toapproximate the whole posterior distribution by using MixtureDensity Networks (MDN, Bishop, 1994)
Fast e-free Inference of Simulation Models with BayesianConditional Density EstimationPapamakarios and Murray (2016) NIPS
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 28 / 41
![Page 119: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/119.jpg)
Methodological aspects of ABCUse modern machine learning tools
This work stands at the interface between Bayesian inferenceand machine learning techniques
As an alternative, Papamakarios and Murray (2016) propose toapproximate the whole posterior distribution by using MixtureDensity Networks (MDN, Bishop, 1994)
Fast e-free Inference of Simulation Models with BayesianConditional Density EstimationPapamakarios and Murray (2016) NIPS
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 28 / 41
![Page 120: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/120.jpg)
Methodological aspects of ABCUse modern machine learning tools
This work stands at the interface between Bayesian inferenceand machine learning techniques
As an alternative, Papamakarios and Murray (2016) propose toapproximate the whole posterior distribution by using MixtureDensity Networks (MDN, Bishop, 1994)
Fast e-free Inference of Simulation Models with BayesianConditional Density EstimationPapamakarios and Murray (2016) NIPS
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 28 / 41
![Page 121: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/121.jpg)
Methodological aspects of ABCUse modern machine learning tools
The MDN strategy consists in using Gaussian mixture modelswith parameters calibrated thanks to neural networks
Idea: iteratively learn an efficient proposal prior (approximatingthe posterior distribution), then to use this proposal to train theposterior, both steps making use of MDN
The number of mixture components and the number of hid-den layers of the networks require calibration
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 29 / 41
![Page 122: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/122.jpg)
Methodological aspects of ABCUse modern machine learning tools
The MDN strategy consists in using Gaussian mixture modelswith parameters calibrated thanks to neural networks
Idea: iteratively learn an efficient proposal prior (approximatingthe posterior distribution), then to use this proposal to train theposterior, both steps making use of MDN
The number of mixture components and the number of hid-den layers of the networks require calibration
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 29 / 41
![Page 123: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/123.jpg)
Methodological aspects of ABCUse modern machine learning tools
The MDN strategy consists in using Gaussian mixture modelswith parameters calibrated thanks to neural networks
Idea: iteratively learn an efficient proposal prior (approximatingthe posterior distribution), then to use this proposal to train theposterior, both steps making use of MDN
The number of mixture components and the number of hid-den layers of the networks require calibration
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 29 / 41
![Page 124: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/124.jpg)
Methodological aspects of ABCUse modern machine learning tools
Deep Learning for Population Genetic InferenceSheehan and Song (2016) PLOS Computational Biology
Deep learning makes use of multilayer neural networks to learna feature-based function from the input (hundreds of correlatedsummary statistics) to the output (population genetic parametersof interest).
Unsupervised pretraining using autoencoders very inter-esting, but requires a lot of calibration
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 30 / 41
![Page 125: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/125.jpg)
Methodological aspects of ABCUse modern machine learning tools
Deep Learning for Population Genetic InferenceSheehan and Song (2016) PLOS Computational Biology
Deep learning makes use of multilayer neural networks to learna feature-based function from the input (hundreds of correlatedsummary statistics) to the output (population genetic parametersof interest).
Unsupervised pretraining using autoencoders very inter-esting, but requires a lot of calibration
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 30 / 41
![Page 126: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/126.jpg)
Methodological aspects of ABCUse modern machine learning tools
Deep Learning for Population Genetic InferenceSheehan and Song (2016) PLOS Computational Biology
Deep learning makes use of multilayer neural networks to learna feature-based function from the input (hundreds of correlatedsummary statistics) to the output (population genetic parametersof interest).
Unsupervised pretraining using autoencoders very inter-esting, but requires a lot of calibration
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 30 / 41
![Page 127: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/127.jpg)
ABC random forestsModel choice
Reliable ABC model choice via random forests Pudlo, Marin, Estoup,Cornuet, Gauthier and Robert (2016) Bioinformatics
Input ABC reference table involving model index and summarystatistics, table used as learning set
possibly large collection of summary statistics: from scien-tific theory input to machine-learning alternatives
For i = 1, . . . ,M
a) Generate mi from the prior π(M = m)b) Generate θ ′mi
from the prior πmi(·)
c) Generate z from the model fmi(·|θ ′mi
)d) Calculate xi = η(zi)
Output a random forest classifier to infer model indexes m(η(y))
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 31 / 41
![Page 128: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/128.jpg)
ABC random forestsModel choice
Reliable ABC model choice via random forests Pudlo, Marin, Estoup,Cornuet, Gauthier and Robert (2016) Bioinformatics
Input ABC reference table involving model index and summarystatistics, table used as learning set
possibly large collection of summary statistics: from scien-tific theory input to machine-learning alternatives
For i = 1, . . . ,M
a) Generate mi from the prior π(M = m)b) Generate θ ′mi
from the prior πmi(·)
c) Generate z from the model fmi(·|θ ′mi
)d) Calculate xi = η(zi)
Output a random forest classifier to infer model indexes m(η(y))
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 31 / 41
![Page 129: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/129.jpg)
ABC random forestsModel choice
Reliable ABC model choice via random forests Pudlo, Marin, Estoup,Cornuet, Gauthier and Robert (2016) Bioinformatics
Input ABC reference table involving model index and summarystatistics, table used as learning set
possibly large collection of summary statistics: from scien-tific theory input to machine-learning alternatives
For i = 1, . . . ,M
a) Generate mi from the prior π(M = m)b) Generate θ ′mi
from the prior πmi(·)
c) Generate z from the model fmi(·|θ ′mi
)d) Calculate xi = η(zi)
Output a random forest classifier to infer model indexes m(η(y))
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 31 / 41
![Page 130: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/130.jpg)
ABC random forestsModel choice
Reliable ABC model choice via random forests Pudlo, Marin, Estoup,Cornuet, Gauthier and Robert (2016) Bioinformatics
Input ABC reference table involving model index and summarystatistics, table used as learning set
possibly large collection of summary statistics: from scien-tific theory input to machine-learning alternatives
For i = 1, . . . ,M
a) Generate mi from the prior π(M = m)b) Generate θ ′mi
from the prior πmi(·)
c) Generate z from the model fmi(·|θ ′mi
)d) Calculate xi = η(zi)
Output a random forest classifier to infer model indexes m(η(y))
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 31 / 41
![Page 131: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/131.jpg)
ABC random forestsModel choice
Reliable ABC model choice via random forests Pudlo, Marin, Estoup,Cornuet, Gauthier and Robert (2016) Bioinformatics
Input ABC reference table involving model index and summarystatistics, table used as learning set
possibly large collection of summary statistics: from scien-tific theory input to machine-learning alternatives
For i = 1, . . . ,Ma) Generate mi from the prior π(M = m)b) Generate θ ′mi
from the prior πmi(·)
c) Generate z from the model fmi(·|θ ′mi
)d) Calculate xi = η(zi)
Output a random forest classifier to infer model indexes m(η(y))
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 31 / 41
![Page 132: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/132.jpg)
ABC random forestsModel choice
Reliable ABC model choice via random forests Pudlo, Marin, Estoup,Cornuet, Gauthier and Robert (2016) Bioinformatics
Input ABC reference table involving model index and summarystatistics, table used as learning set
possibly large collection of summary statistics: from scien-tific theory input to machine-learning alternatives
For i = 1, . . . ,Ma) Generate mi from the prior π(M = m)b) Generate θ ′mi
from the prior πmi(·)
c) Generate z from the model fmi(·|θ ′mi
)d) Calculate xi = η(zi)
Output a random forest classifier to infer model indexes m(η(y))
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 31 / 41
![Page 133: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/133.jpg)
ABC random forestsModel choice
Random forest predicts a MAP model index, from the observeddataset
the predictor provided by the forest is good enough to select themost likely model
but not to derive directly the associated posterior probabil-ities
frequency of trees associated with majority model is noproper substitute to the true posterior probability
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 32 / 41
![Page 134: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/134.jpg)
ABC random forestsModel choice
Random forest predicts a MAP model index, from the observeddataset
the predictor provided by the forest is good enough to select themost likely model
but not to derive directly the associated posterior probabil-ities
frequency of trees associated with majority model is noproper substitute to the true posterior probability
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 32 / 41
![Page 135: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/135.jpg)
ABC random forestsModel choice
Random forest predicts a MAP model index, from the observeddataset
the predictor provided by the forest is good enough to select themost likely model
but not to derive directly the associated posterior probabil-ities
frequency of trees associated with majority model is noproper substitute to the true posterior probability
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 32 / 41
![Page 136: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/136.jpg)
ABC random forestsModel choice
Random forest predicts a MAP model index, from the observeddataset
the predictor provided by the forest is good enough to select themost likely model
but not to derive directly the associated posterior probabil-ities
frequency of trees associated with majority model is noproper substitute to the true posterior probability
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 32 / 41
![Page 137: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/137.jpg)
ABC random forestsModel choice
Estimate of the posterior probability of the selected model
P[M = m(η(y))|η(y)]
random comes from M (bayesian)!
P[M = m(η(y))|η(y)] = 1 − E[I(M , m(η(y)))|η(y)
]
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 33 / 41
![Page 138: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/138.jpg)
ABC random forestsModel choice
Estimate of the posterior probability of the selected model
P[M = m(η(y))|η(y)]
random comes from M (bayesian)!
P[M = m(η(y))|η(y)] = 1 − E[I(M , m(η(y)))|η(y)
]
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 33 / 41
![Page 139: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/139.jpg)
ABC random forestsModel choice
A second random forest in regression
1) compute the value of I(M , m(η(z)) for the trainedrandom forest m and for all terms in the ABC referencetable using the out-of-bag classifiers
2) train a RF regression and get E[I(M , m(η(z)))|η(z)]
]3) returnP[M = m(η(y))|η(y)] = 1 − E
[I(M , m(η(z)))|η(z)]
]on same reference table out-of-bag magic trick avoid over-fitting!
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 34 / 41
![Page 140: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/140.jpg)
ABC random forestsModel choice
A second random forest in regression
1) compute the value of I(M , m(η(z)) for the trainedrandom forest m and for all terms in the ABC referencetable using the out-of-bag classifiers
2) train a RF regression and get E[I(M , m(η(z)))|η(z)]
]3) returnP[M = m(η(y))|η(y)] = 1 − E
[I(M , m(η(z)))|η(z)]
]on same reference table out-of-bag magic trick avoid over-fitting!
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 34 / 41
![Page 141: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/141.jpg)
ABC random forestsModel choice
A second random forest in regression
1) compute the value of I(M , m(η(z)) for the trainedrandom forest m and for all terms in the ABC referencetable using the out-of-bag classifiers
2) train a RF regression and get E[I(M , m(η(z)))|η(z)]
]
3) returnP[M = m(η(y))|η(y)] = 1 − E
[I(M , m(η(z)))|η(z)]
]on same reference table out-of-bag magic trick avoid over-fitting!
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 34 / 41
![Page 142: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/142.jpg)
ABC random forestsModel choice
A second random forest in regression
1) compute the value of I(M , m(η(z)) for the trainedrandom forest m and for all terms in the ABC referencetable using the out-of-bag classifiers
2) train a RF regression and get E[I(M , m(η(z)))|η(z)]
]3) returnP[M = m(η(y))|η(y)] = 1 − E
[I(M , m(η(z)))|η(z)]
]
on same reference table out-of-bag magic trick avoid over-fitting!
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 34 / 41
![Page 143: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/143.jpg)
ABC random forestsModel choice
A second random forest in regression
1) compute the value of I(M , m(η(z)) for the trainedrandom forest m and for all terms in the ABC referencetable using the out-of-bag classifiers
2) train a RF regression and get E[I(M , m(η(z)))|η(z)]
]3) returnP[M = m(η(y))|η(y)] = 1 − E
[I(M , m(η(z)))|η(z)]
]on same reference table out-of-bag magic trick avoid over-fitting!
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 34 / 41
![Page 144: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/144.jpg)
ABC random forestsParameter inference
ABC random forests for Bayesian parameter inference Raynal, Marin,Pudlo, Ribatet, Robert and Estoup (2017) Preprint reviewed and recom-mended by Peer Community In Evolutionary Biology
Input ABC reference table involving parameters values andsummary statistics, table used as learning set
For i = 1, . . . ,M
a) Generate θi from the prior π(·)b) Generate zi from the model f(·|θi)c) Calculate xi = η(zi)
Output some regression RF predictors to infer posterior expec-tations, quantiles, variances and covariances
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 35 / 41
![Page 145: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/145.jpg)
ABC random forestsParameter inference
ABC random forests for Bayesian parameter inference Raynal, Marin,Pudlo, Ribatet, Robert and Estoup (2017) Preprint reviewed and recom-mended by Peer Community In Evolutionary Biology
Input ABC reference table involving parameters values andsummary statistics, table used as learning set
For i = 1, . . . ,M
a) Generate θi from the prior π(·)b) Generate zi from the model f(·|θi)c) Calculate xi = η(zi)
Output some regression RF predictors to infer posterior expec-tations, quantiles, variances and covariances
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 35 / 41
![Page 146: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/146.jpg)
ABC random forestsParameter inference
ABC random forests for Bayesian parameter inference Raynal, Marin,Pudlo, Ribatet, Robert and Estoup (2017) Preprint reviewed and recom-mended by Peer Community In Evolutionary Biology
Input ABC reference table involving parameters values andsummary statistics, table used as learning set
For i = 1, . . . ,M
a) Generate θi from the prior π(·)b) Generate zi from the model f(·|θi)c) Calculate xi = η(zi)
Output some regression RF predictors to infer posterior expec-tations, quantiles, variances and covariances
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 35 / 41
![Page 147: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/147.jpg)
ABC random forestsParameter inference
ABC random forests for Bayesian parameter inference Raynal, Marin,Pudlo, Ribatet, Robert and Estoup (2017) Preprint reviewed and recom-mended by Peer Community In Evolutionary Biology
Input ABC reference table involving parameters values andsummary statistics, table used as learning set
For i = 1, . . . ,Ma) Generate θi from the prior π(·)b) Generate zi from the model f(·|θi)c) Calculate xi = η(zi)
Output some regression RF predictors to infer posterior expec-tations, quantiles, variances and covariances
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 35 / 41
![Page 148: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/148.jpg)
ABC random forestsParameter inference
ABC random forests for Bayesian parameter inference Raynal, Marin,Pudlo, Ribatet, Robert and Estoup (2017) Preprint reviewed and recom-mended by Peer Community In Evolutionary Biology
Input ABC reference table involving parameters values andsummary statistics, table used as learning set
For i = 1, . . . ,Ma) Generate θi from the prior π(·)b) Generate zi from the model f(·|θi)c) Calculate xi = η(zi)
Output some regression RF predictors to infer posterior expec-tations, quantiles, variances and covariances
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 35 / 41
![Page 149: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/149.jpg)
ABC random forestsParameter inference
Expectations Construct d regression RF, one per dimension
Quantiles very nice trick to estimate the cdf, no new forestQuantile Regression Forests Meinshausen (2006) JMLR
Variances use of a out-of-bag trick, no new forest
Covariances new forests for which the responses variables arethe products of out-of-bag errors
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 36 / 41
![Page 150: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/150.jpg)
ABC random forestsParameter inference
Expectations Construct d regression RF, one per dimension
Quantiles very nice trick to estimate the cdf, no new forestQuantile Regression Forests Meinshausen (2006) JMLR
Variances use of a out-of-bag trick, no new forest
Covariances new forests for which the responses variables arethe products of out-of-bag errors
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 36 / 41
![Page 151: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/151.jpg)
ABC random forestsParameter inference
Expectations Construct d regression RF, one per dimension
Quantiles very nice trick to estimate the cdf, no new forestQuantile Regression Forests Meinshausen (2006) JMLR
Variances use of a out-of-bag trick, no new forest
Covariances new forests for which the responses variables arethe products of out-of-bag errors
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 36 / 41
![Page 152: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/152.jpg)
ABC random forestsParameter inference
Expectations Construct d regression RF, one per dimension
Quantiles very nice trick to estimate the cdf, no new forestQuantile Regression Forests Meinshausen (2006) JMLR
Variances use of a out-of-bag trick, no new forest
Covariances new forests for which the responses variables arethe products of out-of-bag errors
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 36 / 41
![Page 153: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/153.jpg)
ABC random forestsParameter inference
We constructed forests able to estimate everywhere in the spaceof summary statistics but we are interested only in one point, theobserved dataset
construct local random forest, thesis of Louis Raynal
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 37 / 41
![Page 154: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/154.jpg)
ABC random forestsParameter inference
We constructed forests able to estimate everywhere in the spaceof summary statistics but we are interested only in one point, theobserved dataset
construct local random forest, thesis of Louis Raynal
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 37 / 41
![Page 155: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/155.jpg)
ABC and PAC-Bayes
Bayesian and PAC-Bayesian frameworks have learned fromeach other
PAC-Bayesian Theory Meets Bayesian InferenceGermain, Bach, Lacoste, Lacoste-Julien (2016) NIPS
choosing the negative log-likelihood loss function: minimizingthe PAC-Bayes bound is equivalent to maximizing the marginallikelihood
A general framework for updating belief distributions Bis-siri, Holmes and Walker (2016) JRSS B
The Safe Bayesian: Learning the Learning Rate via the Mix-ability Gap Grunwald (2012) ALT
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 38 / 41
![Page 156: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/156.jpg)
ABC and PAC-Bayes
Bayesian and PAC-Bayesian frameworks have learned fromeach other
PAC-Bayesian Theory Meets Bayesian InferenceGermain, Bach, Lacoste, Lacoste-Julien (2016) NIPS
choosing the negative log-likelihood loss function: minimizingthe PAC-Bayes bound is equivalent to maximizing the marginallikelihood
A general framework for updating belief distributions Bis-siri, Holmes and Walker (2016) JRSS B
The Safe Bayesian: Learning the Learning Rate via the Mix-ability Gap Grunwald (2012) ALT
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 38 / 41
![Page 157: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/157.jpg)
ABC and PAC-Bayes
Bayesian and PAC-Bayesian frameworks have learned fromeach other
PAC-Bayesian Theory Meets Bayesian InferenceGermain, Bach, Lacoste, Lacoste-Julien (2016) NIPS
choosing the negative log-likelihood loss function: minimizingthe PAC-Bayes bound is equivalent to maximizing the marginallikelihood
A general framework for updating belief distributions Bis-siri, Holmes and Walker (2016) JRSS B
The Safe Bayesian: Learning the Learning Rate via the Mix-ability Gap Grunwald (2012) ALT
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 38 / 41
![Page 158: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/158.jpg)
ABC and PAC-Bayes
Bayesian and PAC-Bayesian frameworks have learned fromeach other
PAC-Bayesian Theory Meets Bayesian InferenceGermain, Bach, Lacoste, Lacoste-Julien (2016) NIPS
choosing the negative log-likelihood loss function: minimizingthe PAC-Bayes bound is equivalent to maximizing the marginallikelihood
A general framework for updating belief distributions Bis-siri, Holmes and Walker (2016) JRSS B
The Safe Bayesian: Learning the Learning Rate via the Mix-ability Gap Grunwald (2012) ALT
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 38 / 41
![Page 159: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/159.jpg)
ABC and PAC-Bayes
Bayesian and PAC-Bayesian frameworks have learned fromeach other
PAC-Bayesian Theory Meets Bayesian InferenceGermain, Bach, Lacoste, Lacoste-Julien (2016) NIPS
choosing the negative log-likelihood loss function: minimizingthe PAC-Bayes bound is equivalent to maximizing the marginallikelihood
A general framework for updating belief distributions Bis-siri, Holmes and Walker (2016) JRSS B
The Safe Bayesian: Learning the Learning Rate via the Mix-ability Gap Grunwald (2012) ALT
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 38 / 41
![Page 160: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/160.jpg)
ABC and PAC-Bayes
Bayesian and PAC-Bayesian frameworks have learned fromeach other
PAC-Bayesian Theory Meets Bayesian InferenceGermain, Bach, Lacoste, Lacoste-Julien (2016) NIPS
choosing the negative log-likelihood loss function: minimizingthe PAC-Bayes bound is equivalent to maximizing the marginallikelihood
A general framework for updating belief distributions Bis-siri, Holmes and Walker (2016) JRSS B
The Safe Bayesian: Learning the Learning Rate via the Mix-ability Gap Grunwald (2012) ALT
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 38 / 41
![Page 161: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/161.jpg)
ABC and PAC-Bayes
ABC approximations are based on the existence and the use ofa generative model
antithetical with the PAC-Bayes paradigm
ABC approximations are not useful when the calculation of thelikelihood is tractable
not a good idea to use ABC as an inferential tools for thePAC-Bayes pseudo-posterior
The ABC toolbox seems unable to bring anything to thePAC-Bayesian framework
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 39 / 41
![Page 162: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/162.jpg)
ABC and PAC-Bayes
ABC approximations are based on the existence and the use ofa generative model
antithetical with the PAC-Bayes paradigm
ABC approximations are not useful when the calculation of thelikelihood is tractable
not a good idea to use ABC as an inferential tools for thePAC-Bayes pseudo-posterior
The ABC toolbox seems unable to bring anything to thePAC-Bayesian framework
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 39 / 41
![Page 163: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/163.jpg)
ABC and PAC-Bayes
ABC approximations are based on the existence and the use ofa generative model
antithetical with the PAC-Bayes paradigm
ABC approximations are not useful when the calculation of thelikelihood is tractable
not a good idea to use ABC as an inferential tools for thePAC-Bayes pseudo-posterior
The ABC toolbox seems unable to bring anything to thePAC-Bayesian framework
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 39 / 41
![Page 164: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/164.jpg)
ABC and PAC-Bayes
ABC approximations are based on the existence and the use ofa generative model
antithetical with the PAC-Bayes paradigm
ABC approximations are not useful when the calculation of thelikelihood is tractable
not a good idea to use ABC as an inferential tools for thePAC-Bayes pseudo-posterior
The ABC toolbox seems unable to bring anything to thePAC-Bayesian framework
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 39 / 41
![Page 165: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/165.jpg)
ABC and PAC-Bayes
ABC approximations are based on the existence and the use ofa generative model
antithetical with the PAC-Bayes paradigm
ABC approximations are not useful when the calculation of thelikelihood is tractable
not a good idea to use ABC as an inferential tools for thePAC-Bayes pseudo-posterior
The ABC toolbox seems unable to bring anything to thePAC-Bayesian framework
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 39 / 41
![Page 166: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/166.jpg)
ABC and PAC-Bayes
On the other hand
ABC concerns are machine learning concerns
Use PAC-Bayes learning on the ABC reference table
Contrary to the standard context
I we generate the learning setI we are only interested in one point, the observed dataset!
Probably approximate Bayesian computation: nonasymp-totic convergence of ABC under misspecification Ridgway(2017) Preprint
Convergence of the ABC posterior under model misspecificationUse of concentration inequalities, PAC-Bayesian analysis
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 40 / 41
![Page 167: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/167.jpg)
ABC and PAC-Bayes
On the other hand
ABC concerns are machine learning concerns
Use PAC-Bayes learning on the ABC reference table
Contrary to the standard context
I we generate the learning setI we are only interested in one point, the observed dataset!
Probably approximate Bayesian computation: nonasymp-totic convergence of ABC under misspecification Ridgway(2017) Preprint
Convergence of the ABC posterior under model misspecificationUse of concentration inequalities, PAC-Bayesian analysis
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 40 / 41
![Page 168: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/168.jpg)
ABC and PAC-Bayes
On the other hand
ABC concerns are machine learning concerns
Use PAC-Bayes learning on the ABC reference table
Contrary to the standard context
I we generate the learning setI we are only interested in one point, the observed dataset!
Probably approximate Bayesian computation: nonasymp-totic convergence of ABC under misspecification Ridgway(2017) Preprint
Convergence of the ABC posterior under model misspecificationUse of concentration inequalities, PAC-Bayesian analysis
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 40 / 41
![Page 169: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/169.jpg)
ABC and PAC-Bayes
On the other hand
ABC concerns are machine learning concerns
Use PAC-Bayes learning on the ABC reference table
Contrary to the standard context
I we generate the learning setI we are only interested in one point, the observed dataset!
Probably approximate Bayesian computation: nonasymp-totic convergence of ABC under misspecification Ridgway(2017) Preprint
Convergence of the ABC posterior under model misspecificationUse of concentration inequalities, PAC-Bayesian analysis
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 40 / 41
![Page 170: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/170.jpg)
ABC and PAC-Bayes
On the other hand
ABC concerns are machine learning concerns
Use PAC-Bayes learning on the ABC reference table
Contrary to the standard contextI we generate the learning set
I we are only interested in one point, the observed dataset!
Probably approximate Bayesian computation: nonasymp-totic convergence of ABC under misspecification Ridgway(2017) Preprint
Convergence of the ABC posterior under model misspecificationUse of concentration inequalities, PAC-Bayesian analysis
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 40 / 41
![Page 171: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/171.jpg)
ABC and PAC-Bayes
On the other hand
ABC concerns are machine learning concerns
Use PAC-Bayes learning on the ABC reference table
Contrary to the standard contextI we generate the learning setI we are only interested in one point, the observed dataset!
Probably approximate Bayesian computation: nonasymp-totic convergence of ABC under misspecification Ridgway(2017) Preprint
Convergence of the ABC posterior under model misspecificationUse of concentration inequalities, PAC-Bayesian analysis
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 40 / 41
![Page 172: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/172.jpg)
ABC and PAC-Bayes
On the other hand
ABC concerns are machine learning concerns
Use PAC-Bayes learning on the ABC reference table
Contrary to the standard contextI we generate the learning setI we are only interested in one point, the observed dataset!
Probably approximate Bayesian computation: nonasymp-totic convergence of ABC under misspecification Ridgway(2017) Preprint
Convergence of the ABC posterior under model misspecificationUse of concentration inequalities, PAC-Bayesian analysis
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 40 / 41
![Page 173: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/173.jpg)
ABC and PAC-Bayes
On the other hand
ABC concerns are machine learning concerns
Use PAC-Bayes learning on the ABC reference table
Contrary to the standard contextI we generate the learning setI we are only interested in one point, the observed dataset!
Probably approximate Bayesian computation: nonasymp-totic convergence of ABC under misspecification Ridgway(2017) Preprint
Convergence of the ABC posterior under model misspecificationUse of concentration inequalities, PAC-Bayesian analysis
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 40 / 41
![Page 174: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/174.jpg)
End
Yesterday with Benjamin and Pascal in a Chinese restaurant, thebill arrives... with a cake and a hidden message:
This year, take comfort in your rituals, but be open to new expe-riences
A clear sign, I should try to use some PAC-Bayesian results
Thank you very much for your attention
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 41 / 41
![Page 175: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/175.jpg)
End
Yesterday with Benjamin and Pascal in a Chinese restaurant, thebill arrives... with a cake and a hidden message:
This year, take comfort in your rituals, but be open to new expe-riences
A clear sign, I should try to use some PAC-Bayesian results
Thank you very much for your attention
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 41 / 41
![Page 176: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/176.jpg)
End
Yesterday with Benjamin and Pascal in a Chinese restaurant, thebill arrives... with a cake and a hidden message:
This year, take comfort in your rituals, but be open to new expe-riences
A clear sign, I should try to use some PAC-Bayesian results
Thank you very much for your attention
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 41 / 41
![Page 177: Some recent advances on Approximate Bayesian … · Thanks Numerous colleagues participated to parts of this work I Pierre Pudlo (Marseille) I Louis Raynal (PhD student Montpellier)](https://reader035.vdocuments.site/reader035/viewer/2022070613/5b9e39a309d3f2083f8b4b01/html5/thumbnails/177.jpg)
End
Yesterday with Benjamin and Pascal in a Chinese restaurant, thebill arrives... with a cake and a hidden message:
This year, take comfort in your rituals, but be open to new expe-riences
A clear sign, I should try to use some PAC-Bayesian results
Thank you very much for your attention
Jean-Michel Marin (UM, CNRS & IMAG) NIPS 17 PAC-Bayes workshop 9 December 2017 41 / 41