Introduction à l’économétrie bayésienne:Exemples d’applications
Guillaume Horny∗
∗Banque de France et [email protected]
Guillaume Horny (Banque de France) 30.09.2008 1 / 71
Plan de la présentation
1 Régression Linéaire2 Modèle AR3 Panel Probit4 Mélange discret de normales5 DSGE
Guillaume Horny (Banque de France) 30.09.2008 2 / 71
Plan de la partie
1 Modèle
2 A priori vague
3 A priori conjugué
Guillaume Horny (Banque de France) 30.09.2008 4 / 71
Modèle
Erreurs normales, indépendantes, homoscédastiques
Modèle: y = Xβ + ε, où ε ∼ N (0, σ2I ).Vraisemblance:
l(y |β, σ2) ∝ (σ2)−n/2 exp
[− 1
2σ2
n∑i=1
(yi − xiβ)2
].
Guillaume Horny (Banque de France) 30.09.2008 5 / 71
Modèle
Choix de l’a priori
Besoin d’un a priori π(β, σ2)On va tester:
1 a priori vague2 a priori informatif
Guillaume Horny (Banque de France) 30.09.2008 6 / 71
A priori vague
Loi a posteriori (I)
A priori uniforme pour (β, lnσ2):
π(β, σ2) ∝ 1/σ2,−∞ < β < ∞, σ2 > 0.
Loi a posteriori:
l(y |β, σ2) ∝ (σ2)−1−n/2 exp[− 1
2σ2 (y − Xβ)′(y − Xβ)
].
Guillaume Horny (Banque de France) 30.09.2008 7 / 71
A priori vague
Illustration (I)
Soit Xβ = µ de dimension (n × 1)Loi a posteriori:
l(y |β, σ2) ∝ (σ2)−1−n/2 exp
[− 1
2σ2
n∑i=1
(y − µ)2
].
⇒ loi bivariée
Guillaume Horny (Banque de France) 30.09.2008 8 / 71
A priori vague
Illustration (II)
mu
sigm
a^2
0.01
0.02
0.03 0.04
0.05
0.06
0.07
0.08
−2 −1 0 1 2
0.5
1.0
1.5
2.0
2.5
3.0
m var
post
Mode: (µ = 0.28, σ2 = 1.49)Ce n’est pas une normale bivariée!
Guillaume Horny (Banque de France) 30.09.2008 9 / 71
A priori vague
Loi a posteriori (II)
On peut réécrire la loi a posteriori:
p(β, σ2|y , X ) ∝ (σ2)−1−n/2 exp[− 1
2σ2 (β − b)′X ′X (β − b)
]exp[− 1
2σ2 e ′e]
,
où b = (X ′X )−1X ′y et e = y − Xb.
Guillaume Horny (Banque de France) 30.09.2008 10 / 71
A priori vague
Loi marginale a posteriori de σ2 (I)
Loi marginale de σ2:
p(σ2|y , X ) ∝ (σ2)−v/2 exp(− vs2
2σ2
)⇔ 1/σ2 ∼ γ(v/2, vs2/2),
où v = n − k et s2 = (y − Xb)′(y − Xb)/v .Estimateur bayésien: E(σ2|y , X ) = s2.
Guillaume Horny (Banque de France) 30.09.2008 11 / 71
A priori vague
Loi marginale de σ2 (II)
0.5 1.0 1.5 2.0 2.5 3.0
0.00
00.
001
0.00
20.
003
0.00
40.
005
0.00
60.
007
sigma^2
post
erio
r de
nsity
of s
igm
a^2
Guillaume Horny (Banque de France) 30.09.2008 12 / 71
A priori vague
Loi marginale a posteriori β (I)
Loi marginale de β:
p(β|y , X ) ∝(
1 +s2
n − k(β − b)′ (X ′X )−1 (β − b)
)−n/2
Estimateur bayésien: E(β|y , X ) = b.Pour chaque élément βj de β, on a:
βj − bj
s√
(X ′X )−1jj
∼ tv ,
où (X ′X )−1jj est le j−ème élément diagonal de (X ′X )−1.
Guillaume Horny (Banque de France) 30.09.2008 13 / 71
A priori vague
Loi marginale de β (II)Cas Xβ = µ
−2 −1 0 1 2
0.0
0.2
0.4
0.6
0.8
1.0
mu
post
erio
r de
nsity
of m
u
Guillaume Horny (Banque de France) 30.09.2008 14 / 71
A priori vague
Simulations de (β, σ2)
Comme β|σ2 ∼ N (b, σ2(X ′X )−1), on échantilonne dans la loi a posteriori:
Algorithme1 tirer σ2 dans sa loi marginale a posteriori2 tirer β sa loi conditionnelle à σ2
3 répéter les étapes 1 et 2
Guillaume Horny (Banque de France) 30.09.2008 15 / 71
A priori vague
Loi marginale a posteriori ε
Estimateur bayésien de ε:
E(ε|y , X ) = V(y − Xβ|y , X ) = y − Xb.
Guillaume Horny (Banque de France) 30.09.2008 16 / 71
A priori conjugué
A priori conjugué
A priori vague:permet de voir le rôle de l’estimateur MVrend les calculs faciles
Autre possibilité: a priori conjugué
Guillaume Horny (Banque de France) 30.09.2008 17 / 71
A priori conjugué
Choix de l’a priori
Comme:
l(y , X |β, σ2) ∝ (σ2)−n/2 exp[− 1
2σ2 (β − b)′X ′X (β − b)
]exp[− 1
2σ2 e ′e]
.
l’a priori conjugué est de la forme:
π(β, σ2) ∝ (σ2)1−α/2 exp[− 1
2σ2 (β − β0)′A(β − β0)
]exp[− 1
2σ2 c]
.
Guillaume Horny (Banque de France) 30.09.2008 18 / 71
A priori conjugué
Interprétation de l’a priori
D’où:
1/σ2 ∼ γ(α/2, c/2),
β|σ2 ∼ N (β0, σ2A−1).
Guillaume Horny (Banque de France) 30.09.2008 19 / 71
A priori conjugué
Estimateur bayésien de β
Estimateur bayésien de β:
E(β|Y , X ) = (X ′X + A)−1(X ′Xb + Aβ0).
⇒ moyenne pondérée de l’estimateur MV et des paramètres a priori.
Guillaume Horny (Banque de France) 30.09.2008 20 / 71
Plan de la partie
1 Modèle
2 A priori
3 Simulation de données
4 A priori excluant une racine unitaire
5 Résultats
6 A priori permettant une racine unitaire
7 Résultats
8 Extensions
Guillaume Horny (Banque de France) 30.09.2008 22 / 71
Modèle
Modèle
∀t = 1, 2 . . . , T :
yt = α + ρyt−1 + εt ,
εt ∼ N (0, σ2).
Guillaume Horny (Banque de France) 30.09.2008 23 / 71
Modèle
Vraisemblance
On note τ = 1/σ2
Vraisemblance:
p(y |y0, α, ρ, τ) ∝ τT/2 exp
[−τ
2
T∑t=1
(yt − α− ρyt−1)2
],
pour (α, ρ) ∈ R2, τ > 0.
Guillaume Horny (Banque de France) 30.09.2008 24 / 71
A priori
Choix de l’a priori
A priori excluant une racine unitaire:
Pour − 1 < ρ < 1 :p(ρ) ∝ 1,
p(ρ) ∝ (1− ρ2)−1/2
. . .
A priori permettant une racine unitaire:
Pour −∞ < ρ < ∞ : p(ρ) ∝ 1,A priori de Jeffrey : expression complexe fonction de T
Guillaume Horny (Banque de France) 30.09.2008 25 / 71
A priori
Vraisemblance sous l’hypothèse de stationnarité
Sous l’hypothèse de stationnarité:
l(y , y0|α, ρ, τ) = p(y |y0, α, ρ, τ)p(y0|α, ρ, τ)
= τT/2 exp
[−τ
2
T∑t=1
(yt − α− ρyt−1)2
]× τ1/2(1− ρ2) exp
[−τ
2(1− ρ2)(y0 − µ)2
],
où µ = α/(1− ρ), valeur de yt à l’état stationnaire
Guillaume Horny (Banque de France) 30.09.2008 26 / 71
A priori
Relations avec le modèle linéaire
La vraisemblance est identique à celle du modèle linéaire!⇒ sous des a priori diffus pour (α, ρ, τ), l’estimateur bayésien est le même!
Guillaume Horny (Banque de France) 30.09.2008 27 / 71
Simulation de données
Données
Données simulées avec t = 50, α = 0.3, ρ = 0.9 et τ = 1
0 10 20 30 40 50
02
46
t
y
Guillaume Horny (Banque de France) 30.09.2008 28 / 71
A priori excluant une racine unitaire
A priori:π(α) gaussienne, π(ρ) uniforme sur ]− 1, 1[ et τ gammapeu informatifs: Var(α)=106, Var(τ)=103
Guillaume Horny (Banque de France) 30.09.2008 29 / 71
Résultats
Résultats
10 000 itérations, retient une itération sur 5 après les 5000 premièresLois marginales a posteriori:
−0.5 0.0 0.5 1.0 1.5
0.0
0.2
0.4
0.6
0.8
1.0
1.2
alpha
dens
ity
0.5 0.6 0.7 0.8 0.9 1.0
01
23
4
rho
dens
ity
0.5 1.0 1.5 2.0
0.0
0.5
1.0
1.5
tau
dens
ity
Guillaume Horny (Banque de France) 30.09.2008 30 / 71
A priori permettant une racine unitaire
A priori:π(α) et π(ρ) impropres, τ gammapeu informatifs: Var(τ)=103
Guillaume Horny (Banque de France) 30.09.2008 31 / 71
Résultats
Résultats
10 000 itérations, retient une itération sur 5 après les 5000 premièresLois marginales a posteriori:
−0.5 0.0 0.5 1.0 1.5
0.0
0.2
0.4
0.6
0.8
1.0
alpha
dens
ity
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2
01
23
4
rho
dens
ity
0.2 0.4 0.6 0.8 1.0 1.2 1.4
0.0
0.5
1.0
1.5
2.0
tau
dens
ity
Guillaume Horny (Banque de France) 30.09.2008 32 / 71
Extensions
Extension aux VAR
Cogley et Sargent (2001, 2002): dynamique de l’inflation aux Etats-UnisModèle:
Yt = BtYt−1 + εt ,
Bt = Bt−1 + ωt ,
où εt ∼ N (0,Σ) et ωt ∼ N (0, V ).
Guillaume Horny (Banque de France) 30.09.2008 33 / 71
Plan de la partie
1 Modèle
2 Données simulées
3 Vraisemblance
4 Distributions a priori
5 Echantillonnage de Gibbs
6 Paramétrisation alternative
7 Algorithme du recuit simulé
Guillaume Horny (Banque de France) 30.09.2008 35 / 71
Modèle
Modèle
Les données comprennent 2 groupes, et on ne sait pas à quel groupeappartient l’individu i .Loi marginale de yi :
f (yi |µ1, µ2, α, τ) = αf (yi |µ1, τ) + (1− α)f (yi |µ2, τ),
où τ > 0 et 0 ≤ α ≤ 1.
Guillaume Horny (Banque de France) 30.09.2008 36 / 71
Données simulées
Observations issues d’un mélange de deux gaussiennes
n = 200, α = 0.4, µ1 = 2, µ2 = −1, τ = 1.
−4 −2 0 2 4
0.00
0.05
0.10
0.15
0.20
y
dens
ity
Guillaume Horny (Banque de France) 30.09.2008 37 / 71
Vraisemblance
Vraisemblance
l(yi |µ1, µ2, α, τ) =n∏
i=1
(αdi f (yi |µ1, τ)di1 + (1− α)1−di f (yi |µ2, τ)1−di
),
où di indique l’appartenance de l’individu i au groupe 1.
Guillaume Horny (Banque de France) 30.09.2008 38 / 71
Distributions a priori
Distributions a priori
α ∼ D(1, 1),
µ1 ∼ N (0, 106),
µ2 ∼ N (0, 106),
τ ∼ γ(0.001, 0.001).
Guillaume Horny (Banque de France) 30.09.2008 39 / 71
Echantillonnage de Gibbs
Echantillonnage de Gibbs (I)
10 000 itérations, retient une itération sur 5 après les 5000 premières
●●
●●
●
●
●
●
●
●●
●
●●
●
●
●
● ●
●● ●
●
●
●●●● ●
●●●
●●
●●
●
●
●●
●
●●
●
●●
● ●●
●
●
●
●● ●●
● ●●
●
●●
●
●
● ●●
●
●
●
●●
●
●
●
●●●●
●
●
●●
●
●
●●●
● ●
●●
●
● ●●●
●●
●●
●●●
●●
●
●
●● ●
●
●● ●
●
●●
●
●
●●●
●
●
●
●●
●●
● ●
●
●●
●●●
●●
●●
●●
●
●
●
●
●●
●●
●●
●
●
●
●
● ●● ●●
● ●
● ●●
● ●●
● ●
●
●
●
●●
●
●●
●●
●
●
●
●
●
● ●
●
●●●
●
●
●
●●● ●●
●
●●●
●
● ●●
●●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●●
● ●
●
●●
●●●●
●●●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
● ●
●
●
●●
● ●
●●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●●
●
● ●
●●●
●
●●●
●
●●●
●
●●
●
●
●
●
●
●●●
●
● ●●
● ●
●
●●
●
●
● ●●
● ●
●●
●
●
●●
●●
●
●● ●
●
●
●
●●● ●●
●●
●
●
●
●
● ●●●
●
●
●●
●
●
●
● ●
●
●
●●●
●
●●
●● ●
●●
●●
●
●
●
●●
●
●●●●
●
●●
● ●●
●
●●
●
●
●
●●
●●●
●●●
●
●
●● ●
●
●● ●
●●
●
●
●●
●
●●
●●
●●
●●
●●
●
●● ●●
●● ●●
●● ●●●
● ●●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●●
●
●●●
●
●● ●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
● ●
● ●●
● ●●●
●●
●●●
●●
●
●
●● ●
●
●
●●●
●
●
●
●
●●●
● ●●
●●
●
●
●
●
●●
●●
● ●
●●●
●●●
●
●●
●●
●
●●
●●●
●
●
●● ●●
●
●
●
●●
●
●●●●
● ●
●● ●●●
●●
●●
●●
●●
●
● ●
● ●
●
●
●●
●
●
●●
●
●●
● ●● ●●
●
●
●
●●
●
● ●●●
●●
● ●
●● ●
● ●●
●●
●●
●●●
● ●●●
●
●● ●
●
●
●
●●
●●●
●
●
●●
●●●
●●
●●
●
●
●●
●
●
● ●●
●
●
●
●
●
●
●
●
●●
● ●
●
●●
●
●●
●●
●
●
● ●●
●
●
●●
●●
● ●
●●
●
●
●
●●
●
●
●
●
● ●
●●●
●
●
●
●●
●
●
●
●
●●●
●●
●●
●●●●
●●
●●●
●
●
●
●
●
● ●●
●
●●
● ●●
●●
●
●●
●
●●
● ●
●● ●● ●
●
●
●
●
●
●●●
●
●●●
●
●●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●●
●
●●
●
●
●●●
●●●
●
●●
●●●
●
●●●
●●
●
●●
●
●●
●●● ●
●●●
●
●●
●●
●●
●
●
●●
●
●
●●
●
●
●●●
●
●
●●
●●
●
●●●● ●
●
●
●●●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●● ●
●●
●●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
● ●
●
● ●
●● ●
●
● ●
●
● ●● ●●●
●●
●●● ●●
● ●
●
●●
●● ●
●
●
●●
●
●
●
● ●
●●
●
−2 −1 0 1 2 3
−1
01
23
Gibbs pour mélange de normales
mu1
mu2
Guillaume Horny (Banque de France) 30.09.2008 40 / 71
Echantillonnage de Gibbs
Echantillonnage de Gibbs (II)
Ré-essai avec mêmes données, valeurs initiales et paramètres dessimulations:
●
●
●
●
●●●
●
●
●
●
● ●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
● ●
●●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●●
●
● ●
●
●●
●
●
●●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●●
●
●
●
●● ●
●
●●
●●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
● ●●
●
●
●
−2 −1 0 1 2 3
−1
01
23
Gibbs pour mélange de normales
mu1
mu2
Guillaume Horny (Banque de France) 30.09.2008 41 / 71
Echantillonnage de Gibbs
Echantillonnage de Gibbs (II)
Ré-essai avec mêmes données, valeurs initiales et paramètres dessimulations:
●
●
●
●
●●●
●
●
●
●
● ●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
● ●
●●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●●
●
● ●
●
●●
●
●
●●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●●
●
●
●
●● ●
●
●●
●●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
● ●●
●
●
●
−2 −1 0 1 2 3
−1
01
23
Gibbs pour mélange de normales
mu1
mu2
Guillaume Horny (Banque de France) 30.09.2008 41 / 71
Echantillonnage de Gibbs
Echantillonnage de Gibbs (III)
méthode stochastique!les deux composantes s’échangent leurs populations à chaque itération
Guillaume Horny (Banque de France) 30.09.2008 42 / 71
Paramétrisation alternative
Reparametrisation
Robert (1994) propose de reparamétrer le modèle:
µ2 = µ1 + θ,
où θ > 0.A priori: θ ∼ T N 0,∞(0, 106)
Guillaume Horny (Banque de France) 30.09.2008 43 / 71
Paramétrisation alternative
Echantillonnage de Gibbs (III)
10 000 itérations, retient une itération sur 5 après les 5000 premières
●●
●
●●
●
●
● ●
●●
●● ●
●
●
●●
●●
●●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●● ●
●
●●
● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
● ●●●
●
●●
●●
●
●
●
●
● ●
●
●
●●●
●
●● ●
●●
●●
●
●
●
●
●
●●●
●●
●
●●●
●
●
●●●
●
●● ● ●
●●●
●
●● ●
●
●●●●
●
●●
●●
● ●
●
●
●
●
●●●
●●●
●
●
● ●
●
●
●●
●
●
●
● ●●
●
●
●●
●●
●
●●●
●●
●
●
●
●
●●●●
●●
●
●
●●
●
●
●
●
●●
●●
●●
●●
● ●
●●
●●●
●
●●●
●●
●
●●
●
●●
●
●
●●
●●
●
●●●
●●
●
●
● ● ●●●●●
● ●
●
●●
●
●
●●●
●
●
●
●●
●
●●
●
●
●
●
●
● ●
●
●
●●
●
● ●●●
●
●●
●
●
●
●
●
●●●
●
●● ●●
●
●
●●
●
●●
●
●
●●
●
● ●
●
●
●
●●●
●● ●●
●
● ●● ●
●
●●
●
● ●●●●
●
●●●
●
●
●
●
●●
●●●
●●
●●
●
●
●●●
●●
●● ●●
●
●
●
●●
●
●
●●●
● ●
●● ●
●●
●
● ●●
●●
● ●
●●
●
●
●
●
●●
●
●●
●
●
● ●
●●
●
●
● ●●
●●●
●● ●
●●
● ●●
●
●●●
●
●●
●
●●
●
●●
●● ●
●
●
●
●●●
●
●●●
●
●
●●
●
●
●●
●● ●
●
●●●
● ●
●
●
●
●
●●●
●●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●●
●●
●●
●● ●●
●
●
●
●
● ●
●●
●
●
●●
●●
●●
●
●
● ●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●●
●●●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●● ●
●●● ●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●●●
●
●
●
●
●
●●
●
●
●
●
● ●●
●
● ●●
●
●●
●●
●
●●●
●
●●
●
●
●
●
●●●
●●●
●
●●
● ●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●● ●
●
●
●
●●
●
●●
●
●●●●
● ●●
●
●
●●●
●
● ●
●
●
●●● ●●
●●●
●●
●
●●
●
●
●
●●
●
●●
●
●●●
●
●
●
●●●●
●●
●
●
●
●●
●●
●
●●
●
●
●● ●
●●
●
●●●● ●●
●
●
●
●
●●
●●●●
●
●
●
● ●●● ●●●
●
●
●
●● ●●
●
●
●
●
● ●● ● ●
●
●
●●●●
●
●
●
●
●
●
●● ●
●●
●●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
● ●
●●
●
●●
●
●●
●●
●●
●●
●
●
●
●
●
●●
●
●
●●●● ●●
●
●
●
●
●
●
● ●● ●
●●
●
●● ●●
●●
●●
●●
● ●
●
●
●
●●●●
●●
●
●
● ●●
●●
●
●
●●
●
●
●●
●●
●
● ●●
●●
●●●
●●
●
●●
●
●
●●
●
●
●
●● ●
●
●●
●
●●
●
●●
●
● ●●
● ●●
●
●
●●
●
● ●
●
●
●
●
●
● ●●●
●●●
●●
●●
●
●●
●
●
●
−2 −1 0 1 2 3
−1
01
23
Gibbs pour mélange de normales
mu1
mu2
Guillaume Horny (Banque de France) 30.09.2008 44 / 71
Paramétrisation alternative
Lois marginales a posteriori
0.0 0.2 0.4 0.6 0.8
01
23
alpha
dens
ity
−1.8 −1.6 −1.4 −1.2 −1.0 −0.8 −0.6
0.0
0.5
1.0
1.5
2.0
2.5
mu1
dens
ity
1.5 2.0 2.5
0.0
0.5
1.0
1.5
2.0
mu2
dens
ity
Guillaume Horny (Banque de France) 30.09.2008 45 / 71
Algorithme du recuit simulé
Méthode de recuit simulé
Algorithme réputé pour optimiser les fonctions multimodalesIssu d’un processus de métallurgie, où l’alternance des cycles derefroidissement lent et de réchauffage minimise l’énergie cinétique desmolécules
Guillaume Horny (Banque de France) 30.09.2008 46 / 71
Algorithme du recuit simulé
Algorithme
Algorithme de recuit simuléInitialiser x0 et T0.Itération (k):
1 tirer xc dans une loi symétrique q(x (k), xc)
2 tirer u ∼ U[0,1]
3 calculer:r = min
(exp[f (xc)− f (x (k))/T (k)
], 1)
.
4
u{≤ r , x (k+1) = xc
> r , x (k+1) = x (k)
5 répéter les étapes de 1 à 3 jusqu’à ce que T (k) → 0.
Guillaume Horny (Banque de France) 30.09.2008 47 / 71
Algorithme du recuit simulé
Intuitions
le tirage est toujours accepté si f (xc) > f (x (k)) et T < 1le tirage est parfois accepté si f (xc) < f (x (k)) et T < 1, permettantd’échapper aux optima locauxlorsque T → 0, x (j+1) est proche de x (j) et les tirages sont de plus enplus concentrés autour l’optimum localla littérature suggère généralement T (j) = 1/ ln j .
Guillaume Horny (Banque de France) 30.09.2008 48 / 71
Algorithme du recuit simulé
Exemple
−40 −20 0 20 40
8010
012
014
016
0
10*sin(0.3*x)*sin(1.3*x^2) + 0.00001*x^4 + 0.2*x+80
x
f(x)
Guillaume Horny (Banque de France) 30.09.2008 49 / 71
Algorithme du recuit simulé
Exemple
−40 −20 0 20 40
8010
012
014
016
0
10*sin(0.3*x)*sin(1.3*x^2) + 0.00001*x^4 + 0.2*x+80
x
f(x)
Guillaume Horny (Banque de France) 30.09.2008 49 / 71
Algorithme du recuit simulé
Exemple
−20 −18 −16 −14 −12
7075
8085
10*sin(0.3*x)*sin(1.3*x^2) + 0.00001*x^4 + 0.2*x+80
x
f(x)
Guillaume Horny (Banque de France) 30.09.2008 49 / 71
Algorithme du recuit simulé
Exemple
−15.65 −15.70 −15.75 −15.80 −15.85 −15.90 −15.95 −16.00
67.4
667
.47
67.4
867
.49
67.5
0
10*sin(0.3*x)*sin(1.3*x^2) + 0.00001*x^4 + 0.2*x+80
x
f(x)
Guillaume Horny (Banque de France) 30.09.2008 49 / 71
Algorithme du recuit simulé
Remarques
le recuit simulé est une méthode Metropolis-Hastings (cf Robert,1996)si T (n), la chaîne de Markov est hétérogène
Guillaume Horny (Banque de France) 30.09.2008 50 / 71
Plan de la partie
1 Modèle
2 Loi a priori
3 Algorithme d’échantillonnage de Gibbs
Guillaume Horny (Banque de France) 30.09.2008 52 / 71
Modèle
Modèle
Soient i = 1, . . . , n, t = 1, . . . , T (panel cylindré).On note yi = (yi1, . . . , yiT ):
yit =
{1 si y∗it > 00 si y∗it < 0
,
y∗it = xitβ + witbi + εit ,
bi ∼ N (0, D),
εit ∼ N (0, 1),
où bi est un vecteur (q × 1) d’effets aléatoires pour l’individu i .
Guillaume Horny (Banque de France) 30.09.2008 53 / 71
Modèle
Densité jointe des observations
Contribution de l’individu i :
Pr(yi |β, D) =
∫Rq
{T∏
t=1
[Φ(xitβ + witbi )]yit [1− Φ(xitβ + witbi )]
1−yit
}φq(bi |0, D)dbi .
l’évaluation de la vraisemblance demande d’intégrer q foisAu delà de q = 3, utiliser la quadrature de Gauss-Hermite est un acte defoi!
Guillaume Horny (Banque de France) 30.09.2008 54 / 71
Modèle
Approche Albert et Chib (1993)
Soit la variable latente zit telle que:
zit |bi ∼ N (xitβ + witbi , 1).
On en déduit:
yit = l1(zit > 0),f (yit |zit , bi ) = l1(yit = 1) l1(zit > 0) + l1(yit = 0) l1(zit < 0).
Guillaume Horny (Banque de France) 30.09.2008 55 / 71
Modèle
Lois a posteriori
p(β, z , b|y) ∝ π(β)n∏
i=1
T∏t=1
f (yit |zit , bi )f (zit |bi )f (bi )
Guillaume Horny (Banque de France) 30.09.2008 56 / 71
Loi a priori
Loi a priori
D ′ ∼ Wishart(v−10 R0, v0),
β ∼ Np(β0, B0).
Guillaume Horny (Banque de France) 30.09.2008 57 / 71
Algorithme d’échantillonnage de Gibbs
Algorithme d’échantillonnage de Gibbs (I)
Algorithme1 tirer β dans β|y , z ,2 tirer z dans z |y , β,3 répéter les étapes 1 et 2.
Guillaume Horny (Banque de France) 30.09.2008 58 / 71
Algorithme d’échantillonnage de Gibbs
Algorithme d’échantillonnage de Gibbs (II)
Lois marginales conditionnelles:la loi de β|y , z , b est identique à la loi de β|z , b, cad gaussiennela loi de z |y , β, b est le produit des lois des zit |yit , β, bi , qui sontnormales tronquées:
zit ∼{
T N (0,∞)(xitβ + witbi , 1) si yit = 1T N (−∞,0)(xitβ + witbi , 1) si yit = 0
Guillaume Horny (Banque de France) 30.09.2008 59 / 71
Algorithme d’échantillonnage de Gibbs
Algorithme d’échantillonnage de Gibbs (III)
On note pour l’individu i :
zi = Xiβ + Wibi + εi , εi ∼ N (0, IT ).
Algorithme1
β|{zit}, D ∼ N
(B
[B−1
0 β0 +n∑
i=1
X ′i V
−1i zi
], B
),
où B =(B−1
0 +∑n
i=1 X ′i V
−1i Xi
)−1, Vi = IT + WiDW ′
i .
Guillaume Horny (Banque de France) 30.09.2008 60 / 71
Algorithme d’échantillonnage de Gibbs
Algorithme d’échantillonnage de Gibbs (IV)
Algorithme (suite)2
zit |zi(−t), yit , β, D ∼{
T N (0,∞)(µit , vit) si yit = 1T N (−∞,0)(µit , vit) si yit = 0
où µit = E[zit |zi(−t), β, D
], vit = Var
[zit |zi(−t), β, D
].
Guillaume Horny (Banque de France) 30.09.2008 61 / 71
Algorithme d’échantillonnage de Gibbs
Algorithme d’échantillonnage de Gibbs (V)
Algorithme (suite)3
bi |y , β, D ∼ N(DiW ′
i (zi − Xiβ, Di )),
où Di =(D−1 + W ′
i Wi)−1.
4D−1|y , β, {bi} ∼ Wishart(v0 + n, R0).
Guillaume Horny (Banque de France) 30.09.2008 62 / 71
Plan de la partie
1 Méthode générale de résolution
2 Algorithme MH général
Guillaume Horny (Banque de France) 30.09.2008 64 / 71
Méthode générale de résolution
Méthode générale de résolution
1 les choix des agents sont issus de programmes d’optimisationsdynamiques
2 caractérisation de l’équilibre de l’économie (état stationnaire)3 (log-)linéarisation des conditions d’équilibre au voisinage de l’état
stationnaire . . .
4 . . . pour en déduire un système d’équations dynamiques5 recherche des racines stables
Guillaume Horny (Banque de France) 30.09.2008 65 / 71
Algorithme MH général
Algorithme MH général d’estimation de DSGE (I)
Algorithme MHLecture des donnéesInitialisation des paramètres en θ(0)
Itération (k) :1 évaluation de π(θ(k))
2 résolution du système dynamique sachant θ(k), recherche desracines stables
3 évaluation de l(Y T |θ(k)) (généralement avec le filtre deKalman)
4 tirage de θc dans N (θ(k),Σ)
5 répéter les étapes de 1 à 3 pour θ(k) = θc .
Guillaume Horny (Banque de France) 30.09.2008 66 / 71
Algorithme MH général
Algorithme MH général d’estimation de DSGE (II)
Algorithme MH (suite)6 évaluer:
r =l(Y T |θc)π(θc)
l(Y T |θ(k))π(θ(k))
7
r
≥ 1, θ(k+1) = θc
< 1,{
θ(k+1) = θc avec probabilité rθ(k+1) = θ(k) avec probabilité 1− r
8 répéter les étapes de 1 à 7
Guillaume Horny (Banque de France) 30.09.2008 67 / 71
Références transversales
Fernandez-Villaverde: Methods in Macroeconomic Dynamics, notes decours.Lancaster (2004): An introduction to modern bayesian econometrics,Blackwell Publishing.Robert (1996): Méthodes de Monte Carlo par chaînes de Markov,Economica.
Guillaume Horny (Banque de France) 30.09.2008 69 / 71
Bibliographie
Albert et Chib (1993): Bayes inference via Gibbs sampling of autoregressive timeseries subject to Markov mean and variance shifts, Journal of Business andEconomic Statistics, 11, 1-15.Cogley et Sargent (2001): Evolving post world war II US inflation dynamics,NBER Macroeconomics annual, 16, 331-373.Chib et Carlin (1999): On MCMC sampling in hierarchical longitudinal models,Statistics and Computing, 9, 17-26.Robert (1994): Discussion of “Markov Chains for exploring posterior distribution”,Annals of Statistics, 22, 1742-1747.
Guillaume Horny (Banque de France) 30.09.2008 70 / 71
Logiciels utilisés
Rlibrairies: R2WinBUGS, BRugsOpenbugs
Guillaume Horny (Banque de France) 30.09.2008 71 / 71