synthetic control methods yiqing xuyiqingxu.org/teaching/17802/synth.pdf · 17.802 { lecture...
TRANSCRIPT
17.802 – LectureSynthetic Control Methods
Yiqing Xu
MIT
Yiqing Xu (MIT) Synth 1 / 43
What’s Different about Panel Data?
The fundamental problem of causal inference
A statistical solution makes use of the population
e.g. T = E [Y1]− E [Y0]
A scientific solution exploits homogeneity or invariance assumptions
e.g. A rock is a rock.e.g. The long-run growth rate of the US economy is 2.5%.
Panel data allow us to construct the counterfactuals of the treatedunits in the post-treatment period using information from boththe control group and treatment group in the pre-treatment period
Yiqing Xu (MIT) Synth 3 / 43
T0 T1 T2
Causal Inference with Panel Data
T0 T1 T2
Solution 1: Time Series Analysis
DOT forecasts of road traffic volume
T0 T1 T2
Solution 2: Matching
T0 T1 T2
Solution 3: Find Parallel Worlds
Causal Inference with Panel Data
Statistical solution: SOO, e.g., matching
Scientific solution: modelling
Panel data make both easier
Matching on lagged outcomes makes SOO more plausible
Parallel trends assumption is somewhat “testable”
Yiqing Xu (MIT) Synth 10 / 43
Many Models, Same Approach
Fixed effects and diff-in-diffs
Yit(0) = αi + δt + εit
Lag dependent variable
Yit(0) = α + θYi ,t−1 + δt + εit
Synthetic control (non-parametric)
Yit(0) =∑j∈C
w∗j Yjt
LimitationsFE and LDV assume homogeneous treatment effectDID assumes fixed treatment timingSynth allows only one treated unit and inference is less formal
Yiqing Xu (MIT) Synth 11 / 43
Theoretic Motivations
What if the true model is as complicated as:
Yit(0) = Xitβ + Ziθt + λtµi + εit ,
Compare with the fixed effects (or DID) model
Yit(0) = Xitβ + Ziθt + δt + αi + εit ,
λtµi are fixed effects interacted with time-varying coefficient, inwhich δt and αi are special cases
Abadie and Gardeazabal (2003), Abadie, Diamond, and Hainmueller(2010) figured out a way to solve this problem when there is only onetreated unit
Yiqing Xu (MIT) Synth 13 / 43
Comparative Case Studies
Comparative Case Studies:
Compare the evolution of an aggregate outcome for the unit affectedby the intervention to the evolution of the same aggregate for somecontrol group (e.g. Card, 1990, Card and Krueger, 1994, Abadie andGardeazabal, 2003)
Events or interventions take place at an aggregate level (e.g., cities,states, countries).
Challenges:
Ntr is small by definition
Selection of control group is often ambiguous
Standard errors do not reflect uncertainty about the ability of thecontrol group to reproduce the counterfactual of interest
Why synthetic control can be useful?
Yiqing Xu (MIT) Synth 15 / 43
The Synthetic Control Method: Setup
Suppose that we observe J + 1 units in periods 1, 2, . . . ,T .
Region “one” is exposed to the intervention during periodsT0 + 1, . . . ,T .
Let Y Nit be the outcome that would be observed for unit i at time t in
the absence of the intervention.
Let Y Iit be the outcome that would be observed for unit i at time t if
unit i is exposed to the intervention in periods T0 + 1 to T .
We aim to estimate the effect of the intervention on the treated unit(α1T0+1, . . . , α1T ), where α1t = Y I
1t − Y N1t = Y1t − Y N
1t for t > T0.
Yiqing Xu (MIT) Synth 16 / 43
1 T0 T
J
1 ?
Setup
Suppose that Y Nit is given by a factor model:
Y Nit = δt + Ziθt + λtµi + εit ,
◦ δt is an unobserved (common) time-dependent factor,◦ Zi is a (1× r) vector of observed covariates,◦ θt is a (r × 1) vector of unknown parameters,◦ λt is a (1× F ) vector of unknown common factors,◦ µi is a (F × 1) vector of unknown factor loadings,◦ εit are unobserved transitory shocks.
λtµi : heterogeneous responses to multiple unobserved factors
Basic idea: reweight the control group such that the synthetic controlunit matches Zi and (some) pre-treatment Yit of the treated unit; asa result, µi is automatically matched
Yiqing Xu (MIT) Synth 18 / 43
1 T0 T
J
1
W*
Theory
Let W = (w2, . . . ,wJ+1)′ with wj ≥ 0 for j = 2, . . . , J + 1 andw2 + · · ·+ wJ+1 = 1. Each value of W represents a potentialsynthetic control
Let Y K1i , . . . , Y KM
i be M linear functions of pre-intervention outcomes(M ≥ F )
Suppose that we can choose W ∗ such that:
J+1∑j=2
w∗j Zj = Z1,
J+1∑j=2
w∗j YK1j = Y K1
1 , · · · ,J+1∑j=2
w∗j YKMj = Y KM
1 .
Then (if T0 is large relative to the scale of εit), an approximatelyunbiased estimator of α1t is:
α1t = Y1t −J+1∑j=2
w∗j Yjt
for t ∈ {T0 + 1, . . . ,T}Yiqing Xu (MIT) Synth 20 / 43
Implementation
Let X1 = (Z1, YK11 , . . . , Y KM
1 )′ be a (k × 1) vector of pre-interventioncharacteristics.
Similarly, X0 is a (k × J) matrix which contains the same variables forthe unaffected units.
The vector W ∗ is chosen to minimize ‖X1 − X0W ‖, subject to ourweight constraints.
We consider ‖X1 − X0W ‖V =√
(X1 − X0W )′V (X1 − X0W ), whereV is some (k × k) symmetric and positive semidefinite matrix.
Various ways to choose V (subjective assessment of predictive powerof X , regression, minimize MSPE, cross-validation, etc.).
Yiqing Xu (MIT) Synth 21 / 43
Application: California’s Proposition 99
In 1988, California first passed comprehensive tobacco control legislation:
increased cigarette tax by 25 cents/pack
earmarked tax revenues to health and anti-smoking budgets
funded anti-smoking media campaigns
spurred clean-air ordinances throughout the state
produced more than $100 million per year in anti-tobacco projects
Other states that subsequently passed control programs are excluded fromdonor pool of controls (AK, AZ, FL, HA, MA, MD, MI, NJ, NY, OR, WA,DC)
Yiqing Xu (MIT) Synth 23 / 43
Cigarette Consumption: CA and the Rest of the U.S.
1970 1975 1980 1985 1990 1995 2000
020
4060
8010
012
014
0
year
per−
capi
ta c
igar
ette
sal
es (
in p
acks
)
Californiarest of the U.S.
Passage of Proposition 99
Yiqing Xu (MIT) Synth 24 / 43
Cigarette Consumption: CA and synthetic CA
1970 1975 1980 1985 1990 1995 2000
020
4060
8010
012
014
0
year
per−
capi
ta c
igar
ette
sal
es (
in p
acks
)
Californiasynthetic California
Passage of Proposition 99
Yiqing Xu (MIT) Synth 25 / 43
Predictor Means: Actual vs. Synthetic California
California Average ofVariables Real Synthetic 38 control statesLn(GDP per capita) 10.08 9.86 9.86Percent aged 15-24 17.40 17.40 17.29Retail price 89.42 89.41 87.27Beer consumption per capita 24.28 24.20 23.75Cigarette sales per capita 1988 90.10 91.62 114.20Cigarette sales per capita 1980 120.20 120.43 136.58Cigarette sales per capita 1975 127.10 126.99 132.81
Note: All variables except lagged cigarette sales are averaged for the 1980-1988 period (beer consumption is averaged 1984-1988).
Yiqing Xu (MIT) Synth 26 / 43
Smoking Gap Between CA and synthetic CA
1970 1975 1980 1985 1990 1995 2000
−30
−20
−10
010
2030
year
gap
in p
er−
capi
ta c
igar
ette
sal
es (
in p
acks
)
Passage of Proposition 99
Yiqing Xu (MIT) Synth 27 / 43
Inference
Strategy:
Whether the effect estimated by the synthetic control for the unitaffected by the intervention is large relative to the effect estimated fora unit chosen at random
Valid regardless of the number of available comparison units, timeperiods, and whether the data are individual or aggregate
Implementation:
Iteratively apply the synthetic method to each state in the “donorpool” and obtain a distribution of placebo effects
Compare the gap for California to the distribution of the placebo gaps
Yiqing Xu (MIT) Synth 28 / 43
Smoking Gap for CA and 38 control states
(All States in Donor Pool)
1970 1975 1980 1985 1990 1995 2000
−30
−20
−10
010
2030
year
gap
in p
er−
capi
ta c
igar
ette
sal
es (
in p
acks
) Californiacontrol states
Passage of Proposition 99
Yiqing Xu (MIT) Synth 29 / 43
Smoking Gap for CA and 19 control states
(Pre-Prop. 99 MSPE ≤ 2 Times Pre-Prop. 99 MSPE for CA)
1970 1975 1980 1985 1990 1995 2000
−30
−20
−10
010
2030
year
gap
in p
er−
capi
ta c
igar
ette
sal
es (
in p
acks
) Californiacontrol states
Passage of Proposition 99
Yiqing Xu (MIT) Synth 30 / 43
Ratio Post-Prop. 99 MSPE to Pre-Prop. 99 MSPE
(All 38 States in Donor Pool)
0 20 40 60 80 100 120
01
23
45
post/pre−Proposition 99 mean squared prediction error
freq
uenc
y
California
Yiqing Xu (MIT) Synth 31 / 43
Application: German unification
Synth in cross-country studies
Cross-country regressions are often criticized because they putside-by-side countries of very different characteristics.
The synthetic control method provides an appealing data-drivenprocedure to study the effects of events or interventions that takeplace at the country level.
Application:
The economic impact of the 1990 German unification in WestGermany.
Donor pool is restricted to 21 OECD countries.
Yiqing Xu (MIT) Synth 32 / 43
Economic Growth Predictors Means
West Synthetic OECD SampleGermany West Germany excl. West Germany
GDP per-capita 8169.8 8163.1 8049.3Trade openness 45.8 54.4 32.6Inflation rate 3.4 4.7 7.3Industry share 34.7 34.7 34.3Schooling 55.5 55.6 43.8Investment rate 27.0 27.1 25.9
Note: GDP, inflation rate, and trade openness are averaged for the1960–1989 period. Industry share is averaged for the 1980–1989 pe-riod. Investment rate and schooling are averaged for the 1980–1985period.
Yiqing Xu (MIT) Synth 33 / 43
West Germany and synthetic West Germany
1960 1970 1980 1990 2000
050
0010
000
1500
020
000
2500
030
000
year
per−
capi
ta G
DP
(P
PP,
200
2 U
SD
)
West Germanysynthetic West Germany
reunification
Yiqing Xu (MIT) Synth 34 / 43
GDP Gap: West Germany and synthetic West Germany
1960 1970 1980 1990 2000
−40
00−
2000
020
0040
00
year
gap
in p
er−
capi
ta G
DP
(P
PP,
200
2 U
SD
)
reunification
Yiqing Xu (MIT) Synth 35 / 43
Country Weights in the Synthetic West Germany
Country Weight Country Weight
Australia 0 Netherlands 0.11Austria 0.47 New Zealand 0.11Belgium 0 Norway 0Canada 0 Portugal 0Denmark 0 Spain 0France 0 Sweden 0Greece 0 Switzerland 0Ireland 0 United Kingdom 0.17Italy 0 United States 0Japan 0 0.14
Yiqing Xu (MIT) Synth 36 / 43
Country Weights in the Synthetic West Germany
Synthetic Regression Synthetic RegressionCountry Weight Weight Country Weight WeightAustralia 0 0.1 Netherlands 0.11 0.18Austria 0.47 0.33 New Zealand 0 -0.08Belgium 0 0.1 Norway 0 -0.07Canada 0 0.09 Portugal 0 -0.14Denmark 0 0.04 Spain 0 0France 0 0.16 Switzerland 0.17 -0.06Greece 0 0.02 United Kingdom 0 -0.04Italy 0 -0.17 United States 0.14 0.21Japan 0.11 0.32
Note: Synthetic Weight: Unit weight assigned by the synthetic controlmethod. Regression Weight: Unit weight assigned by linear regression.
Yiqing Xu (MIT) Synth 37 / 43
Placebo Reunification 1980
1960 1965 1970 1975 1980 1985
050
0010
000
1500
020
000
2500
0
year
per−
capi
ta G
DP
(P
PP,
200
2 U
SD
)
West Germanysynthetic West Germany
placebo reunification
Yiqing Xu (MIT) Synth 38 / 43
Placebo Reunification 1970
1960 1965 1970 1975 1980 1985
050
0010
000
1500
020
000
2500
0
year
per−
capi
ta G
DP
(P
PP,
200
2 U
SD
)
West Germanysynthetic West Germany
placebo reunification
Yiqing Xu (MIT) Synth 39 / 43
Ratio of post- and pre-reunification MSPE
Portugal
UK
Switzerland
Italy
France
Netherlands
Austria
Japan
New Zealand
Denmark
Belgium
Greece
Canada
Australia
Spain
USA
Norway
West Germany
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
5 10 15
Post−Period RMSE / Pre−Period RMSE
Yiqing Xu (MIT) Synth 40 / 43
In Summary
Working with panel data makes identification easier
Time-invariant confounders are controlled forSelection on unobservables is allowedThe identifying assumption is not directly testable, but can besupported by data
However, diff-in-diffs type of research designs (DID/FE/LDV) are notfree from modelling assumptions
The synthetic control method relaxes the model assumptions, butconsiders only one treated unit, requires smooth outcomes, and theinference is less formal → we are working on it
Yiqing Xu (MIT) Synth 41 / 43