differences-in- differences november 10, 2009 erick gong thanks to null & miguel
Post on 19-Dec-2015
213 views
TRANSCRIPT
![Page 1: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/1.jpg)
Differences-in-Differences
November 10, 2009
Erick Gong
Thanks to Null & Miguel
![Page 2: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/2.jpg)
Agenda
Class Scheduling Diff-in-Diff (Math & Graphs) Case Study STATA Help
![Page 3: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/3.jpg)
Class Scheduling
Nov 10: Diff-in-Diff Nov 17: Power Calculations & Guest Speaker Nov 24: Class poll: Who will be here? Dec 1: Review & Presentations
Class Poll: Who will be presenting their research proposals?
![Page 4: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/4.jpg)
The Big Picture
What is this class really about, anyway?
![Page 5: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/5.jpg)
The Big Picture
What is this class really about, anyway? Causality
![Page 6: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/6.jpg)
The Big Picture
What is this class really about, anyway? Causality
What is our biggest problem?
![Page 7: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/7.jpg)
The Big Picture
What is this class really about, anyway? Causality
What is our biggest problem? Omitted variable bias
![Page 8: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/8.jpg)
Omitted Variable Bias
The actual cause is unobserved e.g. higher wages for educated actually caused by
motivation, not schooling Happens when people get to choose their own
level of the “treatment” (broadly construed) Selection bias
Non-random program placement Because of someone else’s choice, “control” isn’t a
good counterfactual for treated
![Page 9: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/9.jpg)
Math Review
(blackboard)
![Page 10: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/10.jpg)
Math Reviewfor those of you looking at these slides later, here’s what we just wrote down:
(1) Yi = a + bTi + cXi + ei
(2) E(Yi | Ti=1) – E(Yi | Ti=0)
= [a + b + cE(Xi | Ti=1) + E(ei | Ti=1)]– [a + 0 + cE(Xi | Ti=0) + E(ei | Ti=0)]
= b + c [E(Xi | Ti=1) – E(Xi | Ti=0)]True effect “Omitted variable/selection bias” term
![Page 11: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/11.jpg)
What if we had data from before the program?What if we estimated this equation using data from before the program?
(1) Yi = a + bTi + cXi + ei
Specifically, what would our estimate of b be?
![Page 12: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/12.jpg)
What if we had data from before the program?What if we estimated this equation using data from before the program?
(1) Yi = a + bTi + cXi + ei
(2) E(Yi0| Ti1=1) – E(Yi0| Ti1=0)
= [a + 0 + cE(Xi0 | Ti1=1) + E(ei0| Ti1=1)]
– [a + 0 + cE(Xi0| Ti1=0) + E(ei0| Ti1=0)]
= c [E(Xi | Ti=1) – E(Xi | Ti=0)] “Omitted variable/selection bias” term
ALL THAT’S LEFT IS THE PROBLEMATIC TERM – HOW COULD THIS BE HELPFUL TO US?
![Page 13: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/13.jpg)
Differences-in-Differences(just what it sounds like)
Use two periods of data add second subscript to denote time
= {E(Yi1 | Ti1=1) – E(Yi1 | Ti1=0)} (difference btwn T&C, post)
– {E(Yi0 | Ti1=1) – E(Yi0 | Ti1=0)} – (difference btwn T&C, pre)
= b + c [E(Xi1 | Ti1=1) – E(Xi1 | Ti1=0)]
– c [E(Xi0 | Ti1=1) – E(Xi0 | Ti1=0)]
![Page 14: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/14.jpg)
Differences-in-Differences(just what it sounds like)
Use two periods of data add second subscript to denote time
= {E(Yi1 | Ti1=1) – E(Yi1 | Ti1=0)} (difference btwn T&C, post)
– {E(Yi0 | Ti1=1) – E(Yi0 | Ti1=0)} – (difference btwn T&C, pre)
= b + c [E(Xi1 | Ti1=1) – E(Xi1 | Ti1=0)]
– c [E(Xi0 | Ti1=1) – E(Xi0 | Ti1=0)]
= b YAY! Assume differences between X don’t change over time.
![Page 15: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/15.jpg)
Differences-in-Differences, Graphically
Pre Post
Treatment
Control
![Page 16: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/16.jpg)
Differences-in-Differences, Graphically
Pre Post
Effect of program using only pre- & post- data from T group (ignoring general time trend).
![Page 17: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/17.jpg)
Differences-in-Differences, Graphically
Pre Post
Effect of program using only T & C comparison from post-intervention (ignoring pre-existing differences between T & C groups).
![Page 18: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/18.jpg)
Differences-in-Differences, Graphically
Pre Post
![Page 19: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/19.jpg)
Differences-in-Differences,Graphically
Pre Post
Effect of program difference-in-difference (taking into account pre-existing differences between T & C and general time trend).
![Page 20: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/20.jpg)
Identifying Assumption
Whatever happened to the control group over time is what would have happened to the treatment group in the absence of the program.
Pre Post
Effect of program difference-in-difference (taking into account pre-existing differences between T & C and general time trend).
![Page 21: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/21.jpg)
Graphing Exercise
Form Groups of 3-4 4 Programs Pre-Post Treatment Effect
Take the difference of post-treatment outcome vs. pre-treatment outcome
Post-intervention (Treatment vs. Control) Comparison
Circle what you think is pre-post effect and post-intervention treat vs. control effect
Ask group volunteers
![Page 22: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/22.jpg)
Uses of Diff-in-Diff
Simple two-period, two-group comparison very useful in combination with other methods
![Page 23: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/23.jpg)
Uses of Diff-in-Diff
Simple two-period, two-group comparison very useful in combination with other methods
Randomization Regression Discontinuity Matching (propensity score)
![Page 24: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/24.jpg)
Uses of Diff-in-Diff
Simple two-period, two-group comparison very useful in combination with other methods
Randomization Regression Discontinuity Matching (propensity score)
Can also do much more complicated “cohort” analysis, comparing many groups over many time periods
![Page 25: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/25.jpg)
The (Simple) Regression
Yi,t = a + bTreati,t+ cPosti,t + d(Treati,tPosti,t )+ ei,t
Treati,t is a binary indicator (“turns on” from 0 to 1) for being in the treatment group
Posti,t is a binary indicator for the period after treatment
and Treati,tPosti,t is the interaction (product)
Interpretation of a, b, c, d is “holding all else constant”
![Page 26: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/26.jpg)
Putting Graph & Regression Together
Pre Post
Yi,t = a + bTreati,t+ cPosti,t + d(Treati,tPosti,t )+ ei,t
d is the causal effect of treatment
aa + c
a + b
a + b + c + d
![Page 27: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/27.jpg)
Putting Graph & Regression Together
Pre Post
Yi,t = a + bTreati,t+ cPosti,t + d(Treati,tPosti,t )+ ei,t
aa + c
a + b
a + b + c + d
Single Diff 2= (a+b+c+d)-(a+c) = (b+d)
Single Diff 1= (a+b)-(a)=b
![Page 28: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/28.jpg)
Putting Graph & Regression Together
Pre Post
Yi,t = a + bTreati,t+ cPosti,t + d(Treati,tPosti,t )+ ei,t
Diff-in-Diff=(Single Diff 2-Single Diff 1)=(b+d)-b=d
aa + c
a + b
a + b + c + d
Single Diff 2 = (a+b+c+d)-(a+c) = (b+d)
Single Diff 1= (a+b)-(a)=b
![Page 29: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/29.jpg)
Cohort Analysis
When you’ve got richer data, it’s not as easy to draw the picture or write the equations cross-section (lots of individuals at one point in time) time-series (one individual over lots of time) repeated cross-section (lots of individuals over several times) panel (lots of individuals, multiple times for each)
Basically, control for each time period and each “group” (fixed effects) – the coefficient on the treatment dummy is the effect you’re trying to estimate
![Page 30: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/30.jpg)
DiD Data Requirements
Either repeated cross-section or panel
Treatment can’t happen for everyone at the same time
If you believe the identifying assumption, then you can analyze policies ex post Let’s us tackle really big questions that we’re
unlikely to be able to randomize
![Page 31: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/31.jpg)
Malaria Eradication in the Americas (Bleakley 2007)
Question: What is the effect of malaria on economic development?
Data: Malaria Eradication in United States South (1920’s) Brazil, Colombia, Mexico (1950’s)
Diff-in-Diff: Use birth cohorts (old people vs. young people) & (regions with lots of malaria vs. little malaria)
Idea: Young Cohort X Region w/malaria
Result: This group higher income & literacy
![Page 32: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/32.jpg)
What’s the intuition
Areas with high pre-treatment malaria will most benefit from malaria eradication
Young people living in these areas will benefit most (older people might have partial immunity)
Comparison Group: young people living in low pre-treatment malaria areas (malaria eradication will have little effect here)
![Page 33: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/33.jpg)
Robustness Checks If possible, use data on multiple pre-program periods to
show that difference between treated & control is stable Not necessary for trends to be parallel, just to know
function for each
If possible, use data on multiple post-program periods to show that unusual difference between treated & control occurs only concurrent with program
Alternatively, use data on multiple indicators to show that response to program is only manifest for those we expect it to be (e.g. the diff-in-diff estimate of the impact of ITN distribution on diarrhea should be zero)
![Page 34: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/34.jpg)
Intermission
Come back if intro to PS4 STATA tips
![Page 35: Differences-in- Differences November 10, 2009 Erick Gong Thanks to Null & Miguel](https://reader038.vdocuments.site/reader038/viewer/2022110322/56649d2c5503460f94a027b4/html5/thumbnails/35.jpg)
Effect of 2ndary School Construction in Tanzania
Villages “Treatment Villages” got 2ndary schools “Control Villages” didn’t Who benefits from 2ndary schools?
Young People benefit
Older people out of school shouldn’t benefit Effect: (Young People X Treatment Villages)