the impact of hurricane sandy on students at the city ...€¦ · tsunami in indonesia (2004)...
TRANSCRIPT
S I M O N M C D O N N E L L
C O L I N C H E L L M A N
G I L J A E L E E
D A V I D C R O O K
O F F I C E O F P O L I C Y R E S E A R C H
T H E C I T Y U N I V E R S I T Y O F N E W Y O R K
The impact of Hurricane Sandy on students at the City University of New
York
1
Hurricane Sandy 2
October 29th 2012
Over 40 New Yorkers died
CUNY:
4 campuses damaged
10 campuses used for displaced NYers
Classes disrupted for a week
Hardship withdrawals
Questions
How many students were impacted?
Who were they?
Where were they?
Did students who lived in the surge zone do worse as a result of being exposed the storm?
Were key measures of academic progress worse for students living in the surge zone relative to similar students?
3
Outline
Our research questions
Some related literature
Data & Methodology
PSM Groups:
DiD
Additional regressions analysis
Results
Fall 2011 FTF -> Fall 2012 V peers
Conclusions & next steps
4
Literature - Potential Disruptions to Academic Progress
Examples Type of Disruption
Type of Impact Academic Non-Academic
Academic
Stopping out associated with lower
likelihood to graduate with a BA
Increased education funding following
tsunami in Indonesia (2004)
Non-Academic Increased financial strain from student
loans repayment after dropping out
Changes in voting patterns following
Hurricane Andrew (1992)
5
Many factors in student success:
High school prep., full-time, gender, race, family history
Natural disasters are historic, disruptive events
Can be seen as disruptions in students’ academic and non-academic lives
Literature - Potential Disruptions to Academic Progress
Examples Type of Disruption
Type of Impact Academic Non-Academic
Academic
Stopping out associated with lower
likelihood to graduate with a BA
Increased education funding following
tsunami in Indonesia (2004)
Non-Academic Increased financial strain from student
loans repayment after dropping out
Changes in voting patterns following
Hurricane Andrew (1992)
6
Disasters: Hurricane Katrina (Herlihy and Phillips, 2009), Indian ocean tsunami (Pelupessy et al), school shootings etc.
Family disruptions (Ver Ploeg, 2002)
Drugs/alcohol (Arria et al, 2013)
Little on higher ed. outcomes
Data
Student-level data from Office of Institutional Research and Assessment at CUNY;
The Primary Land Use Tax Lot Output (PLUTO) geospatial dataset;
Inundation flood maps developed by FEMA;
American Community Survey and decennial census data from the U.S. Census Bureau.
Bad identification of impacted students?
Too inclusive?
7
Initial Results
Fall 2012 snapshot:
17,000 of 225,000 NYC CUNY students (undergrads and grads) in surge zone – 8% of all students
Brooklyn (50%), Manhattan (20%), Queens (18%), Staten Island (8%), and the Bronx (3%).
These numbers were used to reach out to potentially impacted students
8
Methodology 1
PSM – similar to CUNY ASAP methodology
Nearest neighbor, caliper (.20 SD), non-replacement
One Cohort over time
Fall 2011 FTF Retained to Fall 2012
GPA Diff (Semester 3- Semester 1)
Credits Withdrawn Diff (Semester 3 – Semester 1)
Big caveat:
Retained to fall 2012 – more resilient students
Fall 2012 students – similar results…
9
Fall 2011 Students Retained to Fall 2012
Associate Students Baccalaureate Students
Non-Surge Zone Surge Zone Non-Surge Zone Surge Zones
Total Students (fall 2011) 19,958 1,665 8,745 766
Total Students (retained to fall 2012)
12,915
(64.7%)
1,015
(60.9%)
7,599
(86.9%)
658
(85.9%)
10
Methodology 2
DiD as Average Treatment Effect on the Treated
ACADi represents the difference in GPA score and credits withdrawn between semesters 1 and 3 for each student i.
ATT = ∑ i (ACADNon-Surge*Semester3 - ACADNon-Surge*Semester1) - ∑ i (ACADSurge*Semester3 - ACADSurge*Semester1 )
Additional regression analysis on matched groups
Same as surge with additional controls
Boro, college and major fixed effects
11
Methodology 3
Group membership (AA & BA Separately):
SURGEit = β0 + β1SEDit + β2ECOLit + β3CITit + β5HOODit + β5BOROit + εit
SED: NYCHA, race, gender, educationally disadvantaged, Pell status, dependency status
ECOL: No delay, fulltime, credits attempted (sem. 1), SEEK/CD, remediation
CIT: citizenship status
HOOD: neighborhood age, gender, earnings, education, race, building stock
BORO: Boro fixed effects
12
PSM Results
Fall 2011 FTF Retained to Fall 2012
AA:
1,015 student in Surge zone (7.3%)
799 matched to a nearest neighbor (12,915 Non-surge)
BA:
658 student in Surge zone (7.9%)
536 matched to a nearest neighbor (7,599 N0n-surge)
Bias reduction for almost all IV’s & balanced samples
13
Selected Descriptives: Associate
Variable Unmatched/ Matched
Treated (Surge)
Control (Non-Surge) % Bias
Bias Reduction
In NYCHA BBL U 0.203 0.056 44.7
M 0.173 0.173 0 100
No Pell Disbursed U 0.294 0.254 8.9
M 0.305 0.289 3.7 58.9
Educationally Disadvantaged U 0.737 0.771 -7.8
M 0.737 0.762 -5.8 25.5
No delay after high school U 0.808 0.769 9.5
M 0.809 0.796 3.1 67.7
Participant in SEEK/CD Program U 0.035 0.030 3.3
M 0.036 0.036 0 100
In Need of Any Remediation U 0.637 0.665 -5.8
M 0.633 0.647 -2.9 49.8
Age as of FTF date U 19.656 19.920 -6.5
M 19.568 19.835 -6.5 -1.2
Asian female U 0.047 0.082 -14
M 0.049 0.044 2 85.4
Asian male U 0.065 0.088 -9
M 0.065 0.071 -2.4 73.7
Black female U 0.194 0.161 8.7
M 0.204 0.194 2.6 69.9
Black male U 0.130 0.122 2.5
M 0.140 0.126 4.1 -64.4
Hispanic female U 0.141 0.218 -20.1
M 0.139 0.133 1.6 91.8
Hispanic male U 0.090 0.161 -21.5
M 0.093 0.100 -2.3 89.4
14
Fall 2011 FTF Fall 2012: Associate
Variable Sample Treated Controls Difference S.E. T-stat
Difference in GPA Sem. 3 – Sem. 1 Unmatched -0.303 -0.171 -0.132 0.063 -2.09
ATT -0.303 -0.163 -0.140 0.071 -1.96
GPA: Semester 3 Unmatched 2.173 2.260 -0.087 0.059 -1.48
ATT 2.171 2.289 -0.118 0.066 -1.80
GPA: Semester 1 Unmatched 2.476 2.431 0.045 0.053 0.85
ATT 2.474 2.452 0.021 0.060 0.35
Diff. in Credits W/D Sem. 3 – Sem. 1 Unmatched 1.057 0.898 0.159 0.158 1.01
ATT 1.052 0.922 0.130 0.178 0.73
Credits Withdrawn: Semester 3 Unmatched 1.768 1.558 0.211 0.150 1.41
ATT 1.766 1.560 0.206 0.167 1.23
Credits Withdrawn: Semester 1 Unmatched
0.712 0.660 0.052 0.090 0.58
ATT 0.714 0.638 0.076 0.098 0.78
15
Fall 2011 FTF Fall 2012: Baccalaureate
Variable Sample Treated Controls Difference S.E. T-stat
Difference in GPA Sem. 3 – Sem. 1 Unmatched -0.152 -0.148 -0.004 0.053 -0.07
ATT -0.159 -0.115 -0.043 0.063 -0.69
GPA: Semester 3 Unmatched 2.844 2.877 -0.033 0.058 -0.58
ATT 2.841 2.937 -0.096 0.068 -1.41
GPA: Semester 1 Unmatched 2.996 3.026 -0.030 0.045 -0.66
ATT 2.999 3.052 -0.053 0.053 -0.99
Diff. in Credits W/D Sem. 3-Sem. 1 Unmatched 0.461 0.417 0.044 0.158 0.28
ATT 0.473 0.280 0.192 0.185 1.04
Credits Withdrawn: Semester 3 Unmatched 0.906 0.896 0.010 0.146 0.07
ATT 0.905 0.787 0.118 0.166 0.71
Credits Withdrawn: Semester 1 Unmatched
0.445 0.479 -0.034 0.090 -0.38
ATT 0.432 0.507 -0.074 0.107 -0.69
16
What does this mean?
GPA Values - Current
Grade Undergraduate
A+ 4.0
A 4.0
A- 3.7
B+ 3.3
B 3.0
B- 2.7
C+ 2.3
C 2.0
D 1.0
F 0
WU 0
FIN 0
FAB 0
17
Additional Regression Analysis
Academic Outcome (AA & BA Separately):
ACADit = β0 + β1SURGEit + β2SEDit + β3ECOLit + β3CITit + β5HOODit +
β5HSit + β5FXit + εit
GPA Differences and Credit Withdrawn Differences Changed Major between semester 1 and semester 3
Testing impact of additional boro, college and major fixed effects
18
Regression Results: Fall 2011 FTF Fall 2012 (Associate)
Associate Students: GPA DIFFERENCE (Sem. 1 – Sem. 3)
Model 1: BORO FX
Model 2: Model 3: Model 4: Model 2A: Model 3A: Model 4A:
Lived in Surge -0.134** -0.168**
-0.194*** -0.198***
-0.183** -0.207*** -0.211***
SED + + + + + +
ECOL + + + + + +
CIT + + + + + +
N’HOOD + + + + + +
HS + + + + + +
Boro FX + + + + +
College FX + + + +
Major FX + + +
N 1,598 1,467 1,467 1,467 1,467 1,467 1,467
R2 0.005 0.074 0.080 0.081 0.087 0.093 0.094
19
Regression Results: Fall 2011 FTF Fall 2012 (Associate)
Associate Students: CREDITS W/D DIFFERENCE
Model 1: BORO FX
Model 2: Model 3: Model 4: Model 2A: Model 3A: Model 4A:
Lived in Surge 0.155 0.243 0.305* 0.316* 0.252 0.341** 0.334**
SED + + + + + +
ECOL + + + + + +
CIT + + + + + +
N’HOOD + + + + + +
HS + + + + + +
Boro FX + + + + +
College FX + + + +
Major FX + + +
N 1,598 1,467 1,467 1,467 1,467 1,467 1,467
R2 0.007 0.068 0.062 0.073 0.088 0.093 0.095
20
Conclusions
Small, and in many cases, insignificant impact on our students (so far)
But more resilient students are… more resilient.
Interventions by CUNY and other actors?
Suspension of classes? Outreach?
Our sample more resilient than average?
Perhaps our students are resilient – look at 9/11 for instance
Bad identification of impacted students?
Too inclusive?
Work with administrators etc to get a more nuanced sample
Look at longer time frame to confirm treatment and control groups follow same path
21
Next Steps
Should we include a cross sectional of the fall 2012 entering cohort?
Where should we aim this paper at?
22