validating self-reported education: results of a pilot study jesse rothstein cecilia rouse ashley...
Post on 24-Dec-2015
215 Views
Preview:
TRANSCRIPT
Validating Self-Reported Education: Results of a Pilot Study
Jesse RothsteinCecilia RouseAshley Miller
How many high school graduates are there?
Current Population SurveyGives “status completion” rate:
# of 25-year-old graduates / 25-year-old population
Common Core of DataGives “school-based completion rate”:
# of graduates in year t / # of 9th graders in t-3.
CPS estimates much higher than CCD estimates 2000/2 cohorts: 84% CPS vs. 70% CCD
CPS vs. CCD-based graduation rates
60%
65%
70%
75%
80%
85%
90%
95%
1975 1980 1985 1990 1995 2000 2005
CPS (age 25)
CCD (9th grade demoninator)
Two easy issues HS diploma attainment rises with age in CPS
Delayed completion? Adult education / GEDs? Misreporting?
“9th grade bulge” CCD estimates usually use 9th grade enrollment as denominator Many students are held back in 9th grade 9th grade enrollment consistently about 7-10% higher than 8th
grade enrollment through 1990s These reduce the problem, but don’t eliminate it.
CPS vs. CCD-based graduation rates
60%
65%
70%
75%
80%
85%
90%
95%
1975 1980 1985 1990 1995 2000 2005
CPS (age 25)
CCD (9th grade demoninator)
CCD (8th grade denominator)
Not very sensitive to definitions CCD-based estimates all tend to yield similar
answers. Mishel and Roy (2006): Get same answers
with CPS, Census, NELS, NLSY.
Alternative graduation rates, existing data
60%
65%
70%
75%
80%
85%
90%
95%
1975 1980 1985 1990 1995 2000 2005
CPS (age 25) CCD (9th grade demoninator)CCD (8th grade denominator) Swanson (CCD)Greene/ Forster (CCD) Warren/ Halpern-Manners (CCD)Mishel/ Roy (NELS) Mishel/ Roy (NLSY97)Mishel/ Roy (Census) Mishel/ Roy (CPS)
Not very sensitive to definitions CCD-based estimates all tend to yield similar
answers. Mishel and Roy (2006): Get same answers
with CPS, Census, NELS, NLSY. Except!
Some evidence that GEDs can account for a lot of the discrepancy (Heckman and LaFontaine 2007)
Alternative graduation rates, existing data
60%
65%
70%
75%
80%
85%
90%
95%
1975 1980 1985 1990 1995 2000 2005
CPS (age 25) CCD (9th grade demoninator)CCD (8th grade denominator) Swanson (CCD)Greene/ Forster (CCD) Warren/ Halpern-Manners (CCD)Mishel/ Roy (NELS) Mishel/ Roy (NLSY97)Mishel/ Roy (Census) Mishel/ Roy (CPS)Heckman/ LaFontaine (Census)
Potential explanations
1. Attainment increase with age is poorly understood2. Proxy responses in CPS may overstate graduation3. Respondents may overstate own graduation4. “Some college” may not have graduated HS5. Regular diplomas vs. GEDs, certificates of attendance,
adult ed., etc.6. Public vs. private schools7. Immigrants/emigrants since HS8. Grade retention9. Divergent definitions in CCD10. CPS excludes armed forces, prisoners11. Poor coverage in CPS
Potential explanations that we will address
1. Attainment increase with age is poorly understood2. Proxy responses in CPS may overstate graduation3. Respondents may overstate own graduation4. “Some college” may not have graduated HS5. Regular diplomas vs. GEDs, certificates of attendance,
adult ed., etc.6. Public vs. private schools7. Immigrants/emigrants since HS8. Grade retention9. Divergent definitions in CCD10. CPS excludes armed forces, prisoners11. Poor coverage in CPS
Our Sample….
Sample frame = Households likely to have an 18-25 year old member (complied by a commercial vendor).
We purchased 1,000 (randomly selected) phone numbers.
Our target was 50 “completed” inteviews.
Survey implementation consisted of three steps. Step 1: Mimic the CPS Conduct a “CPS-style” interview with a HH member
who was at least 18 years old (the “household respondent”).
Note: We ended up with 2 versions (“A” and “B”) where version “A” asked for a full HH roster while version “B” asked initially if an 18-25 year old lived in the HH; only asked questions about one 18-25 year old.
Probe more on education questions, particularly regarding alternative types of high school degrees.
Ask for the name and location of the last high school attended by the young adult, as well as year of high school graduation.
Step 2: Verify Proxy Report and Respondent Understanding of CPS-style Education Question Ask similar “CPS-style” education questions
of the young adult him or herself (“initial report”).
Probe further on education questions (“final report”).
Also ask for the name and location of last high school attended as well as year of high school graduation.
Step 3: Verify Proxy and Self-reports with Administrative Data Send a letter to the high school requesting
verification of reported high school graduation information.
Follow-up with a telephone call for a subset of schools that did not respond within 10-12 days.
Table 1: Sample sizes (Part I) Total Version A Version B
(1) (2) (3)
N % N % N %
Number of phones called 1000 413 587
Non-working/non-residential 132 13% 93 23% 39 7%
Unable to make contact 223 22% 16 4% 207 35%
Made contact 645 65% 304 74% 341 58%
Made contact 645 304 341
Language problem 10 2% 5 2% 5 1%
Unresolved after max calls 73 11% 2 1% 71 21%
Refusal 418 65% 228 75% 190 56%
Completed HH interview 144 22% 69 23% 75 22%
Table 1: Sample sizes (Part II) Total Version A Version B
(1) (2) (3)
N % N % N %
Completed HH interview 144 69 75
No eligible youth 52 36% 25 36% 27 36%
1+ eligible youth in HH 92 64% 44 64% 48 64%
All eligible youth 117 39 78
Youth was HH respondent 10 9% 4 10% 6 8%
Failed to contact 41 35% 6 15% 35 45%
Refused 18 15% 9 23% 9 12%
Turned out ineligible 3 3% 2 5% 1 1%
Completed self-report 45 38% 18 46% 27 35%
Table 1: Sample sizes (Part III) N %
Completed self or proxy reports 117
Ineligible (outside age range/ambiguous age) 5 4%
Did not attend high school 3 3%
Has GED 4 3%
Still in high school 15 13%
Not enough identifying information 11 9%
Sent letter to high school 79 68%
All letters sent to high schools 79
No response 27 34%
Declined to provide information 6 8%
Provided information 46 58%
Conclusion
We find a high level of agreement between proxy respondents, the young adults themselves and high school administrative records on whether individual has a high school diploma (or equivalent).
We see less agreement between proxy respondents and young adults on college attendance.
Questions about modes of high school completion should be asked of those with a college education as well as those with only a high school education.
top related