small-area estimation of ineligible sample units in a ... · josué de la rosa, u.s. census bureau...

Small-Area Estimation of Ineligible Sample Units in a Household Sample Frame

Josué De La Rosa, U.S. Census Bureau Timothy Kennel, U.S. Census Bureau

American Association for Public Opinion Research

Annual Conference May 15-18, 2014 Anaheim, CA

Disclaimer This report is released to inform interested

parties of (ongoing) research and to encourage discussion (of work in progress). Any views expressed on (statistical,

methodological, technical, or operational) issues are those of the author(s) and not necessarily those of the U.S. Census Bureau.

2

Research Question Does a county level model improve the

accuracy (i.e., lower MSE) of the predicted ineligible rate for small areas compared to a state level model?

3

Overview Introduction to Overcoverage Implications of Estimating Overcoverage Comparison of Methods of Estimating

Overcoverage Results Discussion

4

Overcoverage Error

Overcoverage – Inclusion of Ineligible Units in Sample Frame Ineligible Unit = Census Term Type B or C

Noninterview

Groves, R. M. (2009). Target Populations, Sampling Frames, and Coverage Error. Survey methodology (2nd ed., p. 76). Hoboken, N.J.: Wiley.

5

Ineligible Units

U.S. Census Bureau, (2006). Design and Methodology: Current Population Survey, Technical Paper 66, October 2006, http://www.census.gov/prod/2006pubs/tp-66.pdf.

6

Examples of Household Frame Ineligibles Vacant Demolished House Trailer Moved Condemned Converted to Business or Storage Unit

Why Estimate Overcoverage Level of Precision Influences Sample Size Sampling Ineligible Units Reduces Effective

Sample Size Estimated Overcoverage Influences Sampling

Rate

Desired Sample Size + Ineligible Units = Units Sent out for Interview

7

Background 2000 Sample Design (Legacy Method): Assume Ineligibles Homogeneous Across Counties Calculate 12 Month State Mean Ineligible Rate Apply State Mean to Each County

8

Methodology Binomial Stepwise Regression Model Predict Ineligible Rate at the County Level Covariates Data Sources: Lag + 2 County Ineligible Rate Lag + 2 USPS County Administrative Data Lag + 2 ACS County Demographic Data

9

STATS::GLM Core Team (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL http://www.R-project.org/.

Data 2010 Current Population Survey Data for

California Estimate 2012 Ineligible Rate Weighted Means Square Error

Biemer, P. P. (2010). Total Survey Error: Design, Implementation, and Evaluation. Public Opinion Quarterly, 74(5), 817-848.

10

11

Weighted 2012 County MSE = 31586.77

Legacy Method

0.00%

20.00%

40.00%

60.00%

80.00%

100.00%

0 5 10 15 20 25 30 35 40

Inel

igib

le R

ate

Observations

2012 Ineligible CA Rates Observed and State Mean

Observed 2012 Inelgible Rate 2010 State Mean

Model Diagnostics

12

13

Weighted 2012 County MSE = 5092.30

Model Estimate

0.00%

20.00%

40.00%

60.00%

80.00%

100.00%

0 5 10 15 20 25 30 35 40

Inel

igib

le R

ate

Observations

2012 Ineligible CA Rates Observed and Model Estimates

Observed 2012 Inelgible Rate Model Estimate

Legacy and Model

14

0.00%

20.00%

40.00%

60.00%

80.00%

100.00%

0 5 10 15 20 25 30 35 40

Inel

igib

le R

ate

Observations

2012 Ineligible CA Rates Observed, State mean and Model Estimates

Observed 2012 Inelgible Rate Model Estimate 2010 State Mean

Conclusion Weighted MSE State Model 31586.77 County Model 5092.30

Model Reduces Ineligible Prediction Error Compared to State Average Frame Ineligible Rate Heterogeneous Across

Counties

15

Neighborhood Foreclosures

16

Discussion Model’s Cost and Quality Implications Sophisticated Models Tract or Block level Models Geospatial Model (e.g., Kriging) Time Series Model

17

Acknowledgments Coauthor Timothy Kennel Michel Tzen, Matthew Neiman, Robert

Sanders & Reid Rottach Reviewers

18

Questions If there are any questions about this presentation, I can be reached at: Josue.delarosa@census.gov

19