the role of local specificity in the interpretation of small area estimation
DESCRIPTION
The Role of Local Specificity in the Interpretation of Small Area Estimation. Benmei Liu Scott Gilkeson Gordon Willis Rocky Feuer. 2012 FCSM Statistical Policy Seminar December 4, 2012. Outline. Overview of small area estimation - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/1.jpg)
The Role of Local Specificity in the Interpretation of Small Area
Estimation
Benmei LiuScott GilkesonGordon WillisRocky Feuer
2012 FCSM Statistical Policy SeminarDecember 4, 2012
![Page 2: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/2.jpg)
Outline
I. Overview of small area estimationII. The importance of local specificity and how it
could affect data useIII. An example from a recent project to estimate
cancer risk factors and screening behaviorIV. Discussion
2
![Page 3: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/3.jpg)
I. Overview of Small Area Estimation (SAE)
The demand for survey estimates for small areas (small geographic areas or domains) has increased in many different areas of application (e.g., income and poverty, education, health, substance use) over the past several decades
The standard direct estimation methods for survey data cannot provide reliable estimates due to the small sample size
Model-based methods that combine information from multiple related sources have been developed to increase the precision
3
![Page 4: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/4.jpg)
Basic SAE Model and Estimates
Fay-Herriot model (1979) has been considered the prominent fundamental approach
The final estimate for area derived from the Fay-Herriot class of models:
where:
is the direct estimate; is a regression-based synthetic estimate; is the proportion of the final estimate due to regression based synthetic estimate, or a measure of this borrowed strength;
4
![Page 5: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/5.jpg)
II. The importance of Local Specificity
We label the information about the use of local versus borrowed data based on the SAE techniques as local specificity
We propose that the term local specificity be used as a generalizable and intuitively understandable term for the degree to which local data contribute to the small area estimate for a specified area
5
![Page 6: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/6.jpg)
The importance of Local Specificity (Cont’d)
Local specificity can be an important indicator of fitness for use
We argue that local specificity provides unique information that is not otherwise available
For local data users, a measure of local specificity could be useful
A measure of local specificity was not provided on any of the government websites that release small area estimates data (e.g., SAIPE, NAAL, NSDUH)
6
![Page 7: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/7.jpg)
III. Communicating Local Specificity to End Users: An Example
Combining information from two health surveys to enhance small-area estimation (Raghunathan et al. 2007; Davis et al. 2010)
Project led by National Cancer Institute, with collaboration by:- National Center for Health Statistics- National Center for Chronic Disease Prevention and
Health Promotion- University of Michigan- University of Pennsylvania- Information Management Services
7
![Page 8: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/8.jpg)
Motivation for the Project
Cancer screening and risk factor data are of great interest to cancer control planners at the state and sub-state level, but accurate local statistics have been difficult to obtain
Different surveys have different strengths Combining information from surveys and
borrowing strength from other sources (e.g., Census or administrative records) using small area modeling approach could improve small-area estimates
8
![Page 9: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/9.jpg)
Surveys Used Behavioral Risk Factor Surveillance System
(BRFSS) – the largest U.S. survey tracking health conditions and risk behaviors at the state and sub-state level since 1984
Limitations: Potential nonresponse bias; Undercoverage of hhlds without landline phones
National Health Interview Survey (NHIS) – the principal source of information on the health of the civilian noninstitutionalized population of U.S. since 1957
Limitations: Smaller sample size; only includes data on about ¼ of U.S. counties
9
![Page 10: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/10.jpg)
Project Description Bayesian methods are developed to combine
information from the two surveys; also incorporated telephone coverage rates from the Census
National Cancer Institute released estimates for two time periods: 1997-99 and 2000-03 (http://sae.cancer.gov/)- Smoking, mammography, and pap smear- Counties, health service areas, and states
Current work involves including component for cellphone-only households and for the recent periods
10
![Page 11: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/11.jpg)
Focus Group SuggestionsConducted two focus groups with cancer control
planners and public health professionals at the Comprehensive Cancer Control Leadership Institute in June 2010
Recommendations:Include these estimates within NCI’s State Cancer Profiles
website (http://statecancerprofiles.cancer.gov/)- The website is a comprehensive system of interactive maps and
graphs enabling the investigation of cancer trends at the national, state, and county level
Need a way to describe the differences between the bias-adjusted model-based estimates and existing direct estimates
Data users would appreciate an indicator like local specificity to validate the estimate against local evidence
11
![Page 12: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/12.jpg)
Issues on Communicating Local Specificity
1) How should it be measured? 2) What should it be labeled? 3) What thresholds should be set in assigning
values to it?
12
![Page 13: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/13.jpg)
1) Measuring Local Specificity
The bias-adjusted SAE model is complex and lacks an explicit shrinkage factor
The concept of borrowed strength still applies, depending primarily on the combined BRFSS and NHIS sample size within the area
NHIS sample size is confidential. The sample size of the combined sample is close to the BRFSS sample size
BRFSS sample size is published, and alone was the best practical measure of the amount of local data
13
![Page 14: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/14.jpg)
2) Labeling Local Specificity
Presenting the BRFSS sample size as a number along with the estimates didn’t convey the message of local specificity
Developed the term local specificity and selected qualitative (i.e., high, medium, and low) rather than quantitative descriptors
14
![Page 15: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/15.jpg)
3) Assigning Thresholds
Selected BRFSS sample size of 50 as the threshold for low local specificity
Determining break points for the categories of local specificity deserves further study
15
![Page 16: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/16.jpg)
Ratios of model-based county level current mammography screening rate over the bias-corrected BRFSS direct estimate
16
![Page 17: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/17.jpg)
Small area estimates of mammography screening by county in Pennsylvania, with a mini-map showing local specificity
17
Warren county 2000-2003 percentage = 65.9 (56.6-75.2)
Westmoreland county 2000-2003 percentage = 64.8 (57.5-72.2)
![Page 18: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/18.jpg)
IV. Discussion Our experience has convinced us that such a measure is
critical for end users in their use and interpretation of results
The potential importance of local specificity should not be under-emphasized, given that users demand more from SAEs than from the results of most other statistical models
There is no single computational formula for calculating levels of local specificity that will apply generally across various models and further research is needed
Whenever estimates are based on non-ignorable levels of borrowed strength, it is vitally important to disseminate analyses in such a way that local specificity, as an important index of fitness for use, be conveyed to data users in a clear and unbiased manner
18
![Page 19: The Role of Local Specificity in the Interpretation of Small Area Estimation](https://reader036.vdocuments.site/reader036/viewer/2022070504/56816754550346895ddc07f9/html5/thumbnails/19.jpg)
Thank you!
Contact information:
Benmei Liu, Ph.D.Survey StatisticianNational Cancer [email protected]
19