estimating mental illness in an ongoing national survey joe gfroerer, sarra hedden, peggy barker,...
TRANSCRIPT
Estimating Mental Illness in an Ongoing National Survey
Joe Gfroerer, Sarra Hedden, Peggy Barker, Jonaki Bose Center for Behavioral Health Statistics and Quality, SAMHSA
Jeremy AldworthRTI International
COPAFS MeetingMarch 16, 2012
Outline of Presentation
• Summary of National Survey on Drug Use and Health (NSDUH)
• Design of Mental Health Surveillance Study (MHSS)
• Results
• Methodological issues
National Survey on Drug Use and Health (NSDUH)
• Sponsor: Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Behavioral Health Statistics and Quality (CBHSQ)
• Purpose: Estimate prevalence, correlates and trends of substance use in U.S.
• History: Conducted since 1971, annually since 1990
3
NSDUH Design• Representative nationally and in each state
• Civilian, noninstitutional population, age 12+
• Face-to-face interview using ACASI
• 68,000 respondents each year; oversample age 12-25
• $30 incentive
• Response rates (weighted, 2010):
• 88% of selected households completed screener
• 74% of selected persons completed interview
4
NSDUH Sample Design: Target Sample Sizes by State and Age Group
• Completed Interviews per State
• Large states (8): 3,600 per year
• Small states (43): 900 per year
• Completed Interviews by Age Group
• 1/3 of sample in each age group (12-17, 18-25, 26+)
5
NSDUH Questionnaire
• Use of alcohol, tobacco, and illicit drugs
• Substance use disorders (DSM-IV)
• Substance use and mental health treatment
• Health conditions, service utilization
• Demographics
• Mental health (MDE, suicide)
6
NSDUH Mental Health Surveillance Study (MHSS)
• SAMHSA legislation requires the agency to produce methods to estimate serious mental illness (SMI) (and serious emotional disturbance (SED) in children)
• TAG (2006) recommended NSDUH for SMI (and NHIS for SED)
• MHSS implemented in 2008 NSDUH
7
SAMHSA Definition of Serious Mental Illness (SMI) among Adults
Any DSM-IV mental disorder (other than developmental and substance use disorders)
WITH
serious functional impairment
(both in past year)
Estimating SMI in NSDUH• A complete diagnostic assessment to determine
SMI is not feasible in NSDUH interview • Would require too many questions
• Interviewers are not clinicians
• Alternative approach used by SAMHSA:• Administer clinical interviews on a subsample of NSDUH
respondents, to diagnose SMI
• Include short scales in main NSDUH interview, to be used as predictors of SMI in a model: K6, WHODAS
• Develop a regression model, based on subsample data, and apply to main sample data to predict SMI for each respondent
Kessler 6-item Nonspecific
Psychological Distress Scale (K-6)
• Included in NSDUH and several other large national surveys
• Developed specifically for use in large surveys
• Discriminates between cases and non-cases in community samples
• Demonstrates consistency across population groups
• Responses 0-4 for each item; combined score 0-24
Percentage Distribution of K6 Scores among Persons Aged 18 or Older: 2008
10.710.69.4
7.96.1
5.34.0 3.3 2.7 2.5 2.0 2.6
1.6 1.4 1.1 1.1 1.0 1.40.5 0.5 0.4 0.3 0.2 0.8
22.4
0
5
10
15
20
25
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Percentage
K6 Score
Measuring Impairment in NSDUH Main Sample: WHODAS
• WHO Disability Assessment Schedule (WHODAS)
• 16 items assessing functional impairments in various domains
• Reduced to 8 items for NSDUH based on IRT analysis
• Responses: 0 to 3 for each item
Clinical Interview Subsample
• At end of NSDUH interview, a request for 2nd interview on mental health is made to respondents selected for the clinical followup interview
• $30 incentive
• N=500 to 1500 per year
• Nationally representative, stratified sample
• Interview conducted by a trained clinical interviewer, by telephone, 2-4 weeks after main interview
13
Clinical Interview Content
• Structured Clinical Interview for DSM-IV (SCID): 15 specific mental disorders are covered
• Global Assessment of Functioning scale (GAF)
14
Estimation Step 1: Determine Best Weighted Logistic Regression Model Using Clinical Interview Subsample
Let π = Pr(“true” SMI│X1, X2)
logit(π) = + X1 + X2
• X1 = recoded K6 score (0-17)
• X2 = recoded WHODAS score (0-8)
Estimation Step 2: Determine Minimum-Bias Cutpoint from Clinical Interview
Data1. Based on model, each CI respondent has predicted
Pr(SMI+) =
2. Based on clinical interview, each CI respondent has a “true” SMI diagnosis
3. Select cutpoint, , for which false positives equal false negatives in the CI subsample
- If then predicted SMI status = positive
- If then predicted SMI status = negative
0
0
0
Final Model Based on 2008 Clinical Interview Data
logit( ) = -4.7500+ 0.2098X1 + 0.3839X2
Where X1 = recoded K6 score (0-17)
X2 = recoded WHODAS score (0-8)
Cutpoint: = 0.26972
17
0
Estimation Step 3: Apply Model to Main Sample
1. Based on model, and reported K6 and WHODAS scores, each NSDUH respondent has predicted Pr(SMI+) =
2. If then SMI status = yes
If then SMI status = no
0
0
ROC Statistics: Final SMI Model with K6 and WHODAS
vs. Alternative Model with K6 Only
19
Model Parameters
Predicted Rate
False Pos. Rate
False Neg. Rate
Sensi-tivity
Speci-ficity
Area Under ROC Curve
K6 .046 .029 .028 .387 .971 .679
K6 and WHODAS
.047 .024 .023 .506 .976 .741
Levels of Mental Illness
Level of MI in Past Year Definition
Low/Mild Mental Illness (LMI)
Any disorder, and GAF>59
Moderate Mental Illness (MMI)
Any disorder, and GAF 51-59
Serious Mental Illness (SMI)
Any disorder, and GAF<51
TOTAL/Any Mental Illness (AMI)
Any disorder
Secondary purpose of the MHSS was to generate estimates of “any mental illness” and to designate levels of severity:
Estimating Other Levels of Mental Illness
• Various models were compared
• Result: The SMI model, with different cutpoints, was found to predict as well as any other model
AMI/ SMI Prediction Based on Recoded K6 and WHODAS Scores
8
7
6
5
4
3
2
1
0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Recoded K6 Score
Rec
oded
WH
OD
AS
Sco
re
SMI
LMI or MMI
No MI
Prevalence of Mental Health Problems among Adults (18+): 2010
Percent with disorder/problem in past year
23
20
5
6.8
3.8
0
5
10
15
20
25
Any Mental Illness Serious MentalIllness
Major DepressiveEpisode
Serious Thoughts ofSuicide
46 mil 11 mil 15 mil 9 mil
Any Mental Illness in the Past Year among Adults Aged 18 or Older, by Age and Gender: 2010
Percent with Any Mental Illness (AMI) in the Past Year
24
FigMH2.1
20.0
29.9
22.1
14.3
16.8
23.0
0
5
10
15
20
25
30
35
18 or Older 18 to 25 26 to 49 50 or Older Male Female
Age Group Gender
Serious Mental Illness in the Past Year among Adults Aged 18 or Older, by Age and Gender: 2010
Percent with Serious Mental Illness (SMI) in the Past Year
25
FigMH2.2
5.0
7.7
5.8
3.2 3.4
6.5
0
1
2
3
4
5
6
7
8
9
18 or Older 18 to 25 26 to 49 50 or Older Male Female
Age Group Gender
Receipt of Mental Health Services among Adults Aged 18 or Older, by Level of Mental Illness: 2010
26
FigMH2.9
Percent Receiving Mental Health Services in the Past Year
Past Year Substance Use among Adults Aged 18 or Older, by Any Mental Illness: 2010
27
Percent Using Substance
FigMH4.1
Marijuana
Illicit Drugs1 Psychotherapeutics
InhalantsCocaine
HeroinHallucinogens
1 Illicit Drugs include marijuana/hashish, cocaine (including crack), heroin, hallucinogens, inhalants, or prescription-type psychotherapeutics used nonmedically.
Past Year Substance Dependence or Abuse and Mental Illness among Adults Aged 18 or Older: 2010
28
FigMH4.2
SUD = substance use disorder.
SUD,No Mental
Illness
11.2 Million
SUD and Mental Illness
9.2 Million
20.3 Million Adults Had SUD
45.9 Million Adults Had Mental Illness
36.7 Million
Mental Illness, No SUD
Past Year Substance Dependence or Abuse among Adults Aged 18 or Older, by Level of Mental Illness: 2010
Percent Dependent or Abusing Substance
29
FigMH4.4
Issue: Trend Measurement
Options:
• Update models, parameters, and/or cutpoints each year• Small annual sample high variance
• Continue to accumulate clinical interview data and evaluate models; update model when there is evidence that estimates can be substantially improved• Will need to update all prior estimates
Prevalence of Mental Illness among Adults (18+): 2008 to 2010
Percent in past year
31
19.5
4.4
19.9
4.8
20.0
5.0
0
5
10
15
20
25
Any Mental Illness Serious Mental Illness
2008 2009 2010
Issue: Nonresponse Bias and Weighting
CI Sample Disposition, 2008-2009:
Unwtd. N
Unwtd. Pct.
Wtd. Pct.
TOTAL 3,062 100.0 100.0
Respondents 2,027 66.2 59.5
Immediate refusal 420 13.7 24.3
Agreed, but noncontact 477 15.6 12.5
Other nonresponse 138 4.5 3.7
Nonresponse Bias Assessment:Rates of Key Measures among Respondents, Refusals, and
Noncontacts: Clinical Interview Sample, 2008-9
33
Percent in Past Year
3.7
5.6
15.0
17.1
1.3 1.9
9.9
5.54.9 5.3
15.5
18.2
02468
101214161820
Suicide Thoughts Perceive MH TxNeed
Rec'd MHTreatment
Marijuana Use
Respondent (60%) Refusal (24%) Noncontact (13%)
Nonresponse Bias Assessment:Age and Family Income among Respondents, Refusals,
and Noncontacts: Clinical Interview Sample, 2008-9
34
Percent
16.8
12.09.2 9.7
19.8
29.7
0
5
10
15
20
25
30
35
Age 18-25 <$20K Income
Respondent (60%) Refusal (24%) Noncontact (13%)
Other Issues
• What is the best sample design?• Optimize for modeling?
• Prevent extreme weights
• What is best estimation method?• Variance estimation not straightforward
• Estimate prevalences of specific disorders from the clinical interview sample?
Conclusions
• MHSS provides the only current data on trends in mental illness and its co-occurrence with substance use
• Estimates have been widely cited and used in analyses of the impact of health care reform
• Methods can be replicated in other surveys
• But more work needed to refine the models and estimation methods