tai: julie bershadsky presentation
DESCRIPTION
Julie Bershadsky's presentation from TAI 2014TRANSCRIPT
DATA AND ALL THAT FUN STUFF
Julie Bershadsky, PhDHuman Services Research Institute
2014 ITACC TA Institute
Data - surveys, questionnaires, phone interviews, administrative data, etc…
Data – surveys, questionnaires, phone interviews, administrative data, etc…
BUT…• Data are important!• Measure the right things
– Why, what, when– How
• Why measure?– Anecdotal evidence?– Required
• Policy makers want to see it– Evidence of change
• Baseline• Improvement
– Accountability
What to measure?• Process vs. outcomes• Outcomes:
– Reflect goals– Address issues that can be influenced– Directional– Accurate, valid and reliable– Affordable to collect– Culturally competent– Interpretable– Measurable!
• When to measure– How often?
How to measure?
• Mode– In-person– Phone– Mail– Web
• Response options• Sample?
– Who?
Using the data
• Do nothing• Can compare to an internal standard or goal• Can compare to external standard or goal• Can compare to others
• Last two: complicated…..
The Complication
?
If things were simple:
is better than
But things are not simple
?
Hmmmm…..
?
Solution: Risk adjustment
Aha!
Still complicated…• What should be adjusted?• What to adjust for?
– Differ between GA and MA• E.g. number of nice people• E.g. age
– Affect the outcome– BUT: legitimately affect the outcome (e.g.
biological factors)
National Core Indicators (NCI)
• Multi-state collaboration of state DD agencies interested in measuring how well public systems for people with developmental disabilities perform along several areas, including: employment, community inclusion, choice, rights, and health and safety
• HSRI-NASDDDS collaboration• Started in 1997• Now in over 40 states
• Why: – Allow state-to-state comparisons of performance of state DD systems
– Influence national and state policy
– Inform strategic planning and priority setting at state and national levels
National Core Indicators (NCI)• What:
– Indicator-based, focus on outcomes and quality of life
– Individual characteristics of people receiving services
– The locations where people live
– The activities they engage in during the day including whether they are working
– The nature of their experiences with the supports that they receive (e.g., with case managers, ability to make choices, self-direction)
– The context of their lives – friends, community involvement, safety
– Health and well-being, access to healthcare
National Core Indicators (NCI)• When:
– Yearly cycle• How (NCI Adult Consumer Survey):
– Standardized face-to-face interview– Response options: simple– Sample in each state
• No a priori exclusion criteria, all 18+ adults with IDD receiving at least one service in addition to case management
National Core Indicators (NCI)Using the data
• Reporting reqs• Track improvement• Comparing across groups• Comparing with national “average”• Comparing with other individual states• Formulating policy• Identify areas and priorities for QI• Raising awareness
Specialized Institutional facility
Group Home
Apartment Program
Independent Home/ Apt
Parent or Relative's home
Foster care/ Host home
Nursing Facility
Other
Don't know
0% 5% 10% 15% 20% 25% 30% 35%
6%
27%
4%
17%
34%
6%
1%
4%
0%
Residence
At Least One Psychotropic Medication by Living Arrangement
institu
tion
community
-based re
sidence
independent home/a
pt
parents/
relatives h
ome0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
53%
68%
53%
36%
20
Psych meds
21
Basic Exams & Screenings
• Higher percentages in provider-based settings
• Lowest for people living in parent/relative home
• Similar trend across indicators
institution community-based residence
independent home/apt
parents/relatives home
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% 94%97%
91%
85%
95%
88%
73%69%68%
69%
58%
43%
90%
68%
49%46%
Physical exam
Dental visit
Vision screening
Hearing test (5 years)
Choice
0%
10%
20%
30%
40%
50%
60%
70%
80%
50%45%
64%
80%
60% 60% 58% 59%
22
Wages (‘09-’10)
Table 12.4 Proportion of people who reported they have enough privacy at home
State Overall In State
N Institution Community-Based
Individual Home
Parent’s Home
Significantly Above Average
ME 97% 220 n/a 98% 100% n/a
AZ 97% 311 n/a 95% 95% 98%
KY 95% 319 n/a 94% 100% 98%
GA 95% 381 n/a 96% 100% 93%
PA 94% 835 n/a 95% 96% 95%
Within Average Range
CT 93% 247 n/a 92% 95% 94%
MEORC 93% 279 n/a 92% 94% 93%
LA 93% 258 n/a 87% 95% 95%
OH 91% 345 n/a 89% 88% 94%
NY 91% 1,637 n/a 89% 93% 93%
MA 91% 379 n/a 90% 93% 91%
NC 91% 499 n/a 90% 94% 91%
AR 90% 226 90% 80% 93% 93%
MI 89% 244 n/a 81% 88% 94%
AL 88% 226 n/a 90% n/a 86%
MO 87% 341 n/a 79% 92% n/a
IL 87% 231 n/a 85% n/a 88%
Significantly Below Average
HI 82% 201 n/a n/a n/a 81%
NJ 81% 338 n/a 80% n/a 83%
SC 79% 300 n/a 76% 86% 77%
NCI Average 90% 7,817 90% 88% 94% 91%
Unadjusted:
Table 4.1 Number of times people reported they went out shopping in the past month (Adjusted Variable)
State Adjusted Average
N (observed)
HI 5.8 430
ME 5.8 317
NC 5.3 883
AZ 4.5 351
MI 4.3 408
NY 4.3 2,504
MO 4.2 497
LA 4.2 417
MA 4.2 494
PA 4.1 1,234
CT 4.0 346
GA 3.8 485
AR 3.6 387
AL 3.2 345
IL 2.9 333
MEORC 2.8 307
KY 2.6 471
OH 2.5 273
SC 2.5 402
NJ 2.5 409
NCI Average 3.9 11,293
Adjusted:
Example: Georgia
• Use of psychotropic medications above average• Further study:
– After transition from institutions• Prescribed for the first time• Number of prescribed meds wend up
• Initiative to reduce use • Reflected in data
Example: Kentucky• Behind other states in employment, health
and exercises, psych meds, loneliness and friendship
• Employment: revised waiver, increased payment for supported employment, decreased payment for day activity services
• Psych meds: revised waiver• New service: community access• Targeted health and wellness training
Example: Arizona• Identified deficits in
women’s healthcare outcomes
• Took action to improve women’s ob/gyn access resulting in rate of annual visits increasing from 30% to 70%– Educating parents
– Educating doctors
Prioritizes actions and quality efforts on:• Aging caregivers• Well-woman checkups• Loneliness & self advocacy• Choice of case managers• Exercise• Service Costs• Physician access
Example: Alabama
• Choice and decision-making identified as area for improvement; • Five year plan to increase # of PCP facilitators and offering SRV training
to families
• Using consumer and family surveys to measure satisfaction pre- and post- community placement from closing of state facility
Why Are Data Important?Without Data
• Could not distinguish policy goals and actual outcomes of those policies
• Could not compare the effectiveness and outcomes of specific services
• Could not track important system changes – negative or positive
• Could not track the impact of system reform
With Data
• Can measure and developstrategic goals
• Can enhance transparency• Can involve individuals and
families in the interpretation of results
• Can communicate system values – e.g., choice, health, relationships
• NCI CA Movers Presentation
THE ENDWhat did she say?
