mds 3.0 presenter: debra saliba, md, mph ahca webinar may 14, 2008 making assessment work
TRANSCRIPT
MDS 3.0MDS 3.0
Presenter: Debra Saliba, MD, MPHPresenter: Debra Saliba, MD, MPH
AHCA WebinarAHCA WebinarMay 14, 2008May 14, 2008
Making Making Assessment WorkAssessment Work
2 5/14/08
MDS 3.0 Information Web SitesMDS 3.0 Information Web Sites
MDS 3.0 general information site:MDS 3.0 general information site: http://www.cms.hhs.gov/nursinghomequalityinits/25_nhqimds30.asphttp://www.cms.hhs.gov/nursinghomequalityinits/25_nhqimds30.asp MDS 3.0 draft version--1/15/08MDS 3.0 draft version--1/15/08 http://www.cms.hhs.gov/NursingHomeQualityInits/Downloads/MDS30DraftVersion.pdfhttp://www.cms.hhs.gov/NursingHomeQualityInits/Downloads/MDS30DraftVersion.pdf Timeline--MDS 3.0Timeline--MDS 3.0 http://www.cms.hhs.gov/NursingHomeQualityInits/Downloads/MDS30Timeline.pdfhttp://www.cms.hhs.gov/NursingHomeQualityInits/Downloads/MDS30Timeline.pdf RE: PHQ-9 (one of many reference sites available)RE: PHQ-9 (one of many reference sites available)http://www.depression-primarycare.org/clinicians/toolkits/materials/forms/phq9/http://www.depression-primarycare.org/clinicians/toolkits/materials/forms/phq9/ Link to this and other webinars:Link to this and other webinars:http://www.ahcancal.org/events/Pages/Webinars.aspxhttp://www.ahcancal.org/events/Pages/Webinars.aspx
3 5/14/08
MDS 3.0 Development & TestingMDS 3.0 Development & Testing
Funded by:Funded by:
Centers for Medicare & Medicaid ServicesCenters for Medicare & Medicaid Services
Veteran’s Administration Health Services Veteran’s Administration Health Services Research and Development Research and Development
4 5/14/08
Learning ObjectivesLearning Objectives
By the end of the session, participants will be able toBy the end of the session, participants will be able to
1.1. Report main advances in MDS 3.0Report main advances in MDS 3.0
2.2. Describe how advances were identified and Describe how advances were identified and testedtested
3.3. Identify contribution of resident voice to Identify contribution of resident voice to improved assessment for cognition, mood, improved assessment for cognition, mood, preferences and painpreferences and pain
4.4. Discuss role of facility leaders in supporting Discuss role of facility leaders in supporting improved MDS 3.0 assessmentimproved MDS 3.0 assessment
5 5/14/08
Main Advances in MDS 3.0Main Advances in MDS 3.0
Gives Resident VoiceGives Resident Voice
Increases clinical relevanceIncreases clinical relevance
Increases accuracy (validity & reliability)Increases accuracy (validity & reliability)
Increases clarityIncreases clarity
Reduces time to complete by 45%Reduces time to complete by 45%
6 5/14/08
Why Resident Voice?Why Resident Voice?
CMS’s goal is to increase resident-CMS’s goal is to increase resident-centered care centered care Respect for individual voiceRespect for individual voice Fundamental to high quality & culture changeFundamental to high quality & culture change Residents and families want care to be Residents and families want care to be
individualized and accurateindividualized and accurate
Improves accuracy, feasibility, efficiencyImproves accuracy, feasibility, efficiency General, unfocused questions do not elicit General, unfocused questions do not elicit
meaningful reportsmeaningful reports Detailed daily observations of all behaviors for Detailed daily observations of all behaviors for
all residents is time consuming and not all residents is time consuming and not feasiblefeasible
How did we identify How did we identify and test these and test these
advances?advances?
The Evaluation Team Had The Evaluation Team Had 6 Sets of Players6 Sets of Players
Lead research and administrative teamLead research and administrative team RAND: Debra Saliba, MD, MPH RAND: Debra Saliba, MD, MPH Harvard: Joan Buchanan, PhDHarvard: Joan Buchanan, PhD Administrative Lead: Malia JonesAdministrative Lead: Malia Jones
National VA Nursing Home Research CollaborativeNational VA Nursing Home Research Collaborative Los Angeles, CALos Angeles, CA –– Atlanta, GA Atlanta, GA Philadelphia, PAPhiladelphia, PA – – Bedford, MABedford, MA
Lead Quality Improvement Organization: Lead Quality Improvement Organization: Colorado Foundation for Medical CareColorado Foundation for Medical Care
Instructions, Guides and Form Design: Instructions, Guides and Form Design: Carelink, RRS Consulting, Kleimann Communications Carelink, RRS Consulting, Kleimann Communications
GroupGroup
Centers for Medicare & Medicaid ServicesCenters for Medicare & Medicaid Services Workgroups, consultants, content expertsWorkgroups, consultants, content experts
MDS 3.0 Development Proceeded MDS 3.0 Development Proceeded in 4 Phasesin 4 Phases
Townhall Meeting & Open Comment
VA Validation Protocol Research
Expert Panel Meetings
CMS Revised Draft MDS 3.0 2003
2007
Phase 1: Stakeholder and Expert Feedback
Phase 2: MDS 3.0 Item Development
Integration of Phase 1 Feedback
Phase 2 Improved Key MDS Sections Phase 2 Improved Key MDS Sections and Revised MDS Itemsand Revised MDS Items
The VA Pilot Developed and Tested MDS Items in 8 AreasThe VA Pilot Developed and Tested MDS Items in 8 Areas
MoodMood
Behavior disorders Behavior disorders
Mental statusMental status
DeliriumDelirium
PainPain
FallsFalls
Quality of lifeQuality of life
Diagnostic codingDiagnostic coding
Funded by VA HSR&DFunded by VA HSR&D
Findings of VA Research Findings of VA Research
Depression Depression Self-report is feasible & efficientSelf-report is feasible & efficient Yields more valid estimatesYields more valid estimates
PainPain Self-report is feasible & efficientSelf-report is feasible & efficient Yields more valid estimates than observationYields more valid estimates than observation Ascertaining impact on function is feasible and Ascertaining impact on function is feasible and
provides useful informationprovides useful information
Customary Routine and ActivitiesCustomary Routine and Activities As recommended by TEP and Validation panel, asking As recommended by TEP and Validation panel, asking
importance is feasibleimportance is feasible
Funded by VA HSR&DFunded by VA HSR&D
Findings of VA ResearchFindings of VA Research
DiagnosesDiagnoses Algorithms to define active diagnosis improve Algorithms to define active diagnosis improve
identification compared to administrative dataidentification compared to administrative data
DeliriumDelirium Revised protocol and instruction improved agreementRevised protocol and instruction improved agreement
CognitionCognition Structured interview is feasible and welcomed by staffStructured interview is feasible and welcomed by staff
FallsFalls Simplified response options can be used by NH staff Simplified response options can be used by NH staff
to classify fallsto classify falls
BehaviorBehavior Items can consider impact on residentItems can consider impact on resident
Funded by VA HSR&DFunded by VA HSR&D
13 5/14/08
MDS 3.0 Development Proceeded MDS 3.0 Development Proceeded in 4 Phasesin 4 Phases
Townhall Meeting & Open Comment
VA Validation Protocol Research
Expert Panel Meetings
CMS Revised Draft MDS 3.0
National Pilot Testing
National Test of MDS 3.0
2003
2007
Phase 1: Stakeholder and Expert Feedback
Phase 2: MDS 3.0 Item Development
Integration of Phase 1 Feedback
Phase 3: MDS 3.0 Integration
Workgroup Review
Develop form & Instruction
Phase 4: National Testing
Final Revisions
Data Analysis
14 5/14/08
Revised Form DesignRevised Form Design
Form structured to improve usabilityForm structured to improve usability
Important definitions put on formImportant definitions put on form
Larger font Larger font
Logical breaks, fewer items to a pageLogical breaks, fewer items to a page
Items that were confusing or not needed for Items that were confusing or not needed for programming deletedprogramming deleted
MDS 3.0 Was Tested 71 NHs in 8 States MDS 3.0 Was Tested 71 NHs in 8 States
CACO
TXGA
NC
IL
PANJ
3800 residents participated in different 3800 residents participated in different parts of the evaluationparts of the evaluation
16 5/14/08
Different Types of Data Collectors Different Types of Data Collectors Tested MDS 3.0Tested MDS 3.0
Each state had 2 gold Each state had 2 gold standard nurse data standard nurse data collectorscollectors
Each nursing home Each nursing home had 1 facility nurse had 1 facility nurse data collector data collector
The National Test Measured The National Test Measured Reliability and Validity of MDS 3.0Reliability and Validity of MDS 3.0
Reliability of MDS 3.0Reliability of MDS 3.0 Inter-rater reliability measures the extent to Inter-rater reliability measures the extent to
which two data collectors achieve the same which two data collectors achieve the same results when assessing the same eventresults when assessing the same event
1.1. Gold-standard to Gold StandardGold-standard to Gold Standard
2.2. Gold-standard to Facility NurseGold-standard to Facility Nurse
Validity of MDS 3.0Validity of MDS 3.0 Validity assesses the degree to which items Validity assesses the degree to which items
measure the intended concept measure the intended concept
18 5/14/08
Additional Evaluation ElementsAdditional Evaluation Elements
Time to completeTime to complete Recorded all start and stop times for both MDS 2.0 and Recorded all start and stop times for both MDS 2.0 and
MDS 3.0MDS 3.0
Two Anonymous SurveysTwo Anonymous Surveys Mailed to all nurses who participated Mailed to all nurses who participated MDS 2.0 survey firstMDS 2.0 survey first MDS 3.0 survey completed at end of studyMDS 3.0 survey completed at end of study Provided feedback onProvided feedback on
Clinical usefulness of measuresClinical usefulness of measures Clarity and ease of completionClarity and ease of completion Satisfaction with assessment instrumentSatisfaction with assessment instrument
MDS 2.0 collected to allow cross walk between instruments MDS 2.0 collected to allow cross walk between instruments and into payment cellsand into payment cells
Review of 5 Sections with Review of 5 Sections with Major RevisionsMajor Revisions
20 5/14/08
1. MDS 3.0 Cognitive Assessment1. MDS 3.0 Cognitive Assessment
Brief Interview for Mental Status (BIMS)Brief Interview for Mental Status (BIMS) New structured test replaces staff assessment for New structured test replaces staff assessment for
residents who can be understoodresidents who can be understood
Staff Assessment for Mental StatusStaff Assessment for Mental Status Only completed for residents who cannot complete Only completed for residents who cannot complete
interviewinterview
Validated Confusion Assessment Method (CAM)Validated Confusion Assessment Method (CAM) Replaces old delirium itemsReplaces old delirium items
Rationale for Cognitive ChangesRationale for Cognitive Changes
Old cognitive item:Old cognitive item: Providers express discomfort with observation-based scoring Providers express discomfort with observation-based scoring
““long term memory OK” and “short term memory OK” items long term memory OK” and “short term memory OK” items are not recognized by most providersare not recognized by most providers
Only 29% thought MDS 2.0 easy to complete accuratelyOnly 29% thought MDS 2.0 easy to complete accurately
Instructs to use a formal assessment, but does not provide Instructs to use a formal assessment, but does not provide assessment or cross walk from standard assessment to 2.0assessment or cross walk from standard assessment to 2.0
CPS and COGs scales are not readily completed by NH staffCPS and COGs scales are not readily completed by NH staff
New cognitive item:New cognitive item: Directly tests domains common to most cognitive tests in Directly tests domains common to most cognitive tests in
other settings –registration, temporal orientation, recallother settings –registration, temporal orientation, recall Partial credit for close answers & response to prompts makes Partial credit for close answers & response to prompts makes
more relevant for population more relevant for population Supports validated delirium assessment protocolsSupports validated delirium assessment protocols
22 5/14/08
Rationale for Delirium ChangesRationale for Delirium Changes
Delirium is a serious condition associated with increased Delirium is a serious condition associated with increased mortality, morbidity, costs and institutionalizationmortality, morbidity, costs and institutionalization
Old delirium items:Old delirium items: Reliability in some studies worse than chanceReliability in some studies worse than chance Independent evaluations show significant under-Independent evaluations show significant under-
detection with unstructured observationdetection with unstructured observation
New delirium items = Confusion Assessment Method New delirium items = Confusion Assessment Method (CAM)(CAM)
CAM is cited as appropriate tool by Royal College of CAM is cited as appropriate tool by Royal College of Physicians, NCQA, other guidelinesPhysicians, NCQA, other guidelines
Improved sensitivity & specificity for detecting deliriumImproved sensitivity & specificity for detecting delirium
BIMS Feedback Survey ResultsBIMS Feedback Survey Results
80% thought BIMS improved 80% thought BIMS improved ability to calculate score and ability to calculate score and trigger RAPstrigger RAPs
78% preferred BIMS interview to 78% preferred BIMS interview to old assessment itemsold assessment items
88% reported that BIMS 88% reported that BIMS provided new insights into provided new insights into resident’s cognitive abilitiesresident’s cognitive abilities
24 5/14/08
Results: Cognitive Item Performance Results: Cognitive Item Performance
BIMS showed excellent reliability BIMS showed excellent reliability (kappa for score = .95)(kappa for score = .95)
Completion rates were highCompletion rates were high 90% of residents were able to 90% of residents were able to
complete complete Scores ranged from 0-15Scores ranged from 0-15
25 5/14/08
BIMS had excellent performance as a BIMS had excellent performance as a test to detect impairmenttest to detect impairment
BIMS was more highly correlated with gold-BIMS was more highly correlated with gold-standard measurestandard measure MDS. 3.0 BIMS = 0.91 (< .0001)MDS. 3.0 BIMS = 0.91 (< .0001)
MDS 2.0 CPS = - 0.74 (<.0001) MDS 2.0 CPS = - 0.74 (<.0001)
26 5/14/08
Delirium Feedback Survey ResultsDelirium Feedback Survey Results
85% found definitions on form 85% found definitions on form clearclear
71% felt that CAM helped them 71% felt that CAM helped them do a better job of screening for do a better job of screening for delirium delirium (7% disagreed)(7% disagreed)
64% reported that BIMS led them 64% reported that BIMS led them to observe new delirium to observe new delirium behaviors that differed from behaviors that differed from those in medical recordthose in medical record
27 5/14/08
Delirium showed very good reliabilityDelirium showed very good reliability
Item reliabilities ranged from Item reliabilities ranged from
kappa = .75 to .89kappa = .75 to .89
28 5/14/08
Delirium prevalence Delirium prevalence more consistent with expected ratesmore consistent with expected rates
2.0 and 3.0 Delirium Prevalence
7%
3%
7%
0%
5%
10%
15%
20%
MDS 3.0 (N=3234) MDS 2.0 (N=3262)
Subdelirium
Delrium
29 5/14/08
2. MDS 3.0 Depression Assessment2. MDS 3.0 Depression Assessment
PHQ-9PHQ-9New resident interview replaces staff observations for New resident interview replaces staff observations for residents who can report mood symptomsresidents who can report mood symptoms
Staff Assessment of PHQ-9-OVStaff Assessment of PHQ-9-OVNew observational items replace old staff assessment New observational items replace old staff assessment and only completed for residents who cannot self-and only completed for residents who cannot self-reportreport
Includes irritability itemIncludes irritability item
30 5/14/08
Rationale for Replacing Mood ItemsRationale for Replacing Mood Items
Old mood item:Old mood item: Repeatedly shown to have poor correspondence Repeatedly shown to have poor correspondence
with independent mood assessmentwith independent mood assessment Does not comport with accepted standard of self-reportDoes not comport with accepted standard of self-report Requires time consuming systematic observations of Requires time consuming systematic observations of
ALL residents across all shifts. Difficult to achieve.ALL residents across all shifts. Difficult to achieve. Only 22% reported that 2.0 section was easy to complete Only 22% reported that 2.0 section was easy to complete
accuratelyaccurately
Questionable utility for gauging response to treatment, Questionable utility for gauging response to treatment, since appropriate approach is targeting DSM-IV signs and since appropriate approach is targeting DSM-IV signs and symptomssymptoms
31 5/14/08
Rationale for Replacing Mood ItemsRationale for Replacing Mood Items
New mood item (PHQ-9) New mood item (PHQ-9)
Based on DSM-IV criteriaBased on DSM-IV criteria
Validity well established in other settingsValidity well established in other settings
Increasing use and recognition by cliniciansIncreasing use and recognition by clinicians
Allows threshold definition AND rapid sum of a Allows threshold definition AND rapid sum of a severity score that can track change over timeseverity score that can track change over time
Has been used in outpatient elders, hospital, Has been used in outpatient elders, hospital, rehabilitation (post stroke) and home health rehabilitation (post stroke) and home health populations in addition to younger adult populationspopulations in addition to younger adult populations
32 5/14/08
Mood Feedback ResultsMood Feedback Results
87% nurses rated the mood section 87% nurses rated the mood section as improved over 2.0 sectionas improved over 2.0 section
88% felt PHQ-9 interview was better 88% felt PHQ-9 interview was better than 2.0 observation for capturing than 2.0 observation for capturing resident moodresident mood
84% felt the items could inform care 84% felt the items could inform care plansplans
86% reported that items provided 86% reported that items provided new insights into moodnew insights into mood
33 5/14/08
Feedback Feedback Staff Mood Assessment Staff Mood Assessment
90% felt that detection and 90% felt that detection and communication about mood would communication about mood would improve if staff learned to watch for improve if staff learned to watch for these signs and symptomsthese signs and symptoms
72% found PHQ-9-OV assessment 72% found PHQ-9-OV assessment easier than MDS 2.0easier than MDS 2.0
34 5/14/08
Results: PHQ-9 PerformanceResults: PHQ-9 Performance
PHQ-9 had excellent reliabilityPHQ-9 had excellent reliability
Resident Mood Interview Resident Mood Interview Kappa: 0.94Kappa: 0.94
Staff Mood Observations Staff Mood Observations Kappa: 0.93Kappa: 0.93
86% of non-comatose residents 86% of non-comatose residents were able to complete interviewwere able to complete interview
35 5/14/08
PHQ-9 Interview was More Valid PHQ-9 Interview was More Valid than MDS 2.0than MDS 2.0
0.69
0.52
0.15
0
0.2
0.4
0.6
0.8
PHQ9 GDS MDS 2.0
Agreement withgold standard
36 5/14/08
PHQ-9 OV was More Valid for PHQ-9 OV was More Valid for Residents unable to self-reportResidents unable to self-report
0.84
0.28
0
0.2
0.4
0.6
0.8
1
PHQ9OV MDS 2.0
Correlation withGold StandardMeasure
37 5/14/08
3. MDS 3.0 Behavior Items 3. MDS 3.0 Behavior Items
Hallucinations and psychosis Hallucinations and psychosis moved from checklist in section J & definitions put on moved from checklist in section J & definitions put on
formform
Behaviors Behaviors Revised language clearer, linked to operational definitions Revised language clearer, linked to operational definitions Revised symptom groupings to match constructsRevised symptom groupings to match constructs Replaced “alterability” with specific impact questionsReplaced “alterability” with specific impact questions Replaced “resisting care” with “reject care” and Replaced “resisting care” with “reject care” and
refocused on resident’s goals of carerefocused on resident’s goals of care
Wandering rated separately from the 3 behavioral symptoms Wandering rated separately from the 3 behavioral symptoms groups, and impact replaces alterabilitygroups, and impact replaces alterability
38 5/14/08
Rationale For ChangesRationale For Changes
Old behavior item groupings were not consistent Old behavior item groupings were not consistent with recognized factorswith recognized factors
Only 41% of nurses rated MDS 2.0 items as easy Only 41% of nurses rated MDS 2.0 items as easy to complete accuratelyto complete accurately
Consumers viewed old behavior items as pejorativeConsumers viewed old behavior items as pejorative
Staff varied widely in definition of “alterability”Staff varied widely in definition of “alterability”
Did not identify behaviors requiring interventionDid not identify behaviors requiring intervention
New specific impact items give insight into severity New specific impact items give insight into severity and potential need for treatment/interventionand potential need for treatment/intervention
39 5/14/08
Behavior Behavior Feedback Survey ResultsFeedback Survey Results
90% rated behavioral symptoms as 90% rated behavioral symptoms as easy to complete accuratelyeasy to complete accurately
91% nurses preferred the 3.0 91% nurses preferred the 3.0 behavior item section behavior item section (1% disagreed)(1% disagreed)
90-94% rated new behavioral 90-94% rated new behavioral symptoms items as clearsymptoms items as clear
88% rated impact items as providing 88% rated impact items as providing important severity informationimportant severity information
40 5/14/08
Behavior section Behavior section reliability was excellentreliability was excellent
Psychosis, kappa = 0.96Psychosis, kappa = 0.96
Overall kappa for all other Overall kappa for all other behavioral items = 0.94behavioral items = 0.94
41 5/14/08
Behavior & Psychoses Items Behavior & Psychoses Items Had Higher Agreement with Gold StandardHad Higher Agreement with Gold Standard
Gold-StandardGold-Standard MDS 3.0 KappaMDS 3.0 Kappa
(95% CI)(95% CI)
MDS 2.0 KappaMDS 2.0 Kappa
(95% CI)(95% CI)
Physical toward Physical toward othersothers
.86 .86 (.74, .97)(.74, .97) .23 .23 (.03, .43)(.03, .43)
Verbal toward Verbal toward othersothers
.73 .73 (.61, .84)(.61, .84) .31 .31 (.16, .45)(.16, .45)
Other BehaviorOther Behavior .53 .53 (.42, .66)(.42, .66) .22 .22 (.12, .31)(.12, .31)
HallucinationsHallucinations .92 .92 (.81, 1.00)(.81, 1.00) .23 .23 (.03, .43)(.03, .43)
DelusionsDelusions .88 .88 (.79, .98)(.79, .98) .31 .31 (.16, .45)(.16, .45)
42 5/14/08
Type of Impact on Resident VariesType of Impact on Resident Varies
MDS 3.0 Behavioral Symptoms: Impact on Resident (N=317)
24%
33%36%
0%
10%
20%
30%
40%
Puts resident atrisk
Interferes withcare
Interferes withactivities
43 5/14/08
4. MDS 3.0 Customary Routine & 4. MDS 3.0 Customary Routine & ActivitiesActivities
Preferred RoutinePreferred Routine New interview replaces 20 Customary Routine staff New interview replaces 20 Customary Routine staff
assessment items for residents who can be interviewedassessment items for residents who can be interviewed Current importance rating replaces check all that applied in Current importance rating replaces check all that applied in
past yearpast year
New interview for activities preferences replaces 12 staff New interview for activities preferences replaces 12 staff assessment items for residents who can be interviewedassessment items for residents who can be interviewed
Want to talk to someone about returning to community Want to talk to someone about returning to community
Staff Assessment of Activity and Daily PreferencesStaff Assessment of Activity and Daily Preferences Only completed for residents who cannot complete interviewOnly completed for residents who cannot complete interview Major changes to several items; instructed to observe resident Major changes to several items; instructed to observe resident
response during exposure to activityresponse during exposure to activity
44 5/14/08
Rationale for changesRationale for changes
Old itemsOld items Not perceived as helping with care planningNot perceived as helping with care planning
Prior practice could be related to ability, illness, Prior practice could be related to ability, illness, access, not to preferenceaccess, not to preference
Only 30% rated 2.0 as helping care planningOnly 30% rated 2.0 as helping care planning
TEP and Validation Panels both recommended TEP and Validation Panels both recommended replace with importance scalesreplace with importance scales
New Items (Preference Assessment Tool or PAT)New Items (Preference Assessment Tool or PAT) Grounded in residential care quality Grounded in residential care quality Map to U Minnesota QoL domainsMap to U Minnesota QoL domains Focuses on resident as central to determining Focuses on resident as central to determining
activitiesactivities
45 5/14/08
Residents were able to complete Residents were able to complete
Primary Respondent for Preferred Routine & Activities (N=3246)
4%11%
84%
SelfSignificant OtherNot completed
Customary Routine Feedback ResultsCustomary Routine Feedback Results
81% rated the interview items as more 81% rated the interview items as more useful for care planninguseful for care planning
80% found that the interview changed 80% found that the interview changed their impression of resident’s wantstheir impression of resident’s wants
More likely to report that post-acute More likely to report that post-acute residents appreciated being askedresidents appreciated being asked
Only 1% felt that some residents who Only 1% felt that some residents who responded didn’t understand the itemsresponded didn’t understand the items
• Activity Feedback was similarActivity Feedback was similar
47 5/14/08
Agreement for Customary Routine & Agreement for Customary Routine & Activities was ExcellentActivities was Excellent
Preferred Routine Preferred Routine kappa = 0.97kappa = 0.97
Activities 0.96Activities 0.96kappa = 0.96kappa = 0.96
Overall scores were similar across Overall scores were similar across cognitive groups for daily routinecognitive groups for daily routine
Cognitive groupCognitive group RangeRange Mean (sd)Mean (sd)
IntactIntact(n=1384)(n=1384)
0 - 40 - 4 2.442.44 (1.08) (1.08)
ImpairedImpaired(n = 734)(n = 734)
0 - 40 - 4 2.602.60 (1.05) (1.05)
Severely impairedSeverely impaired(n=827)(n=827)
0 - 40 - 4 2.462.46 (1.09) (1.09)
Same pattern was seen with activity itemsSame pattern was seen with activity items
49 5/14/08
5. MDS 3.0 Pain Assessment Items5. MDS 3.0 Pain Assessment Items
Treatment items addedTreatment items added
Resident interview replaces staff observations for Resident interview replaces staff observations for residents who can report pain symptomsresidents who can report pain symptoms
Section expanded to capture effect on functionSection expanded to capture effect on function
Staff assessment of pain changed to an Staff assessment of pain changed to an observational checklist of pain behaviors and only observational checklist of pain behaviors and only completed for residents who cannot self-reportcompleted for residents who cannot self-report
50 5/14/08
Rationale for Replacing Pain ItemsRationale for Replacing Pain Items
Old pain itemOld pain item Repeatedly shown to have poor correspondence Repeatedly shown to have poor correspondence
with independent pain assessmentswith independent pain assessments Does not comport with accepted standard of self reportDoes not comport with accepted standard of self report
Requires time consuming systematic observations of Requires time consuming systematic observations of all residents across all shiftsall residents across all shifts
Detection bias penalizes more vigilant facilitiesDetection bias penalizes more vigilant facilities
Providers and consumers frustrated that section Providers and consumers frustrated that section addresses limited characteristics, insufficient to addresses limited characteristics, insufficient to capture pain experience capture pain experience
3 point severity response insufficient and not match 3 point severity response insufficient and not match commonly used pain scales. Want severity response commonly used pain scales. Want severity response between “moderate” & “horrible or excruciating”between “moderate” & “horrible or excruciating”
51 5/14/08
Rationale for New Pain ItemsRationale for New Pain Items
Self-report is the gold standard for pain Self-report is the gold standard for pain assessment assessment
Pilot test showed ability to recall over 5 Pilot test showed ability to recall over 5 daysdays
With pain being reported as “5With pain being reported as “5thth vital sign” vital sign” providers have increasingly used 0-10 providers have increasingly used 0-10 scales in NHs & other settingsscales in NHs & other settings
0-10 scale would allow comparison 0-10 scale would allow comparison across settingsacross settings
Pain Feedback was PositivePain Feedback was Positive
88% rated MDS 3.0 pain items as 88% rated MDS 3.0 pain items as improved over MDS 2.0improved over MDS 2.0
94% reported that new pain items 94% reported that new pain items could inform care planscould inform care plans
Even during testing, pain interview Even during testing, pain interview provided new insights into provided new insights into resident’s pain (85%)resident’s pain (85%)
90% felt that all residents who 90% felt that all residents who responded, understood (3% responded, understood (3% disagreed)disagreed)
85% felt the observable behaviors 85% felt the observable behaviors would improve reporting of possible would improve reporting of possible painpain
53 5/14/08
Results Pain Item PerformanceResults Pain Item Performance
Pain Items showed excellent reliabilityPain Items showed excellent reliability Pain treatment & interview (J1-J7)Pain treatment & interview (J1-J7)
kappa = 0.92kappa = 0.92 Staff assessment of pain (J9) Staff assessment of pain (J9)
kappa = 0.97kappa = 0.97
Completion rates were highCompletion rates were high 85% of non-comatose residents were 85% of non-comatose residents were
able to complete the pain interviewable to complete the pain interview
54 5/14/08
Pain PresencePain Presence
Pain Presence Validation Sample
64%
50%
0%
20%
40%
60%
80%
MDS 3.0
MDS 2.0
55 5/14/08
Other Measures of AgreementOther Measures of Agreement
Temporal reliabilityTemporal reliability Interview 24 hours later (different assessor Interview 24 hours later (different assessor
also)also)
kappa = .9242 (.8837, .9647)kappa = .9242 (.8837, .9647)
Agreement with MDS 2.0Agreement with MDS 2.0
kappa = .4812 (.3962, .5662)kappa = .4812 (.3962, .5662)
56 5/14/08
Staff Assessment of Pain Staff Assessment of Pain
18%
27%29%
20%
0%
5%
10%
15%
20%
25%
30%
Non VerbalSounds
VocalComplaints
FacialExpressions
BodyMovements
3.0 Staff Assessment of Pain (N=45)
43 % with any symptom43 % with any symptom
57 5/14/08
Other Sections with Other Sections with Important ChangesImportant Changes
Pressure ulcer Pressure ulcer eliminated reverse stagingeliminated reverse staging adds present on admitadds present on admit
Balance Balance refocused on movement and transitionsrefocused on movement and transitions
Falls Falls introduced type of injuryintroduced type of injury
Bowel & bladderBowel & bladder no longer rate catheter as continentno longer rate catheter as continent improved toileting program itemimproved toileting program item
58 5/14/08
Other Sections with Other Sections with Important ChangesImportant Changes
Activities of daily living – single response scaleActivities of daily living – single response scale
Goals of care item addedGoals of care item added
Oral/dental item improvedOral/dental item improved
Swallowing item - checklist of observable signs Swallowing item - checklist of observable signs and symptomsand symptoms
Restraints – separate bed and chairRestraints – separate bed and chair
59 5/14/08
Overall, Nurses Judged MDS 3.0 Overall, Nurses Judged MDS 3.0 Clinical Utility & Clarity ImprovedClinical Utility & Clarity Improved
85% rated MDS 3.0 as likely to help identify 85% rated MDS 3.0 as likely to help identify unrecognized problemsunrecognized problems
81% rated MDS 3.0 as more relevant than 2.081% rated MDS 3.0 as more relevant than 2.0
84% reported that MDS 3.0 interview items 84% reported that MDS 3.0 interview items improved their knowledge of the residentimproved their knowledge of the resident
85% rated MDS 3.0 questions as more clearly 85% rated MDS 3.0 questions as more clearly wordedworded
60 5/14/08
Nurses also rated Nurses also rated Validity HighValidity High
89% rated MDS 3.0 as providing a more 89% rated MDS 3.0 as providing a more accurate report of resident characteristics accurate report of resident characteristics than MDS 2.0than MDS 2.0
76% rated MDS 3.0 as better reflecting best 76% rated MDS 3.0 as better reflecting best clinical practice or standardsclinical practice or standards
61 5/14/08
MDS 3.0 Took Less TimeMDS 3.0 Took Less Time
MDS 3.0 TimeMDS 3.0 Time
Average time: Average time:
62 Min62 Min
MDS 2.0 TimeMDS 2.0 Time
Average time: Average time:
112 Min112 Min
<<
62 5/14/08
Given these Advances, What Role Given these Advances, What Role Might Facility Leaders play?Might Facility Leaders play?
1.1. Assist staff in understanding clinical utility & Assist staff in understanding clinical utility & efficiency of more accurate assessmentefficiency of more accurate assessment
2.2. Educate primary care providers and medical Educate primary care providers and medical consultants on availability and relevance of new consultants on availability and relevance of new assessment itemsassessment items
3.3. Assist staff in learning how to interviewAssist staff in learning how to interview
4.4. Help identify linkages with culture change Help identify linkages with culture change activitiesactivities
63 5/14/08
Take-Home Message: Take-Home Message: MDS 3.0 Revisions are Based onMDS 3.0 Revisions are Based on
Feedback from usersFeedback from users
Input from ExpertsInput from Experts
Advances in assessment scienceAdvances in assessment science
Improve clinical care in nursing homeImprove clinical care in nursing home
Improve communication with providersImprove communication with providers
Improve ability to track care and Improve ability to track care and patient progress across settings patient progress across settings
Testing of performance in NH populationsTesting of performance in NH populations
64 5/14/08
Take Home Messages: Take Home Messages: MDS 3.0 GainsMDS 3.0 Gains
National testing showed increased resident voice and National testing showed increased resident voice and refined measures in MDS 3.0:refined measures in MDS 3.0:
Increase measurement reliability & validityIncrease measurement reliability & validity Together these improve clinical detection and Together these improve clinical detection and
assessment accuracyassessment accuracy
Both Facility and Study Nurses from 71 NHs who used Both Facility and Study Nurses from 71 NHs who used MDS 3.0 reported higher satisfaction due to:MDS 3.0 reported higher satisfaction due to:
Increased clinical relevanceIncreased clinical relevance Increased clarityIncreased clarity Increased knowledge about residentIncreased knowledge about resident
National testing showed reduced completion time by 45%National testing showed reduced completion time by 45%
MDS 3.0MDS 3.0
Presenter: Debra Saliba, MD, MPHPresenter: Debra Saliba, MD, MPH
AHCA WebinarAHCA WebinarMay 14, 2008May 14, 2008
Making Making Assessment WorkAssessment Work
PointClickCare is the World’s #1 OnDemand PointClickCare is the World’s #1 OnDemand Long Term Care Software solution, offering Long Term Care Software solution, offering Providers of all sizes a comprehensive suite of Providers of all sizes a comprehensive suite of integrated applications specifically designed for integrated applications specifically designed for LTC.LTC.
RUG Tools' software helps nursing facilities RUG Tools' software helps nursing facilities increase PPS and case mix reimbursement and increase PPS and case mix reimbursement and correct MDS errors by highlighting the regulatory correct MDS errors by highlighting the regulatory impacts of each MDS assessment.impacts of each MDS assessment.
The MDS 3.0The MDS 3.0
Questions and Comments:Questions and Comments:
www.cms.hhs.gov/nursinghomequalityinits/25_nhqimds30.aspwww.cms.hhs.gov/nursinghomequalityinits/25_nhqimds30.asp
Improving Improving AssessmentAssessment
69 5/14/08
MDS 3.0 Information Web SitesMDS 3.0 Information Web Sites
MDS 3.0 general information site:MDS 3.0 general information site: http://www.cms.hhs.gov/nursinghomequalityinits/25_nhqimds30.asphttp://www.cms.hhs.gov/nursinghomequalityinits/25_nhqimds30.asp MDS 3.0 draft version--1/15/08MDS 3.0 draft version--1/15/08 http://www.cms.hhs.gov/NursingHomeQualityInits/Downloads/MDS30DraftVersion.pdfhttp://www.cms.hhs.gov/NursingHomeQualityInits/Downloads/MDS30DraftVersion.pdf Timeline--MDS 3.0Timeline--MDS 3.0 http://www.cms.hhs.gov/NursingHomeQualityInits/Downloads/MDS30Timeline.pdfhttp://www.cms.hhs.gov/NursingHomeQualityInits/Downloads/MDS30Timeline.pdf RE: PHQ-9 (one of many reference sites available)RE: PHQ-9 (one of many reference sites available)http://www.depression-primarycare.org/clinicians/toolkits/materials/forms/phq9/http://www.depression-primarycare.org/clinicians/toolkits/materials/forms/phq9/ Link to this and other webinars:Link to this and other webinars:http://www.ahcancal.org/events/Pages/Webinars.aspxhttp://www.ahcancal.org/events/Pages/Webinars.aspx