Implementing EBP: It’s Time Implementing EBP: It’s Time Implementing EBP: It s Time Implementing EBP: It s Time We Paid Attention to We Paid Attention to M i Cli i l P fM i Cli i l P fMeasuring Clinical PerformanceMeasuring Clinical Performance

Anthony Delitto, PT, Ph.D, FAPTANovember 2, 2009November 2, 2009Philadelphia, PA

Pontiac Assembly Line 1970 Grand Prix

Performance Ass ssm ntAssessment–– I punch in on timeI punch in on time

h h –– I punch out on I punch out on timetimeI d ’t h I d ’t h –– I don’t punch my I don’t punch my coco--workers or my workers or my boss (affective boss (affective boss (affective boss (affective domain)domain)

Union contractsUnion contracts–– 55--8% raises 8% raises

regardless of regardless of regardless of regardless of performanceperformance

While Detroit Slept: How While Detroit Slept: How Toyota Invaded the Toyota Invaded the Toyota Invaded the Toyota Invaded the American Car MarketAmerican Car Market

Toyota’s success–– Cost advantage was the result of its Cost advantage was the result of its

innovative Toyota Production System (TPS).innovative Toyota Production System (TPS).–– Detroit carmakers were unwilling to adapt new Detroit carmakers were unwilling to adapt new

manufacturing techniques and therefore lost manufacturing techniques and therefore lost manufacturing techniques and therefore lost manufacturing techniques and therefore lost tremendous market sharetremendous market share

ArroganceArroganceLack of teamLack of team--approach approach

–– Union versus ManagementUnion versus Management

O i l E ll i hil h f l d hi k d blOperational Excellence is a philosophy of leadership, teamwork and problem solving resulting in continuous improvement throughout the organization by focusing on the needs of the customer, empowering employees, and optimizing existing activities in the process.activities in the process.

Toyota has turned operational excellence into a strategic weapon. This operational excellence is based in part on tools and quality improvement methods made famous b To ot in the m n f t ing o ldby Toyota in the manufacturing world

K El tKey Elements•Measurement•AccountabilityAccountability•Empowerment

O i l E ll i hil h f l d hi k d blOperational Excellence is a philosophy of leadership, teamwork and problem solving resulting in continuous improvement throughout the organization by focusing on the needs of the customer, empowering employees, and optimizing existing activities in the process.activities in the process.

Toyota has turned operational excellence into a strategic weapon. This operational excellence is based in part on tools and quality improvement methods made famous b To ot in the m n f t ing o ldby Toyota in the manufacturing world

Operational ExcellenceOperational ExcellenceOperational ExcellenceOperational Excellence

The continuous improvement is not only about improving HR quality, but y p g q y,also it is about the processes and standards improvement.pValues lie within Safety, Quality, Productivity Human Development Productivity, Human Development, Cost, and Implementation

Operational Excellence: Basic Operational Excellence: Basic ppTenetTenet

You can not improve if you improve if you do not measuremeasure

Managing LBP in the ClinicManaging LBP in the ClinicManaging LBP in the ClinicManaging LBP in the Clinic

Now we must return to the Now we must return to the clinicclinic

ClinicalAttitudeKnowledge ClinicalPerformance

BehaviorChange

Better Patient Outcome

How do we measure How do we measure performance?performance?

Clinical Performance Instruments–– Qualitative, at Qualitative, at

bestbestbestbestChart audits

Perhaps the Perhaps the –– Perhaps the Perhaps the greatest waste of greatest waste of time in clinical time in clinical

iienvironmentsenvironments

Performance InstrumentsPerformance InstrumentsPerformance InstrumentsPerformance Instruments

APTA CPIPitt Instrument (Clinical Internship Pitt Instrument (Clinical Internship Evaluation Tool)

Present Clinical Performance Present Clinical Performance InstrumentsInstruments

Good tools, but insufficient for accurate and comprehensive pmeasurement of clinical performanceConsider the recency of these Consider the recency of these instruments–– 2007 Pitt CIET2007 Pitt CIET–– 2007 Pitt CIET2007 Pitt CIET–– 2004 APTA CPI2004 APTA CPIWhat were we doing before that???What were we doing before that???

Chart auditsChart auditsChart auditsChart audits

How well you document document versus how well you well you practice

Good documentation; Chart Good documentation; Chart Audits and “QI” (as we know Audits and “QI” (as we know Audits and QI (as we know Audits and QI (as we know it today)it today)

Promote measurable practice?Standardize tests, measures, outcomes?Promote reliable and EBP process of care approaches?Develop exceptional practitioners?Promote learning and relentless greflection?

Back to performance: Why Back to performance: Why p yp ymeasure it?measure it?

We cannot assess quality unless we measure itLong overdue

Just because we have gotten away with Just because we have gotten away with –– Just because we have gotten away with Just because we have gotten away with murder is no excuse to keep committing murder is no excuse to keep committing murdermurder

TargetTargetTargetTarget

P l S tti S tPersonal• Individual PT

Environment

Setting• Multiple PTs

Similar

System• One large

clinic• Environment • Similar environments

clinic• Multiple

Settings• Partners

Target (for today at least)Target (for today at least)Target (for today, at least)Target (for today, at least)

P l S tti S tPersonal• Individual PT

Environment

Setting• Multiple PTs

Similar

System• One large

clinic• Environment • Similar environments

clinic• Multiple

Settings• Partners

Our approachOur approachOur approachOur approach

P l S tti S tPersonal• Individual PT

Environment

Setting• Multiple PTs

Similar

System• One large

clinic• Environment • Similar environments

clinic• Multiple

Settings• Partners

Low Back PainLow Back PainLow Back PainLow Back Pain

$6.2 Million on 937 cases3rd largest “cost bucket” behind

l d neoplasms and cardiopulmonary

Where is the money Where is the money spent?spent?Where is the money Where is the money spent?spent?

Unnecessary imagingPharmacologyPharmacologyUnnecessary Procedures

id ls ith t di l si s id ls ith t di l si s–– e.g., epidurals without radicular signse.g., epidurals without radicular signsRepeated visits to “rehab providers”–– Majority chiropractorsMajority chiropractors

We overestimate spontaneousWe overestimate spontaneous recovery

What is the opportunity for What is the opportunity for pp ypp your department? our department?

Development–– EBP educationEBP education

Education & TrainingEducation & Training–– Education & TrainingEducation & Training

Implementp–– Develop process of Develop process of

carecare–– MeasurableMeasurableMeasurableMeasurable–– SurveillanceSurveillance–– Measurement of Measurement of

effecteffecteffecteffect

Determinants of Clinical Performance

CLINICAL Do you have the knowledge and skills

COMPETENCErequired to do it correctly?

D t t d it tl ?MOTIVATION Do you want to do it correctly?

Will i t it t d

+

BARRIERS Will circumstances permit you to do it correctly?-

CLINICALPERFORMANCE=

How do we evaluate How do we evaluate performance????performance????

Y i h Y i h You cannot improve what You cannot improve what you do not measureyou do not measureyou o not m asuryou o not m asur

Minimal dataMinimal datasetset

What What is included?is included?What What is included?is included?

Process data by which you can answer

“How well do I adhere to a practice “How well do I adhere to a practice standard that I prospectively setstandard that I prospectively set?”?”

Minimal dataMinimal datasetset

Minimal dataMinimal datasetset

Minimal dataMinimal datasetset

The C stThe C st Effectiveness f adherence Effectiveness f adherence The CostThe Cost--Effectiveness of adherence Effectiveness of adherence to a Treatmentto a Treatment--Based Classification Based Classification (TBC) Approach compared to a non(TBC) Approach compared to a non--(TBC) Approach compared to a non(TBC) Approach compared to a nonadherent approach in the Management adherent approach in the Management of Lowof Low--Back Pain (LBP) in the Back Pain (LBP) in the ( )( )Outpatient Physical Therapy SettingOutpatient Physical Therapy Setting

McGee JC, Landry MD, Childs JC, Fitzgerald GK, Wilson JW and Delitto A

Overall DesignOverall Designgg

ID All ICD-9 Codes related to LBP

CRS Data Base• Minimal Data Set

UPMC Health Plan Data Base

k d • Common identifier in CRS and Insurance data bases

• Collected at initial visit

• Establish on/off protocol cohorts

• Track downstream costs • ONE YEAR

• Overall costs• PT costsprotocol cohorts• Member burden

PurposePurposePurposePurpose

To obtain an inference regarding the cost-effectiveness of adherence versus non-adherence to a TBC approach in the physical therapy pp p y pymanagement of LBP in terms of direct health care costs and physical p ytherapy costs

DesignDesignDesignDesign

Part 1: Cost-MinimizationConsecutive patients enrolled in UPMC LBI from Consecutive patients enrolled in UPMC LBI from O 15 h 2007 O 14 h 2008)O 15 h 2007 O 14 h 2008)Oct 15th, 2007 to Oct 14th, 2008)Oct 15th, 2007 to Oct 14th, 2008)All 42 UPMC CRS clinics in Southwestern, PAAll 42 UPMC CRS clinics in Southwestern, PAConducted from a payer perspective examining Conducted from a payer perspective examining p y p p gp y p p gcharges from initial PT visit until April 15th, 2009 charges from initial PT visit until April 15th, 2009 (standard 4% per year discounting rate applied to (standard 4% per year discounting rate applied to account for inflationary changes)account for inflationary changes)Data extracted from CRS & UPMC clinical outcomes Data extracted from CRS & UPMC clinical outcomes and financial databasesand financial databasesPayer perspectivePayer perspective

DesignDesignDesignDesign

Part 2:Decision Analysis Model –– To make inference regarding costTo make inference regarding cost--To make inference regarding costTo make inference regarding cost

effectiveness of adherence to TBC effectiveness of adherence to TBC versus nonversus non--adherenceadherence

MethodsMethodsMethodsMethods

Inclusion Criteria–– All patients newly All patients newly

referred to referred to

Exclusion Criteria–– Presence of any Presence of any

medical “red flags” medical “red flags” referred to referred to physical therapy at physical therapy at CRS with any of CRS with any of the 27 LBI the 27 LBI

medical red flags medical red flags (e.g., cancer, (e.g., cancer, compression compression fracture, fracture,

the 27 LBI the 27 LBI diagnostic codes diagnostic codes

–– 18 18 –– 65 years of 65 years of

osteoporosis, osteoporosis, infection, etc.)infection, etc.)

–– Current pregnancyCurrent pregnancyP i l b i P i l b i ageage

–– No need for No need for informed consentinformed consent

–– Prior lumbar spine Prior lumbar spine surgery surgery

–– NonNon--English speakingEnglish speaking

Measuring Performance: Measuring Performance: ggImportance of surveillanceImportance of surveillance

MDS Surveillance Program (Oct 24th through Nov 30th, 2007)g , )

Tracking Spreadsheet

150200

s # CASES

050

100150

007

007

007 007

007

007

# C

ases # CASES

COMPLETEINCOMPLETE

10/24

/20010

/31/200

11/7/

200

11/14

/20011

/21/200

11/28

/200

Date

* Only 17 85% complete through Oct 24th 2007 Only 17.85% complete through Oct 24t , 2007

MethodsMethodsMethodsMethods

MDS Surveillance Program–– Programming developed and validated to Programming developed and validated to

id tif i i i bl b th i tid tif i i i bl b th i tidentify missing variables by therapistidentify missing variables by therapist–– Weekly reports sent to CRS Quality Weekly reports sent to CRS Quality

Assurance Director (“Big Brother”)Assurance Director (“Big Brother”)Assurance Director ( Big Brother )Assurance Director ( Big Brother )Emails provided to clinicians and managersEmails provided to clinicians and managersIf no If no ∆∆ x 4wks, then f/u by CRS Directorx 4wks, then f/u by CRS Director

N nN n punitiv int rn l inc ntivpunitiv int rn l inc ntiv–– NonNon--punitive internal incentivepunitive internal incentiveFrequency of reporting weeklyFrequency of reporting weekly

–– ↓↓ every 2 weeks as of June 2008every 2 weeks as of June 2008

MethodsMethodsMethodsMethods

MDS Surveillance Program through Jan 2009

Tracking Spreadsheet

1000120014001600

ses # CASES

0200400600800

4/20

07

4/20

07

4/20

08

4/20

08

4/20

08

4/20

08

4/20

08

4/20

08

# C

a s COMPLETEINCOMPLETE

10/2

4

12/2

4

2/24

4/24

6/24

8/24

10/2

4

12/2

4

Date

* 95.5% complete as of Jan 2nd, 2009

Results: TBC Adherence Results: TBC Adherence Results: TBC Adherence Results: TBC Adherence

8090

100

89

103

121

149

40506070

On ProtocolOff Protocol380 370

%245

750

253

498121

10203040

1428

010

Total Man Stab Spec Ex

14

• 63.1% of 363 Stab. Neg. Prediction Rule candidates treated off-protocol• 82.2% of 135 Stab. Prediction Rule candidates treated on-protocol

Does it all matter?Does it all matter?Does it all matter?Does it all matter?

Develop evidence-based guidelines to standardize careDisseminate guidelinesDevelop quality indicatorsDevelop quality indicatorsTrack performanceTrack costsLink performance to costs and poutcomes

Cost SavingsCost SavingsCost SavingsCost Savings

Total Direct Net Health Care Costs–– TBC OnTBC On--Protocol $658,477.94 ($157.82 per Protocol $658,477.94 ($157.82 per

member month)member month)–– TBC Off Protocol $941,897.55 ($235.69 per TBC Off Protocol $941,897.55 ($235.69 per

member month)member month)–– $ 283,419.61 Incremental Cost Savings$ 283,419.61 Incremental Cost Savings

Total Direct Physical Therapy Costs–– TBC OnTBC On--Protocol $182,746.85 ($43.80 per Protocol $182,746.85 ($43.80 per

member month) 27.75% of total costsmember month) 27.75% of total costs$ $$ $–– TBC Off Protocol $211,054.57 ($52.81 per TBC Off Protocol $211,054.57 ($52.81 per

member month) 22.40% of total costsmember month) 22.40% of total costs–– $ 28,307.92 Incremental Cost Savings$ 28,307.92 Incremental Cost Savings

Cost SavingsCost SavingsCost SavingsCost Savings

Member Burden “Out-of Pocket Costs”–– TBC OnTBC On--Protocol $90,779.56 ($21.76 per Protocol $90,779.56 ($21.76 per

member month)member month)–– TBC Off Protocol $118,987.48 ($29.77 per TBC Off Protocol $118,987.48 ($29.77 per

member month)member month)–– $ 28,207.92 Incremental Cost Savings$ 28,207.92 Incremental Cost Savings

Physical Therapy Member Burden–– TBC OnTBC On--Protocol $43,377.70 ($10.40 per Protocol $43,377.70 ($10.40 per

member month) 47.78% of total MBmember month) 47.78% of total MB$ $$ $–– TBC Off Protocol $47,046.95 ($11.77 per TBC Off Protocol $47,046.95 ($11.77 per

member month) 39.54% of total MBmember month) 39.54% of total MB–– $ 3,669.25 Incremental Cost Savings$ 3,669.25 Incremental Cost Savings

Room Room for Improvement?for Improvement?Room Room for Improvement?for Improvement?

8090

100

89

103

121

149

40506070

On ProtocolOff Protocol380 370

%245

750

253

498121

10203040

1428

010

Total Man Stab Spec Ex

14

• 63.1% of 363 Stab. Neg. Prediction Rule candidates treated off-protocol• 82.2% of 135 Stab. Prediction Rule candidates treated on-protocol

Barriers or Motivation???Barriers or Motivation???Barriers or Motivation???Barriers or Motivation???

Internal #1–– Resistance to Resistance to

Internal #2–– Development needsDevelopment needs

change change behaviorsbehaviorsThe belief that the The belief that the expectation of expectation of

Clearly the issue Clearly the issue with MT/thrust with MT/thrust procedures procedures pp

adherence to a adherence to a standard standard is is somehow an somehow an

f f

–– PTs feel less PTs feel less confidentconfident

BUTBUTinfringement on infringement on their autonomytheir autonomy“You’re taking away “You’re taking away h ”h ”the art…”the art…”

Reasons given for nonReasons given for non--ggadherence: What would you do?adherence: What would you do?

I don’t want to do it differentlyI did not graduate from Pitt so I I did not graduate from Pitt so I don’t use thrust on everyone that comes in the cliniccomes in the clinicMy present way “works in my hands”

N m nti n f h it ks ith N m nti n f h it ks ith –– No mention of how it works with No mention of how it works with patientspatients

Y ur t kin th ARTYour taking away the ART

SystemSystem widewideSystemSystem--widewide

P l S tti S tPersonal• Individual PT

Environment

Setting• Multiple PTs

Similar

System• One large

clinic• Environment • Similar environments

clinic• Multiple

Settings• Partners

Cost savings for whom?Cost savings for whom?Cost savings for whom?Cost savings for whom?

Payer and memberWhat about Provider???What about Provider???––

What is the incentive for What is the incentive for the provider?the provider?

Increases quality of care and decreases the cost of careIt’s the right thing to doIt saves money It saves money

Aligning finances to share Aligning finances to share g gg gcost savingscost savings

Partner with payers to support QI initiativeIncentivize member adherence to QI Incentivize member adherence to QI Initiative–– Global coGlobal co--paypayp yp y

Use QI Initiative to maintain and grow the revenue

C d ti liC d ti li Increase–– CredentialingCredentialing–– Gold cardingGold carding–– Case paymentCase payment

Increase Patient Volume

Case paymentCase payment