Bridgingthe Science /Practice Gap:

The Promiseand Perils of Evidence-Based Treatment

WIPHL 3/08

Suppose you had a life-threatening illness and went to a physician who told you:

““I really don’t pay much attention to medical I really don’t pay much attention to medical research. I’ve been treating people like you research. I’ve been treating people like you for 30 years, and I know what works. for 30 years, and I know what works. Medical research isn’t all that relevant to my Medical research isn’t all that relevant to my practice, and besides I’m too busy to be practice, and besides I’m too busy to be reading journals.”reading journals.”

In behavioral health care: Practice has been guided by whatever Practice has been guided by whatever

approach the provider prefersapproach the provider prefers There has been no requirement to use There has been no requirement to use

science-based methodsscience-based methods Reimbursement has been linked to generic Reimbursement has been linked to generic

contextscontexts of care like “group therapy,” of care like “group therapy,” “evaluation,” and “inpatient treatment”“evaluation,” and “inpatient treatment”

What goes on behind closed doors has been What goes on behind closed doors has been left to professional judgmentleft to professional judgment

However . . . .However . . . .

Standards of care are changing

It is abundantly clear that not It is abundantly clear that not allall “treatment “treatment works”works”

> 1000 clinical trials published in addiction> 1000 clinical trials published in addiction Cities, states, and other funding sources are Cities, states, and other funding sources are

increasingly demanding the use of EBTsincreasingly demanding the use of EBTs Closer integration of behavior health with Closer integration of behavior health with

healthcare will apply same standardshealthcare will apply same standards

The writing is on the wall

Those who are not providing empirically supported treatment are going to have a harder time getting paid for their services

“Anything goes” is gone.

Six questions

WhyWhy use EBTs? use EBTs? WhichWhich are EBTs and who decides? are EBTs and who decides? WhereWhere can EBTs be used in services? can EBTs be used in services? WhenWhen should EBTs be used? should EBTs be used? How How do clinicians learn EBTs?do clinicians learn EBTs? WhatWhat are the potential pitfalls? are the potential pitfalls?

1. Why should we use EBTs?

All treatments are not created equalAll treatments are not created equal Treatment methods and therapists differ Treatment methods and therapists differ

widely in efficacywidely in efficacy It’s going to be required; be an early It’s going to be required; be an early

adopter rather than playing catch-upadopter rather than playing catch-up Closer integration with healthcareCloser integration with healthcare We owe it to our clientsWe owe it to our clients

The Change Point mission is to provide the most effective substance abuse, mental health, and domestic violence intervention treatment utilizing culturally competent, evidence-based approaches designed to give the best possible outcomes to our clients.

Change Point, Portland Oregon, 1998

CASAA shall operate only those treatments, services and programs for which there exists evidence of efficacy in the current scientific literature. Services with unproven efficacy will be designated as experimental procedures and offered only within the context of appropriately designed research to determine their efficacy.

University of New Mexico Center on Alcoholism,

Substance Abuse and Addictions Adopted 1994

2. Which are EBTs and how is that decided?

Few clinicians have the time or expertise to Few clinicians have the time or expertise to analyze hundreds of clinical trialsanalyze hundreds of clinical trials

Relies on reviews of the literatureRelies on reviews of the literature Two refinements to reduce bias in reviewsTwo refinements to reduce bias in reviews

Systematic Systematic reviewsreviews Meta-analysisMeta-analysis

What is admissible evidence?

1. Strongest evidence: Randomized clinical 1. Strongest evidence: Randomized clinical trialstrials

Well-designed randomized trials provide a persuasive, though imperfect, correction for human self-deception.

A hierarchy of evidence

1.1. Randomized clinical trialsRandomized clinical trials2.2. Quasi-experimental designs that control Quasi-experimental designs that control

for some sources of biasfor some sources of bias3.3. Correlational studies with systematic Correlational studies with systematic

observationobservation4.4. Case reports, professional opinion, and Case reports, professional opinion, and

“best practice” consensus guidelines“best practice” consensus guidelinesHow much evidence is enough for an EBT?How much evidence is enough for an EBT?

Consistency of evidenceConsistency of evidenceCross-site replicationCross-site replication

Agreement across ten reviews of substance abuse outcome studies

Documentation: Documentation:

Miller, W. R., Zweben, J. & Johnson, W. R. (2005). Evidence-based treatment: Why, what, where, when and how? Journal of Substance Abuse Treatment, 29, 267-276.

9 out of 10 reviews agree . . .

Cognitive-behavioral treatmentCognitive-behavioral treatment Community reinforcement approachCommunity reinforcement approach Motivational interviewingMotivational interviewing Relapse prevention (cognitive-behavioral)Relapse prevention (cognitive-behavioral) Social skill trainingSocial skill training

Less consensus on . . .

0 1 2 3 4 5 6 7 8

Behavioral Marital Therapy

Brief Intervention

Behavioral Self-Control

CRA+ Vouchers

12 Step Facilitation

Behavior Contracting

Self-Help Books

Methadone + Therapy

Number of Reviews

Methods shown in multiple trials to be ineffective

Educational lectures and filmsEducational lectures and films Exploratory psychotherapies Exploratory psychotherapies Undifferentiated counselingUndifferentiated counseling ConfrontationConfrontation Mandated AAMandated AA Time in milieu (inpatient/residential)Time in milieu (inpatient/residential)

Some Treatment Methods with No Controlled Studies CENAPS Relapse Prevention (Gorski)CENAPS Relapse Prevention (Gorski) Rational RecoveryRational Recovery Reality Therapy (Glasser)Reality Therapy (Glasser) Solution-Focused TherapySolution-Focused Therapy Spiritual CounselingSpiritual Counseling Transactional AnalysisTransactional Analysis Women for SobrietyWomen for Sobriety

Commonly Practiced?Commonly Practiced?

Minnesota ModelMinnesota Model ConfrontationConfrontation EducationEducation FilmsFilms General CounselingGeneral Counseling Group TherapyGroup Therapy Mandated AAMandated AA Milieu TherapyMilieu Therapy

The gap “could hardly be The gap “could hardly be larger if one intentionally larger if one intentionally constructed treatment constructed treatment programs from those programs from those approaches with the least approaches with the least evidence of efficacy”evidence of efficacy”

Miller, Wilbourne & Hettema (2003)Miller, Wilbourne & Hettema (2003)

Handbook of Alcoholism Treatment Handbook of Alcoholism Treatment Approaches: Effective AlternativesApproaches: Effective Alternatives

Why the wide gap?

Providers are rarely trained in EBTsProviders are rarely trained in EBTs ““School” allegiance or eclecticismSchool” allegiance or eclecticism

Disease model linked to less use of EBTDisease model linked to less use of EBT Low accountability – anything goesLow accountability – anything goes

Reimbursement is not linked to EBTReimbursement is not linked to EBT Licensure is not linked to EBTLicensure is not linked to EBT Training programs rarely teach EBTTraining programs rarely teach EBT No QA for delivering EBTNo QA for delivering EBT

Why Isn’t Practice Self-Correcting?

Surgeons vs. PsychotherapistsSurgeons vs. Psychotherapists Lack of reliable feedbackLack of reliable feedback CPE has little effect on practice CPE has little effect on practice Very low expectations in behavioral health Very low expectations in behavioral health Staff turnover; devaluing of addiction Staff turnover; devaluing of addiction

professionalsprofessionals

When Should EBTs Be Used?

Whenever possibleWhenever possible With more than 1000 controlled clinical With more than 1000 controlled clinical

trials in the literature for alcohol, tobacco trials in the literature for alcohol, tobacco and illicit drug use, it is no longer and illicit drug use, it is no longer defensible to say that there is too little defensible to say that there is too little research from which to draw conclusions.research from which to draw conclusions.

When would one not use an EBT?

When there is no EBT availableWhen there is no EBT available Solvent abuseSolvent abuse Specific comorbiditySpecific comorbidity

Draw on EBTs for similar problems Draw on EBTs for similar problems For example, EBTs for the separate For example, EBTs for the separate

concomitant problemsconcomitant problems

“Not my population” ?

Insufficient research available for efficacy Insufficient research available for efficacy of treatment methods with specific of treatment methods with specific populationspopulations

What then?What then? Draw on EBTs from other populationsDraw on EBTs from other populations Remove specific barriersRemove specific barriers Evaluate in your own populationEvaluate in your own population

Where EBT can be provided?

Individual Individual providerprovider level level ProgramProgram level level

Program policy to provide EBTProgram policy to provide EBT Necessary system changes to support the Necessary system changes to support the

use of EBTuse of EBT Treatment Treatment systemsystem level level

Beware unfunded mandatesBeware unfunded mandates System support for retrainingSystem support for retraining Complexity of quality assuranceComplexity of quality assurance

How do clinicians learn EBTs?

In order to provide EBTs, clinicians must In order to provide EBTs, clinicians must develop proficiency in themdevelop proficiency in them

Often EBTs were not learned during initial Often EBTs were not learned during initial trainingtraining

How Substance Abuse Clinicians Learn New Treatment Methods

Training36%

Colleague17%

Reading18%

Experience27%

Research2%

Informal Methods (Colleagues+Experience+Reading) = 62%

Heidi Erickson, UNM Master’s Thesis, 1999

What does it take to learn EBTs?

Limitations of CPE workshops (let alone Limitations of CPE workshops (let alone treatment manuals)treatment manuals)

Feedback and coachingFeedback and coaching Need for taping and supervisionNeed for taping and supervision HireHire staff who are competent in EBTs staff who are competent in EBTs Start NOW teaching EBTs to the next Start NOW teaching EBTs to the next

generation of providers!generation of providers!

Pitfalls with EBTs

Efficacy versus effectivenessEfficacy versus effectiveness Efficacy varies across sites and providersEfficacy varies across sites and providers Without QA monitoring, EBT policy simply Without QA monitoring, EBT policy simply

requires requires sayingsaying that you deliver EBTs that you deliver EBTs Clinician self-reported proficiency can be Clinician self-reported proficiency can be

unrelated to actual proficiencyunrelated to actual proficiency Program directors may be clueless about Program directors may be clueless about

what actually happens behind closed doorswhat actually happens behind closed doors

Problems with lists of EBTs

Arbitrary criteria (e.g. APA Division 12)Arbitrary criteria (e.g. APA Division 12) Need for continual updatingNeed for continual updating Limitations of available researchLimitations of available research OssificationOssification Inhibition of innovationInhibition of innovation What about unevaluated methods?What about unevaluated methods?

Effective until proven otherwise?Effective until proven otherwise?

Evidence-Based Relationships

Consistent evidence that substance abuse Consistent evidence that substance abuse treatment providers differ significantly in treatment providers differ significantly in effectivenesseffectiveness

Often the largest predictor of clients’ Often the largest predictor of clients’ outcome is the counselor to whom they outcome is the counselor to whom they were assignedwere assigned

Accurate empathy, as defined by Carl Accurate empathy, as defined by Carl Rogers, is a particularly strong predictorRogers, is a particularly strong predictor

Take-Home Messages

1.1. It makes a difference It makes a difference whatwhat we do we do2.2. It makes a difference It makes a difference how how we do it we do it

(and (and whowho does it) does it)3.3. We already know how to do better We already know how to do better

than we dothan we do4.4. Changing to EBTs is difficult; Changing to EBTs is difficult;

requiring it even moresorequiring it even moreso5.5. EBTs are learnableEBTs are learnable6.6. The real beneficiaries are our clientsThe real beneficiaries are our clients