aug 1, 2007 1 conducting cost-effectiveness analyses of behavioral interventions todd h. wagner,...
TRANSCRIPT
Aug 1, 2007Aug 1, 2007 11
Conducting Cost-Effectiveness Conducting Cost-Effectiveness Analyses of Behavioral Analyses of Behavioral
InterventionsInterventions
Todd H. Wagner, Ph.D.Todd H. Wagner, Ph.D.
Mary K. Goldstein, M.D. Mary K. Goldstein, M.D.
Aug 1, 2007Aug 1, 2007 22
AcknowledgementsAcknowledgements
Partial funding was through a grant from the Partial funding was through a grant from the National Cancer Institute (2PO1 CA 55112-National Cancer Institute (2PO1 CA 55112-05A1). 05A1).
Dr. Goldstein’s contribution was informed in part Dr. Goldstein’s contribution was informed in part by work conducted with a grant from the by work conducted with a grant from the National Institute on Aging (R01 AG15110).National Institute on Aging (R01 AG15110).
David Meltzer, Jodi Prochaska, Lisa Faulkner, David Meltzer, Jodi Prochaska, Lisa Faulkner, and Stanford University seminar participants and Stanford University seminar participants provided helpful comments.provided helpful comments.
Aug 1, 2007Aug 1, 2007 33
Outline of the TalkOutline of the Talk
Background on cost-effectiveness analysis Background on cost-effectiveness analysis (CEA)(CEA)
A bias in CEAs for behavioral A bias in CEAs for behavioral interventionsinterventions
How to fix the problemHow to fix the problem ExampleExample Study design considerationStudy design consideration
Aug 1, 2007Aug 1, 2007 44
Economic EvaluationsEconomic Evaluations
Most trials don’t include economic Most trials don’t include economic analysisanalysis
Economic analysis in only 0.2% of 50,000 Economic analysis in only 0.2% of 50,000 trials conducted between 1966-1988trials conducted between 1966-1988– Adams et al. Medical Care 30(3):231-43 Adams et al. Medical Care 30(3):231-43
Economic analysis in randomized clinical Economic analysis in randomized clinical trials is increasingtrials is increasing
Aug 1, 2007Aug 1, 2007 55
Dollars and SenseDollars and Sense
Economic studies inform decisionsEconomic studies inform decisions– formularyformulary– adoption of new technologyadoption of new technology– scope of benefitsscope of benefits– strategies for management of carestrategies for management of care– organization of health care organization of health care
Aug 1, 2007Aug 1, 2007 66
Cost Effectiveness RatioCost Effectiveness Ratio
1 0
1 0
C C
E EICER
Where C1 is the average cost of the intervention groupC0 is the average cost of the control groupE1 is the average effectiveness of the intervention groupE0 is the average effectiveness of the control group
Aug 1, 2007Aug 1, 2007 77
StandardizationStandardization
USPTF guidelines (Gold et al., 1996; USPTF guidelines (Gold et al., 1996; summarized in three JAMA articles)summarized in three JAMA articles)
Drummond et al. (1997)Drummond et al. (1997) Created, along with journal editors, Created, along with journal editors,
standards for bothstandards for both– MethodsMethods– ReportingReporting
Aug 1, 2007Aug 1, 2007 88
Whose Costs?Whose Costs?
Guidelines recommend a societal Guidelines recommend a societal perspective for costsperspective for costs
IncludeInclude– ProviderProvider– PayerPayer– PatientPatient
Aug 1, 2007Aug 1, 2007 99
EffectivenessEffectiveness
Guidelines recommend QALYsGuidelines recommend QALYs
Values both quality and quantity of lifeValues both quality and quantity of life
Each year of life is multiplied by a Each year of life is multiplied by a weighting factor (utility)weighting factor (utility)
Utilities measure the preference of Utilities measure the preference of different health states on a 0-1 scaledifferent health states on a 0-1 scale
Aug 1, 2007Aug 1, 2007 1010
Behavioral InterventionsBehavioral Interventions
CEA framework holds, but caveats…CEA framework holds, but caveats…
– Behavior change is a “slow” processBehavior change is a “slow” process
– Treat many to prevent a fewTreat many to prevent a few
– Use of intermediate outcomes (proxies)Use of intermediate outcomes (proxies)
Aug 1, 2007Aug 1, 2007 1111
Intermediate OutcomesIntermediate Outcomes Outcome is clinically relevant and predicts Outcome is clinically relevant and predicts
mortality or morbiditymortality or morbidity– Receipt of a mammogramReceipt of a mammogram– Substance use abstinenceSubstance use abstinence– Change in dietary fiberChange in dietary fiber
QALYs would require huge and/or very QALYs would require huge and/or very long studieslong studies
Aug 1, 2007Aug 1, 2007 1212
CEA with an Intermediate CEA with an Intermediate OutcomeOutcome
Sufficient for publicationSufficient for publication Hard to interpret ICERHard to interpret ICER
– Can’t easily compare two CEAs with different Can’t easily compare two CEAs with different intermediate outcomesintermediate outcomes
– Can’t compare CEA to other CEA from Can’t compare CEA to other CEA from another clinical areaanother clinical area
Sometimes only feasible approachSometimes only feasible approach
Aug 1, 2007Aug 1, 2007 1313
CEA with QALYsCEA with QALYs
Measure QALYs orMeasure QALYs or Translate intermediate outcome to QALYsTranslate intermediate outcome to QALYs
– Either build a model de novo or use an Either build a model de novo or use an existing modelexisting model
– Requires a lot of resourcesRequires a lot of resources Most useful, but most challengingMost useful, but most challenging
Aug 1, 2007Aug 1, 2007 1414
CEA and Behavior ChangeCEA and Behavior Change
CEA analysts treat behavior change as a CEA analysts treat behavior change as a dichotomous outcomedichotomous outcome
Partial behavior change is not the same as Partial behavior change is not the same as no behavior changeno behavior change
““Getting the person to recognize that they Getting the person to recognize that they have a problem is half the battle.”have a problem is half the battle.”
Aug 1, 2007Aug 1, 2007 1515
What’s Missing? What’s Missing?
Partial behavior change is missing from Partial behavior change is missing from current modelscurrent models– People who progressed in their process of People who progressed in their process of
changing their behavior but did not changing their behavior but did not successfully change their behavior at the end successfully change their behavior at the end of the study (Stages of Change) of the study (Stages of Change)
Aug 1, 2007Aug 1, 2007 1616
Partial Behavior ChangePartial Behavior Change
All behavioral interventions yield some All behavioral interventions yield some partial behavior changepartial behavior change
Amount depends on Amount depends on – the duration of the studythe duration of the study– baseline stage of changebaseline stage of change
Aug 1, 2007Aug 1, 2007 1717
Partial Behavior ChangePartial Behavior Change
Should we and can we value partial Should we and can we value partial behavior change in a CEA?behavior change in a CEA?
Aug 1, 2007Aug 1, 2007 1818
Should We?Should We?
NONO Intentions are not the Intentions are not the
same as observed same as observed behavior changebehavior change
Stage of change is Stage of change is flawedflawed
Too difficultToo difficult
YESYES Behavior change Behavior change
takes timetakes time Unobservable Unobservable
differences existdifferences exist Future benefits are Future benefits are
importantimportant Otherwise favors Otherwise favors
med/surg txmed/surg tx
Aug 1, 2007Aug 1, 2007 1919
Can We?Can We? Behaviors are like “value chains”Behaviors are like “value chains”
– A series of linked processesA series of linked processes– Interventions may be designed to improve a link Interventions may be designed to improve a link
(stage-matched design)(stage-matched design) Matching chemotherapy protocol to cancer stageMatching chemotherapy protocol to cancer stage
– Interventions may have differential effects on Interventions may have differential effects on different linksdifferent links
Stages of Change (TTM) Model by Prochaska Stages of Change (TTM) Model by Prochaska and DiClimenteand DiClimente
Aug 1, 2007Aug 1, 2007 2020
Stages of ChangeStages of Change People progress through successive stages until People progress through successive stages until
they change their behaviorthey change their behavior– PrecontemplationPrecontemplation: no self-recognition of a problematic behavior.: no self-recognition of a problematic behavior.
– ContemplationContemplation: self-recognition of a problem without action.: self-recognition of a problem without action.
– PreparationPreparation: Planning to change behavior soon.: Planning to change behavior soon.
– ActionAction: In the process of change.: In the process of change.
– Maintenance: Maintenance: adherence to the new behavior over time.adherence to the new behavior over time.
Art of behavioral interventions is to achieve Art of behavioral interventions is to achieve action, and hopefully maintenanceaction, and hopefully maintenance
Aug 1, 2007Aug 1, 2007 2121
Stages of Change and CEAStages of Change and CEA
Stages of change is critical for Stages of change is critical for interpreting CEA resultsinterpreting CEA results– Stage of change may be associated with Stage of change may be associated with
receptivity or motivationreceptivity or motivation– Intervention effects can vary by stage of Intervention effects can vary by stage of
changechange– Incremental cost-effectiveness ratio can vary Incremental cost-effectiveness ratio can vary
by stage of changeby stage of change
Aug 1, 2007Aug 1, 2007 2222
Integrating Stages of Change in a Integrating Stages of Change in a CEACEA
Aug 1, 2007Aug 1, 2007 2323
What’s Needed?What’s Needed?
Data on stages of changeData on stages of change
Probability of moving from partial to Probability of moving from partial to successful behavior changesuccessful behavior change– Note: these probabilities are not observed in Note: these probabilities are not observed in
the intervention groupthe intervention group
Aug 1, 2007Aug 1, 2007 2424
Two Step ProcessTwo Step Process Step 1Step 1
– Calculate ICERs by baseline stage of changeCalculate ICERs by baseline stage of change– This alone can provide much more useful and This alone can provide much more useful and
interpretable information for stage-matched interpretable information for stage-matched interventionsinterventions
Step 2Step 2– Estimate probability of people moving into successive Estimate probability of people moving into successive
stages of changestages of change– For example, % moved from precontemplation to For example, % moved from precontemplation to
contemplation and preparationcontemplation and preparation
Aug 1, 2007Aug 1, 2007 2525
Source of the ProbabilitiesSource of the Probabilities
Probabilities from the literature orProbabilities from the literature or The control group (if possible)The control group (if possible)
– Possible when control group gets usual carePossible when control group gets usual care– If 10% of precontemplators in the control If 10% of precontemplators in the control
group changed behavior, this is the probability group changed behavior, this is the probability for the modelfor the model
Aug 1, 2007Aug 1, 2007 2626
Hypothetical ExampleHypothetical Example
Aug 1, 2007Aug 1, 2007 2727
Randomized Controlled TrialRandomized Controlled Trial
2700 participants enrolled in 3 arms2700 participants enrolled in 3 arms– Post card reminder (n=900)Post card reminder (n=900)– Reminder phone call (n=900)Reminder phone call (n=900)– Personal motivational phone call (n=900)Personal motivational phone call (n=900)
Sample from managed care planSample from managed care plan Managed care organization wants to know Managed care organization wants to know
which reminder to usewhich reminder to use
Aug 1, 2007Aug 1, 2007 2828
Cost and EffectivenessCost and EffectivenessAvg Unit
Cost EffectivenessPostcard $3.50 23.6%Reminder call $19.00 34.2%Motivational call $24.00 33.0%
Incremental cost
Incremental Effectiveness ICER
Reminder call vs postcard $15.50 10.6% $147Motivational call vs postcard $20.50 9.3% $220Reminder call vs motivational call -$5.00 1.2% -$409
Aug 1, 2007Aug 1, 2007 2929
Report to MCOReport to MCO
MCO should choose between postcard or MCO should choose between postcard or reminder callreminder call
Aug 1, 2007Aug 1, 2007 3030
Transition ProbabilitiesTransition ProbabilitiesBegan in precontemplationFollow-up Stage PC RC MC
Precontemplation 63.3% 51.7% 46.3%Contemplation 25.0% 29.0% 16.7%Preparation 7.0% 13.3% 21.3%Action 4.7% 6.0% 15.7%
Began in contemplationFollow-up Stage
Contemplation 63.3% 42.7% 45.3%Preparation 7.0% 13.3% 21.3%Action 29.7% 44.0% 33.3%
Began in preparationFollow-up Stage
Preparation 63.3% 47.3% 50.0%Action 36.7% 52.7% 50.0%
Total Action at Follow-up 23.6% 34.2% 33.0%
Aug 1, 2007Aug 1, 2007 3131
Including Partial Behavior Including Partial Behavior ChangeChange
Note motivational call is now most Note motivational call is now most effective overalleffective overall
PC RC MCTotal Action at Follow-up 23.6% 34.2% 33.0%
Estimated action including partial behavior change 35.9% 45.6% 46.7%
Aug 1, 2007Aug 1, 2007 3232
Reminder Call vs postcard ICERBaseline Stage of Change
Precontemplation $384Contemplation $99Preparation $161
Motivational Call vs postcardBaseline Stage of Change
Precontemplation $127Contemplation $246Preparation $256
Motivational Call vs reminder callBaseline Stage of Change
Precontemplation $123Contemplation -$68Preparation -$312
ICERsICERs
No single strategy No single strategy is always is always preferred.preferred.
Motivational call Motivational call is dominated by is dominated by reminder call for reminder call for contemplators and contemplators and those in actionthose in action
Aug 1, 2007Aug 1, 2007 3333
Origin of These IdeasOrigin of These Ideas
Aug 1, 2007Aug 1, 2007 3434
Patient RemindersPatient Reminders
Mailed reminder vs no reminder for Mailed reminder vs no reminder for mammographymammography
A meta analysis of 16 studies A meta analysis of 16 studies – US studies Pooled OR=1.48 (participants US studies Pooled OR=1.48 (participants
come from provider files)come from provider files)
– Aust / NZ studies Pooled OR=5.57 Aust / NZ studies Pooled OR=5.57 (participants from voter lists)(participants from voter lists)
Wagner TH. The effectiveness of mailed patient reminders on mammography screening: a meta-analysis. Am J Prev Med. 1998;14(1):64-70.
Aug 1, 2007Aug 1, 2007 3535
CEA and Stage of ChangeCEA and Stage of Change
Fishman P. et al. Cost-effectiveness of Fishman P. et al. Cost-effectiveness of strategies to enhance mammography use. strategies to enhance mammography use. Eff Clin Pract. Eff Clin Pract. 20002000
Compared three alternative methods for Compared three alternative methods for increasing mammography screeningincreasing mammography screening– Reminder postcardReminder postcard– Reminder callReminder call– Motivational callMotivational call
Aug 1, 2007Aug 1, 2007 3636
Reminder postcard
Motivational call
Reminder call
Sample size 590 590 585
Efficacy 36.1% 50.3% 52.6%Women screened 213 297 308Women not screened 377 293 277
Total costs $3,551.80 $15,334.10 $12,413.70Incremental CER dominated $92.12
Fishman’s (2000) resultsFishman’s (2000) results
ICER varied by prior mammography status ICER varied by prior mammography status (i.e., maintenance)(i.e., maintenance)
Aug 1, 2007Aug 1, 2007 3737
Feeling Lucky?Feeling Lucky?
The implications of a CEA should not vary The implications of a CEA should not vary by who enters the trialby who enters the trial
Two randomized trials with same Two randomized trials with same interventionintervention– RCT 1: all participants are in preparationRCT 1: all participants are in preparation– RCT 2: all participants are in RCT 2: all participants are in
precontemplationprecontemplation
Aug 1, 2007Aug 1, 2007 3838
Clinical Trial DesignClinical Trial Design
If the effect of the intervention might vary If the effect of the intervention might vary by stage of changeby stage of change– Qx: Enroll people from all stages?Qx: Enroll people from all stages?– Enroll sufficient numbers in each stageEnroll sufficient numbers in each stage– Protect randomizationProtect randomization
Aug 1, 2007Aug 1, 2007 3939
ConclusionsConclusions
Subgroup analysis by stage of change may Subgroup analysis by stage of change may be critical for interpreting the CEAbe critical for interpreting the CEA
Partial behavior change is important and Partial behavior change is important and can affect interpretationcan affect interpretation
These methods are appropriate for stage-These methods are appropriate for stage-matched studiesmatched studies
Need to consider study design implicationsNeed to consider study design implications