![Page 1: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/1.jpg)
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this site.
Copyright 2007, The Johns Hopkins University and Kevin Frick. All rights reserved. Use of these materials permitted only in accordance with license rights granted. Materials provided “AS IS”; no representations or warranties provided. User assumes all responsibility for use, and all liability related thereto, and must independently review all materials for accuracy and efficacy. May contain materials owned by others. User is responsible for obtaining permissions for use from third parties as needed.
![Page 2: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/2.jpg)
Study Designs
Lecture 3
Kevin Frick
![Page 3: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/3.jpg)
Outline
• What do we call a cost-effectiveness expert?
• Carry over of epidemiological and statistical concepts into economic evaluation
• Study designs• Modeling• Examples
![Page 4: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/4.jpg)
10 Things We Might Call An Economist Specializing in Health-
Related Cost-Effectiveness?
![Page 5: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/5.jpg)
1 Interpretation of the #1 Thing to Call an Economist Specializing in Health-related Cost-Effectiveness
![Page 6: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/6.jpg)
Epidemiological, Statistical, and Economic Concepts
• Reliability• Validity
– Internal – External
• Bias• Precision
![Page 7: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/7.jpg)
Reliability
• Generally has to do with reproducibility• Reliability of economic measures is no
different an issue than reliability of other measures– Key is that we now have not only one
construct associated with our primary outcome but at least two constructs
• Effectiveness and costs
![Page 8: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/8.jpg)
Internal Validity
• Generally, is a study designed to appropriately measure a difference in outcome between treatment groups?– In cost-effectiveness
• Is the study designed to appropriately measure a different in costs?
• Is the study designed to appropriately measure a different in outcomes?
![Page 9: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/9.jpg)
External Validity
• Can we generalize findings from a study?– Can costs be generalized?– Can effects be generalized?
![Page 10: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/10.jpg)
Bias
• Bias in one measure can lead to incorrect inferences of something being different when it is not or something not being different when it is
• Bias in a combination of measures can have ambiguous effects on the cost-effectiveness ratio
![Page 11: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/11.jpg)
Effects of Bias
• Assume that one intervention is more expensive and more effective– Bias toward larger difference in costs and
smaller difference in effectiveness• Higher ratio
– Bias toward smaller different in cost and larger difference in effectiveness
• Smaller ratio– Bias for costs and effects in same direction
• Ambiguous effect on ratio
![Page 12: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/12.jpg)
Precision
• More precise measure associated with a smaller confidence interval in general study
• More precise measures in cost-effectiveness– Effect on estimate of confidence interval of
ratio is less clear– Concern about covariation as well as variation
![Page 13: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/13.jpg)
Designs
• Cost-effectiveness alongside a clinical trial run to measure effectiveness
• Cost-effectiveness trial• Epidemiological model
![Page 14: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/14.jpg)
Cost-Effectiveness Alongside an Effectiveness RCT
• Often referred to as piggy-backing the economic evaluation
• Take advantage of fixed costs of developing and running an RCT
• Opportunity for excellent internal validity• May lack external validity
– Patients in RCT are probably atypical
![Page 15: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/15.jpg)
Special Type of RCT
• Multi-center trials may have even more threats to external validity of economic evaluation– Varying local costs– Varying local styles of practice– Varying populations– Varying expectations for quality of life– Possibly even international variation
![Page 16: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/16.jpg)
Multi-Center RCT and Cost-Effectiveness
• Framing a study becomes exceptionally important in this case– Are we interested in the variation among
sites?– Are we interested in a specific type of
practice?– Are we interested in a specific price structure?
![Page 17: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/17.jpg)
Cost-Effectiveness Trial
• Efficacy of an intervention demonstrated previously
• Design a trial specifically to collect cost-effectiveness data
• May be able to use more valid, reliable, unbiased, and precise measures because resources are dedicated specifically to cost-effectiveness assessment
• Aim for higher external validity
![Page 18: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/18.jpg)
Epidemiological Model
• Use past results or publicly available data• Probabilities• Prices• Quantities• Prevalence• For screening-sensitivity and specificity
![Page 19: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/19.jpg)
Combination
• May have a trial that covers several parts of a process we would like to model but then need to extend the model further
Likely significantincrease in riskof event withoutintervention
Enter exercisetrial
End of follow-upmeasures ofclinical changes
End of lifeexpectancy
![Page 20: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/20.jpg)
Depiction of a Model
• Model is often depicted as a “decision tree”
• Decision trees include choices, random events, sometimes repeated random events, and “payoffs”– Payoffs can be negative (costs or health
decreases) or positive (effects or cost savings)
![Page 21: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/21.jpg)
Choice Node
• More than one path that is chosen by a policy maker, a physician, or a patient
• Generally depicted by a square with “branches”
Screen
Yes
No
![Page 22: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/22.jpg)
Chance Node
• More than one possible event• Generally depicted by a circle with
“branches”
Flu?
Yes
No
![Page 23: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/23.jpg)
End Node
• Last item represented in a sequence of events
• Does not necessarily have to be the “final event in a person’s lifetime”
• Generally represented as a triangle and describe payoffs
Flu?
Yes
No
![Page 24: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/24.jpg)
“Advanced” Chance Node
• Build a “Markov model”• Repeated sequence of events• Often represented similarly to a chance node,
but with an “M” inside• Example
– Cancer free person at risk for progressing to cancer each year
• Can remain cancer free• Can have an incident case of cancer• May die
![Page 25: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/25.jpg)
Data Analysis – Point Estimate
• Difference in mean costs• Difference in mean effects• Cost-effectiveness ratio
– (C1-C2)/(E1-E2)• Incremental Net Benefit
– (B2-C2) – (B1-C1)
![Page 26: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/26.jpg)
Data Analysis – Bootstrapping
• Use data collected– Draw a sample that is the size of the original
study• Sample with replacement
– Conduct analysis after random draws– Repeat and describe distribution of economic
evaluation results
![Page 27: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/27.jpg)
Data Analysis - Simulation
• Describe distributions to software– Draw parameters from distributions– Perform analyses– Repeat and describe distribution of economic
evaluation results
![Page 28: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/28.jpg)
Examples
• Surgery for dysfunctional uterine bleeding• Intervention to increase breastfeeding
duration among low income mothers• Screening preschoolers for vision
disorders
![Page 29: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/29.jpg)
Surgery for Dysfunctional Uterine Bleeding
• Multi-center randomized clinical trial• Originally designed as an effectiveness trial but not a
cost-effectiveness trial– Randomization between two alternative surgeries– Performed at over two dozen centers– Measures actual resources used
• 24 month follow-up– Main “events” of interest were time to return to usual activities,
need for follow-up treatment, and relief of symptoms• Gather QOL and resource utilization data• Assign costs• Would like to model out to menopause or entire lifetime
– Need epidemiological, QOL, and cost data from other sources
![Page 30: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/30.jpg)
Increasing Breastfeeding• Study designed primarily for cost-effectiveness• Randomization at two hospitals in one city with
one set of staff• Community-based intervention
– Had been shown to be effective in pilot work– Is pilot work externally valid over time?– Is pilot work externally valid for different personnel?
• Focus on infancy• Gather resource utilization data• Use publicly available data on costs• Could model mothers’ and children’s lifetimes
![Page 31: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/31.jpg)
Vision Screening in Preschoolers
• Build on study that conducted multiple screening activities and gold standard exams on the same set of children– Compare 4 screening devices and two types of
personnel• Use point estimates and standard errors of
sensitivity from study• Use distribution of times taken for screening
from study• Take specificity as fixed• Use publicly available resource values
![Page 32: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/32.jpg)
Vision Screening Model - I
![Page 33: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/33.jpg)
Vision Screening Model - II
![Page 34: Lecture 3 - Johns Hopkins Bloomberg School of Public Health](https://reader031.vdocuments.site/reader031/viewer/2022021305/620736b949d709492c2f14bd/html5/thumbnails/34.jpg)
Vision Screening Model - III
• Is it appropriate to end at “confirm and receive follow-up care” for the true positives rather than trying to develop a full model over the course of a lifetime– Yes because intervention does not affect
anything after treatment is initiated