Matching the Research Design to the Study Question
Lisa Simpson, MD, MPHJuly 25, 2011
Overview Overview of AcademyHealth Matching the research to the need Matching the design to the question Implications for the CER enterprise
2
AcademyHealth: Improving Health & Health Care
As the professional society for health services researchers and health policy analysts, our mission is three-fold:
1. Support the development of health services research
2. Facilitate the use of the best available research and information
3. Assist health policy and practice leaders in addressing major health challenges
We work to both “push” the production of research and promote the “pull” by decision makers
3
In Other Words…
We help build the knowledge base AND
move knowledge into actionthrough programs and services for
research producers and research users.
4
Overview Overview of AcademyHealth Matching the Research to the Need
National Quality Strategy
5
National Quality Strategy
Secretary of HHS mandated by Affordable Care Act to develop a National Strategy for Quality Improvement in Health
Requires strategic plan and implementation metrics for national, state, and local efforts to improve health care quality on common aims, priorities and goals
6
National Quality Strategy: Three Aims
Better Care: Improve the overall quality, by making health care more patient-centered, reliable, accessible, and safe.
Healthy People/Healthy Communities: Improve the health of the U.S. population by supporting proven interventions to address behavioral, social and, environmental determinants of health in addition to delivering higher-quality care.
Affordable Care: Reduce the cost of quality health care for individuals, families, employers, and government.
7
National Quality Strategy: Six Priorities
Safer care Effective care coordination Person and family centered care Prevention and treatment of leading
causes of mortality Supporting better health in communities Making care more affordable
8
NQS and IOM: Priorities in Sync Safer care Effective care
coordination Person and family
centered care Prevention and
treatment of leading causes of mortality
Supporting better health in communities
Making care more affordable
Methods to eradicate MRSA and reduce HAI
Effectiveness of comprehensive care programs
None explicitly “patient-centered”
Primary prevention methods vs. clinical interventions
Effectiveness of dissemination and translation techniques
Reduce healthcare disparities including literacy
Localized prostate cancer management strategies
9
NQS and FCC CER Safer care Effective care
coordination Person and family
centered care
Prevention and treatment of leading causes of mortality
Supporting better health in communities
Making care more affordable
None explicitly on safety
Decision support tools
Focusing on immediate, specific patient needs; family as surrogate decision makers
Prevention
Focus on priority populationsDelivery system strategies,
including community-based models
10
National Quality Strategy Principles
1. Person-centeredness and family engagement
2. Specific health considerations
3. Eliminating disparities in care
4. Aligning the efforts of public and private sectors
5. Quality improvement
6. Consistent national standards
7. Primary care will become a bigger focus
8. Coordination will be enhanced
9. Integration of care delivery
10. Providing clear information
CER focused at the individual level, responsive
to patient needs
CER examining discrete &/or complex
comorbid conditions
Importance of sub group analyses, examination of context, & within group
heterogeneity
Enterprise dependent on public & private sources
Examination of system level effectiveness and co-occurring
interventions
Methodologic standards for priority setting, engaging patients
& research
Focus on real world setting and heterogeneous patient populations as
well as practice variations
Implementation and Dissemination CER
Importance of system level studies
11
So is this…
12
A brief (& Selected) History of Time…
1989AHCPR
1999AHRQ
2003MMA
2009ARRA
2009ACA
Outcomes PORTS
OutcomesCERTS
Quality & Safety
AHRQ EHC Program
Priority Conditions Payer focus Public input
CER enterprisePriority Questions
Methods & Standards
Stakeholder input
PCORIPublic/Private Funding
Priority QuestionsMethods & Standards
Stakeholder input
13
Over That Time…
A more than tripling of the NIH budget in 20 years
~$10 million in 1991 to $31 billion Growing understanding of the
many gaps in quality, outcomes Pervasive disparities The “inexorable cost curve”
Increasing demand for research ROI
For identified needs to drive funding
Improved methods for seeking public, user input
Increasing emphasis on priority setting
Focus on comparisons Expansion beyond discrete
clinical comparisons Focus on methodologic
standards
14
It is still wine, but…
15
CER Defining Characteristics
Objective of directly informing clinical or health policy decision
Compares at least 2 alternatives Results at population and subgroup level
16Tunis, 2009
“70 Kilo White Male”
17
CER Defining Characteristics
Objective of directly informing clinical or health policy decision
Compares at least 2 alternatives Results at population and subgroup level Measures outcomes important to patients Methods and data sources appropriate for the
decision of interest Conducted in real world settings
18Tunis, 2009
CER vs PCOR: is there a difference?
Patient-Centered Outcomes Research
Comparative Effectiveness Research
Comparisons
No patient reported data
Not comparative
Patient reported outcomes
Subgroups
19
So before we can match the study design to the question, we have to match the
question to the need.And that need varies depending on who
you ask and how you ask.
20
Overview Overview of AcademyHealth Matching the research to the need Matching the design to the question
21
Who is asking the question? Why?
Policymakers Coverage, delivery system regulations, payment, quality
Payers Coverage, payment, quality
Industry Regulatory requirements, impact on innovation
Providers Health care choices
Consumers/Patients Health care choices
22
The design depends on who is asking the question
Constructing relationships Constructing questions Constructing answers Constructing messages
(Translation)
Source: Zerzan, JT et al. “Improving State Medicaid Policies with Comparative Effectiveness Research: A Key Role for Academic Health Centers.” Academic Medicine, Vol. 86, No. 6. June 2011. Pgs 695-700.
Categories of CER Methods
Systematic reviews of existing research Decision modeling, with or without cost information Retrospective analysis of existing clinical or
administrative data– Significant new investments in data infrastructure
Prospective non-experimental studies, including registries
Experimental studies, including randomized clinical trials (RCTs)
Tunis, 2009
The skeptical, “purist” view...
[…] “if American medicine is to avoid the mistakes of high-dose chemotherapy, hormone therapy, antiarrhythmic agents, and numerous other failed interventions, that our profession adopted on the basis of inferior evidence.”
Source: Lauer, MS. “Will Academia Embrace Comparative Effectiveness Research?” Academic Medicine, Vol. 86, No. 6. June 2011. Pgs 671-673.
Matching the Designs
Source: Thorpe KE et al, Journal of Clinical Epidemiology, 2009
Matching the Designs
Pediatric quality demonstrations 10 grants, 18 states, 47 projects Multiple stakeholders
– CMS– State policymakers– Health plans– Providers– Patients & families
27
Considerations
Degree of harm– Individual morbidity & mortality, versus population outcomes
Amount of existing evidence– System level decisions even more “evidence free”
Market entry vs prevalent practice Measuring the fidelity &/ adaptation of the intervention Opportunity to reconsider the choice
– Mid course corrections– Coverage with evidence development– Lifecycle approach to evidence development
28
Matching the Design to the Question
Observational Studies
RCTs
Balance
Rigor
Relevance
Timeliness
29
All methods have a role
Inevitable tradeoff between internal validity and feasibility, generalizability, cost, time
The nature of the research question, and the decision maker will influence best practices
Experimental studies will have a crucial role in CER, and there is need for improving design and implementation
Non-experimental methods hold great promise, particularly as methods are refined and data infrastructure is improved
Tunis, 2009
31
Overview Overview of AcademyHealth Matching the research to the need Matching the design to the question Implications for the CER enterprise
32
An approach that advances a balanced portfolio of activities including:
Variety of study designs Evidence generation Evidence translation
Methodologic research Infrastructure enhancements
Data Workforce(s)
An approach that builds in “meta-learning”
A high performing evidence system
Source: Adapted from Van Lare, Conway & Sox, NEJM, 2010
33
Questions?
Lisa Simpson, MB, BCh, MPH, FAAPPresident and CEO