systematic review module 1: refining key questions c. michael white, pharmd, fcp, fccp professor and...
TRANSCRIPT
Systematic Review Module 1: Systematic Review Module 1: Refining Key QuestionsRefining Key Questions
C. Michael White, PharmD, FCP, FCCPC. Michael White, PharmD, FCP, FCCPProfessor and DirectorProfessor and Director
University of Connecticut / Hartford HospitalUniversity of Connecticut / Hartford HospitalEvidence-based Practice CenterEvidence-based Practice Center
Speaker has no actual or potential conflicts of interest in relation to this activity
2
Learning ObjectivesLearning Objectives At the conclusion of this activity, the
successful learner will be able to– Describe what the Effective Healthcare
Program is– Describe how topics are selected for key
question refinement– Identify the importance of key questions– Describe the role of key informants in topic
refinement– Describe a longitudinal and transparent topic
refinement process
3
Effective Healthcare Program Principles
The work of the EHC program is intended to be– Relevant and timely
– Objective and scientifically rigorous
– Transparent with public participation
Whitlock EP, Lopez SA, Chang S, et al. Identifying, selecting, and refining topics. Rockville, MD: Agency for Healthcare Research and Quality. Methods guide for comparative effectiveness reviews; 2009 Apr.
4
Topic Refinement in the CER Process
Develop topic:
Identify and triage topics
based on appropriateness,
importance, desirability of new research/duplication,
feasibility, and potential impact.
Refine topic:
Identify patient, intervention, comparator,
outcomes, timing and setting for
each topic.
Review topic:
Prepare topic, search for and select studies, abstract data, analyze and
synthesize data, present findings.
Engage stakeholders in clarifying areas
for future research:
Identify future research needs to inform real-world
healthcare decisions
5
Topic RefinementTopic Refinement
Topics for refinement will be – Appropriate, high priority, feasible, not
duplicative, and of high potential value
A key feature of topic refinement is the formulation of key questions
Key questions are objective demonstrations of EHC program principles– Should reflect uncertainty stakeholders have
Decisionmakers, clinicians, patients, others
Whitlock EP. Identifying, prioritizing, and developing research topics within the Effective Healthcare Program. Rockville, MD: Agency for Healthcare Research and Quality. Methods guide for comparative effectiveness reviews; 2009 Apr.
6
Key Questions (I)
Key questions guide the entire systematic review process– Literature search
– Inclusion and exclusion criteria
– Types of data extracted
– Data synthesized and reported
Key questions must be– Clear and precise
– Relevant to stakeholdersWhitlock EP. Identifying, prioritizing, and developing research topics within the Effective Healthcare Program. Rockville, MD: Agency for Healthcare Research and Quality. Methods guide for comparative effectiveness reviews; 2009 Apr.
7
Key Questions (II) Key questions should not be posed because
– The answer is assumed to be known– There seems to be adequate literature to evaluate it
An understanding of the topic area is critical– Understand PICOTS (explained in next slide),
understand the topic – Background reading and key informants are critical to
understanding PICOTS Elements of PICOTS outlined in general form
during topic development– Comprehensive development of PICOTS is required
during topic refinement before key questions are posed
Whitlock EP. Identifying, prioritizing, and developing research topics within the Effective Healthcare Program. Rockville, MD Agency for Healthcare Research and Quality. Methods guide for comparative effectiveness reviews; 2009 Apr.
8
Key Questions DerivedKey Questions DerivedFrom Assessing PICOTSFrom Assessing PICOTS
Population—Who is being evaluated? Intervention—What intervention is being
evaluated? Comparator—What is the intervention
being compared with? Outcomes—What are the benefits and
harms being evaluated? Timing—What is the follow-up time? Setting—What are the settings of interest?
Whitlock EP. Identifying, prioritizing, and developing research topics within the Effective Healthcare Program. Rockville, MD: Agency for Healthcare Research and Quality. Methods guide for comparative effectiveness reviews; 2009 Apr.
9
Devise Analytic Framework Devise Analytic Framework (See Module 2)(See Module 2)
Intermediate Outcomes Maximal concentrations Minimal concentrations at steady state Area under the curve Average steady state concentrations Dose needed to control seizure Switchback rate from generic to innovator antiepileptic medication[Pharmacokinetic data sought in
epileptic patients only]
Final Health Outcomes Mortality Hospitalization Office/emergency department visits Composite of ambulance services, hospitalization, or emergency department visits for epilepsy Health-related quality of life Breakthrough seizures Frequency of seizures Secondary seizure injury (fracture, laceration, head injury, aspiration pneumonia) Status epilepticus Loss of driver’s license Loss of employment
Total adverse events Neurological adverse
effects Hypotension Rash Suicidal ideation Dizziness
(KQ 1)
(KQ 3)
(KQ 2)
Patients with epilepsy using generic versus innovator antiepileptic medication
White CM. AHRQ Topic Refinement Report, Submitted 7.15.09.
10
Key Informants (I) Key informants (stakeholders) serve a
variety of important functions:– Help formulate key questions that address
real-world dilemmas Key questions should be devised to address
important health care dilemmas, not focus on interests pertinent to researchers
– Provide context to help discern content area and applicability
– Ensure transparency in the process
Whitlock EP. Identifying, prioritizing, and developing research topics within the Effective Healthcare Program. Rockville, MD: Agency for Healthcare Research and Quality. Methods guide for comparative effectiveness reviews; 2009 Apr.
11
Key Informants (II)
Beliefs about advantages or disadvantages of various alternative treatments are an important target for exploration with key informants– Some beliefs will be based on long-term RCTs
– Others will be based on dogma, extrapolations from surrogate end points, pathophysiology of disease, or pharmacologic profiles of drugs Key questions addressing these areas can
profoundly affect practice
Whitlock EP. Identifying, prioritizing, and developing research topics within the Effective Healthcare Program. Rockville, MD: Agency for Healthcare Research and Quality. Methods guide for comparative effectiveness reviews; 2009 Apr.
12
Key Informants (III)
Clinicians– Clarify (sub)populations to focus on– Identify areas in which studies differ that
can affect applicability– Identify what contemporary practice looks
like Patients
– Provide insight not usually appreciated by other stakeholders May focus more on quality of life and harms
Whitlock EP. Identifying, prioritizing, and developing research topics within the Effective Healthcare Program. Rockville, MD: Agency for Healthcare Research and Quality. Methods guide for comparative effectiveness reviews; 2009 Apr.
13
Public Comment (I)
After key informants provide insight, a final draft of the topic refinement document is devised
This document will undergo public comment – Anyone with an opinion and access to a
computer can comment Public comment is a critical final check
of underlying assumptions about relevance and ensures transparency for all stakeholders
Whitlock EP. Identifying, prioritizing, and developing research topics within the Effective Healthcare Program. Rockville, MD: Agency for Healthcare Research and Quality. Methods guide for comparative effectiveness reviews; 2009 Apr.
14
Public Comment (II)
All public comments that can be answered with the existing Methods Guide are answered generically
All other public comments are answered individually by the EPC– Changes to the topic refinement document are
made or further explanations of why certain aspects of the topic refinement project are not being changed are laid out
Revised topic refinement document and dispensation of public comments are reviewed and approved
Whitlock EP. Identifying, prioritizing, and developing research topics within the Effective Healthcare Program. Rockville, MD: Agency for Healthcare Research and Quality. Methods guide for comparative effectiveness reviews; 2009 Apr.
.
15
Topic Refinement Process: UCONN/HH EPC Example (I)
Step 1: Core Leadership team reviews topic development document with title, preliminary PICOTS, brief review of what systematic reviews and studies exist, and topic refinement template– Topic refinement on treatments for epilepsy
Step 2: Core Leadership team selects a team from within the EPC that has knowledge, interest, and/or contacts within the topic area– Lead investigator, senior scientist, research fellow identified
Step 3: Selected team performs general review of the topic area, focusing on tertiary texts describing the disease process or interventions of interest and then any practice guidelines– General review on epilepsy conducted, presented by senior
scientist as a lecture with a neurologist invited
16
Topic Refinement Process: UCONN/HH EPC Example (II)
Step 4: Team reviews the topic development documents and continues more focused background reading
Step 5: Preliminary search strategy proposed– Approved by medical librarian and conducted
Step 6: Citations reviewed, body of literature identified, data on PICOTS extracted; NO RESULTS ARE REVIEWED
Step 7: Team determines PICOTS and analytic framework
17
Analytic FrameworkAnalytic Framework
White CM. AHRQ Topic Refinement Report. Submitted 7/15/2009.
Intermediate Outcomes Maximal concentrations Minimal concentrations at steady state Area under the curve Average steady state concentrations Dose needed to control seizure Switchback rate from generic to innovator antiepileptic medication[Pharmacokinetic data sought in
epileptic patients only]
Final Health Outcomes Mortality Hospitalization Office/emergency department visits Composite of ambulance services, hospitalization, or emergency department visits for epilepsy Health-related quality of life Breakthrough seizures Frequency of seizures Secondary seizure injury (fracture, laceration, head injury, aspiration pneumonia) Status epilepticus Loss of driver’s license Loss of employment
Total adverse events Neurological adverse
effects Hypotension Rash Suicidal ideation Dizziness
(KQ 1)
(KQ 3)
(KQ 2)
Patients with epilepsy using generic versus innovator antiepileptic medication
18
Topic Refinement Process: UCONN/HH EPC Example (III)
Step 8: Preliminary key questions posed Step 9: List of questions for key
informants posed Step 10: Key informants shown the
background, PICOTS, analytic framework, and preliminary key questions
Step 11: After 2 weeks, the topic refinement document discussed with key informants, questions posed
19
Topic Refinement Process: UCONN/HH EPC Example (IV)
Step 12: Based on feedback, revised topic refinement document generated
Step 13: Topic refinement document sent to AHRQ
Step 14: After editorial review, posted for public comment
20
Topic Refinement Process: UCONN/HH EPC Example (V)
Step 15: Public comment sent to the EPC, revisions made, comments responded to
Step 16: EPC sent final version to AHRQ for final approval and official posting
21
The topic refinement process is devised to ensure quality, clinical relevance, and transparency
Understanding the topic area is important to describing the PICOTS and devising the analytic framework
When studies are evaluated, the focus is on the methods and demographics, not the results
Key informants provide context and ensure relevance and transparency
Key informants cannot simply include researchers in the field
Public comment is a final verification of relevance and ensures transparency
Key Messages