question development jane beehler, mlis amy faulkner, mist · training? who else might be able to...
TRANSCRIPT
Healthy People, Healthy Places
Question Development Jane Beehler, MLIS
Amy Faulkner, MISt
November 2018
Welcome
Approx. 2 hours
Break when needed
Slides available in print & email
Safe Space - Ask questions!
Local Statistics, Surveillance Data
Published & Unpublished
Current political climate
Financial Resources
Reproduced with permission by NCCMT National Collaborating Centre for Methods and Tools. (2012). A Model for Evidence-Informed Decision-Making in Public Health. [fact sheet]. Retrieved from http://www.nccmt.ca/pubs/FactSheet_EIDM_EN_WEB.pdf
EIDM Model
Evidence Informed Decision Making in Public Health
7 Stages of EIDM
The National Collaborating Centre for Methods and Tools (NCCMT) identifies seven stages of EIDM. For more information on each stage please visit http://www.nccmt.ca/eiph/index-eng.html
………………………Clearly define your question or problem
………………...Critically appraise the research sources
………………Synthesize & form recommendations
……………………….Adapt the information to a local context
…..Decide whether (and plan how) to implement the evidence into practice or policy
…Evaluate the effectiveness of the implementation efforts (and disseminate your findings)
....…………………Efficiently search for research evidence
Define
Search
Appraise
Adapt
Synthesize
Implement
Evaluate
Today’s Focus
Today’s Goals
1. Things to consider before selecting a
review type
2. Understand the importance of developing
a research question
3. Introduction to concept mapping and
frameworks
Choosing a Review Type
Scoping
ReviewsUmbrella Reviews
Mixed-Methods Reviews
Meta-Ethnography
Narrative Reviews
Meta-Analysis
Rapid Realist ReviewsMeta-Syntheses
Mapping Reviews
Cochrane Reviews
Realist Reviews
Rapid Reviews
Living Systematic Reviews
Regardless of review type – being systematic in your approach, and transparent with your methods provides credence to your work Adsfadsf
aadf
Before You Start…
1. Do I have the necessary resources (time, staff, access to software)?
2. Do I have the skill level required? (Do I need support? Training? Who else might be able to assist with this project?)
3. What has already been done / is being done on this topic?
4. What is this review trying to answer? (To Describe, To Analyze, To Explore, To Prove)
5. How will the research impact health unit activities?
6. What will the final product be? Who is the audience? (i.e. Briefing Report, Article, Program Implementation, Presentation, etc.)
7. Will the final product be disseminated? (How widely? Forwarded to colleagues provincially? Where will the information live
internally?)
8. What type of data will be included? (Statistics? Peer Reviewed Articles? Grey Literature? Graphics?)
9. What happens if there isn't any research available?
10. What happens if the research is in opposition to (community, health unit, individual) beliefs?
11. Do I have any preconceived ideas or biases?
12. What is the risk? (Reputation?, Academic?, Financial?, Time?)
Before starting a research project and selecting your review type, consider the following questions :
Risks?
ReputationWill it undermine relevance of your results?Impact your conclusions and actions?Question the health unit as a source of credible information?
Academic
How much time and resources will go into completing the review? How much time may be lost if the review cannot be utilized to provide a recommendation?
Adopting the wrong review type
Finance
Time
Jeopardize your publication efforts?
What is the financial impact of the decision being made from your research? Are you implementing a costly program?
Strengths & WeaknessSample Review Types Strengths Weaknesses Most Commonly Used For:
Background Reading Enhance personal knowledge on a topic
High bias, low methodological rigor
Gathering information before beginning a review or for verifying a single fact
Rapid Review Aim to be rigorous and explicit in method but make concessions to breadth or depth by limiting aspects of the review process
Pending which concessions are made, fast-tracking may result in publication bias, poor quality assessment or overlooked inconsistencies in synthesis
To inform policy and program decision making
Systematic Review Seeks to draw together all known knowledge on a topic area
Requires adherence to strict methodologies and may take long periods of time and resources to complete
Complex search questions (i.e. why a particular intervention is effective)
Decision Making Confidence Scale
Background Reading
Rapid Reviews
Systematic Reviews
Low
High
Groupwork
Question: Which Review Type is appropriate?
Using the scenarios provided on the handout, discuss which
sample review type should be used.
What are some of the issues you discussed? Challenges?
Question Development
• Be sure you ask a question (do not make a statement)
• What type of question are you asking? (qualitative, quantitative)
• Are you asking a question about: effectiveness, cost/economic burden, adverse effects, diagnosis, policy, user experiences, risks, protective factors, etc.?
• Only ask one question at a time (some large research projects may have sub-questions)
• Document your reasoning (remind yourself why you made a decision)
FINER Description
Feasible
Interesting
Novel
Ethical
Relevant
Refining Your Research Question (FINER)
Is the scope of your question matched to the timeframe?
Interesting [to you] as a researcher or collaborator?
New findings or extension of previous findings? Guidance from mentors and experts
Following ethical guidelines? Regulatory approval from Institutional Review Board?
Influence on practice? Does it further research and/or health policy?
Hulley S, Cummings S, Browner W, et al. Designing clinical research. 3rd ed. Philadelphia (PA): Lippincott Williams and Wilkins; 2007.
Sceptic Description
Significant
Clear
Ethical
Parsimonious
Timely
Imaginative
Contextualized
Refining Your Research Question (SCEPTIC)
Source: Dr. David Naylor DCH Class 2002
Does the answer to your question make a difference? Will it help you write the report?
Is each term in the question defined or definable?
Are you following ethical guidelines? Do you have approval from Institutional Review Board?
Are you working from a general question to specific sub-questions and hypotheses?
Is the scope of the question matched to your resources and time frame?
Are you thinking creatively about how to answer the question?
How will the literature be used? What type of knowledge are you trying to access? (user knowledge, practitioner knowledge?)Which populations will the question be applicable or generalizable to?
Really, what’s the worst that can happen?
An ill-formed question can result in:• An inconclusive answer• Irrelevant results• A biased answer• Confusion during title/abstract
screening• Inconsistent team approaches to
synthesizing the literature • Wasted time • Poor final product
Remember: the question period can be iterative until you start searching
Concept Mapping
• Can be applied to any question• Best for visual learners• Helps focus your question and
parameters• Generates group discussion &
consensus • Brainstorm tool – get creative!
Example
Topic: STI prevention
Campaigns
STI Infections
At-Risk
Populations
Question: Do STI prevention campaigns reduce rates of STI infection in at-risk populations?
Indigenous
YouthLGBTQ2IA
ABC
Incarcerated
New Canadians
In-Person
Counselling
Presentation
Distributing Condoms
STI clinic
Schools
Community
Chlamydia
HIVGonorrhea
Messaging
Getting Tested
Protection
Partner Notification
MethodsTransmissionIncidence / Rates
Health Effects
Ads
Print Poster
Social Media
Online
Delivery
Example
Topic: STI prevention
Campaigns
STI Infections
At-Risk
Populations
Question: Do STI prevention campaigns reduce rates of STI infection in at-risk populations?
Ads
Print Poster
Indigenous
YouthLGBT
ABC
Prisoners
New Immigrants
In-Person
Counselling
Presentation
Distributing Condoms
STI clinic
Schools
Community
Chlamydia
HIVGonnoreah
Messaging
Getting Tested
Protection
Partner Notification
MethodsTransmissionIncidence / Rates
Health Effects
Social Media
Online
Delivery
In the next 5 minutes…
Using the example below, draft a research question.
Be prepared to share with the group
I want to know if vision screening in schools improves grades.
Concept Mapping Practice
Does vision screening in schools improve grades?
Vision Screening
Frameworks
Manage and break down research questions
Help identify key concepts in your question
Set the foundation for your search strategy
Determine your inclusion and exclusion criteria
What is PICO?
PICO
P
I
C
O
Patient or Population
Intervention (exposure, prognostic factor, or test)
Control /Comparison/Comparator
Outcome you would like to measure or achieve
The Many Variations of PICOFramework Components
PICO Population, Intervention, Control/Comparison/Comparator, Outcome
PICOT Population, Intervention, Comparison, Outcome, Time
PICOC Population, Intervention, Comparison, Outcome, Context
PO Population/Phenomena Outcome
PESICO Population, Environment, Stakeholders, Intervention, Comparison, Outcome
EPICOT Evidence, Population, Intervention, Comparison, Outcome, Timestamp
PICOTT/PICOTS
Population, Intervention, Comparison, Outcome, Type of question, Type of study designPopulation, Intervention, Comparison, Outcome, Study type
PECODR Problem, Exposure/Intervention, Comparison, Outcome, Duration, Results
PISCO Population, Intervention, Setting/Comparison, Outcome
PIPOH /S Population, Intervention, Professionals, Outcome, Healthcare/Setting
PCC Population, Concept, Context
Adapted with permission from:Pach B, Massarella S, Sharma M. To PICO or not to PICO: what is the question? Frameworks for developing answerable research questions [Internet]. Presented at: PHO Grand Rounds. 2016 Jun 7 [cited 2018 Nov 8]; Toronto, ON.
Even More FrameworksFramework Components
ECLIPSe(formerly CLIP)
Expectation, Client group, Location, Impact, Practitioner/Professional Service
SPICE Setting, Perspective, Intervention, Comparison, Evaluation
SPIDER* Sample, Phenomenon of Interest, Design, Evaluation, Research type
CIAOClient characteristics, Intervention, Alternate intervention, OutcomeContext, Interaction, and Outcome
PEO Population and their problem, Exposure, Outcome and themes
PS Population, Situation
MIP Methodology, Issues, Participants
PIE Patient/Problem/Population, Intervention/Issue, Effect/Evaluation
CIMO Context, Intervention, Mechanism, Outcome
PCC Population, Context, Comparison
Adapted with permission from:Pach B, Massarella S, Sharma M. To PICO or not to PICO: what is the question? Frameworks for developing answerable research questions [Internet]. Presented at: PHO Grand Rounds. 2016 Jun 7 [cited 2018 Nov 8]; Toronto, ON.
Frameworks by Discipline or Study TypeFramework Discipline Type of research
PICO, PICOTT, PICOT, PICOC, PECODR, EPICOT, PO, PS, EPICOTVarious (largely based on clinical settings)
Quantitative
PIPOH Guidelines Quantitative
PESICO Speech Language Pathology Quantitative
PISCO Public Health Qualitative/Mixed methods
ECLIPSE (formerly CLIP) Health policy & management Evaluation
Concept mapping Any Any
SPIDER, PIE Social Sciences Qualitative
CIAO Social Work Qualitative/Mixed methods
SPICE Social Sciences Mixed methods
PEO, PO, PS Various Qualitative
CIMO Management / organization Qualitative / Mixed methods
MIP Medical ethics Qualitative
Adapted with permission from:Pach B, Massarella S, Sharma M. To PICO or not to PICO: what is the question? Frameworks for developing answerable research questions [Internet]. Presented at: PHO Grand Rounds. 2016 Jun 7 [cited 2018 Nov 8]; Toronto, ON.
Today’s Focus
• PICO Framework
• PISCO Framework
• SPIDER Framework
Reminder: this is an introduction to frameworks – ask questions and don’t
hesitate to reach out for support!
PICO
PICO
P
I
C
O
Patient or Population
Intervention
Outcome you would like to measure or achieve
Control /Comparison/Comparator
PICO
Patient or Population
Intervention
Control
Outcome
PICO Example
Question: Are breastfeeding rates higher in online post-natal support groups or traditional in-person support groups for adolescent mothers?
Topic: Breast Feeding Rates
Online post-natal support groups
In-person post-natal support groups
Breastfeeding Rates
Adolescent MothersConcept 1
Concept 2
Concept 3
Concept 4
PISCO
PISCO
P Population
I Intervention
S OR Setting (if appropriate) OR
C Comparison (if appropriate)
O Outcome you would like to measure or achieve
PISCO Example
PISCOPopulation
Intervention
Setting or
Comparison
Outcome
Topic: Older Adults and Physical Activity
Question: Does access to parks in cities impact the amount of physical activity of older adults (65+)?
Older Adults 65+
Access to Parks
Cities
Amount of Physical Activity
Concept 1
Concept 2
Concept 3
Concept 4
Concept 5
N/A
SPIDER
SPIDERS Sample (of the general population)
P I Phenomena of Interest (what is occurring)
D Design (study design that will influence analysis and findings)
E Evaluation (outcome measures)
R Research Type (qualitative, quantitative, mixed-methods, etc.)
SPIDER Example
Topic: Women Attending Prenatal Classes
Question: What are the perceptions of woman aged 35-45 who attend prenatal classes?
Sample
Phenomenon of Interest
Design
Evaluation
(outcome measures)
Research Type
Women age 35-45
Perceptions of Prenatal Classes
Questionnaires, surveys, interviews, focus groups, case studies, observations
Views or attitudes or opinions or perceptions or beliefs or feelings
Qualitative or Mixed Methods
Concept 1
Concept 2
Concept 3
Concept 4
Concept 5
Group Work: Frameworks
Step 1:
Match the question to the framework
Step 2:
Using the table, pull the main concepts from your question into
the appropriate columns
Does hand washing among healthcare workers
reduce hospital acquired infections?
Broader
PICO
Patient or Population Healthcare Workers
Intervention Hand Washing
Control No hand washing / Usual Practices
Outcome Reduced Infections
Narrower
PISCO
Population Healthcare Workers
Intervention Hand Washing
Setting Hospitals
Comparison No hand washing / Usual Practices
Outcome Reduced Infections
Expanding and
Narrowing
Put simply: the more concepts you search, the narrower your focus
Caution: narrowing too far may exclude relevant results
What if I Hate Frameworks?
Feeling overwhelmed? Not sure where to
start?
Trying to jam your question into a
framework?
Solution
Start by using a simple table and pull your main concepts into it
Concept 1 Concept 2 Concept 3 Concept 4 Concept 5
Main
Concepts
Recap
1. Consider the Type of Review
Helpful Tools: Before You Start… (list of questions)
RAFT assessment
2. Develop your Question
Helpful Tools: FINER or SCEPTIC tables
Concept Mapping
Frameworks
Stuck? Try (or request) a sample search using the main concepts in your question. Sometimes the results can help you clarify further your final question.
Next Step?
• You are now ready to begin
building your search strategy
• Comprehensive Keywords and
Subject Headings
• Translating a search into a
database
• Three R’s - Record, Retrieve
and Review
ReferencesGrant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information and Libraries Journal; 2009 June, 26(2):91-108.
Jewell ST, Foster MJ. Matching review type to research question: which review is right for you? [Webinar] Medical Library Association. February 27 2018.
Booth A. High jump or decathlon? Working out the appropriate review type for your research project [Webinar]. ScHARR, University of Sheffield. 2017
Booth A. Systematic approaches to a successful literature review. 2nd ed. Sage Publications, 2016.
Pach B, Massarella S, Sharma M. To PICO or not to PICO: what is the question? Frameworks for developing answerable research questions [Internet]. Presented at: PHO Grand Rounds. 2016 Jun 7 [cited 2018 Nov 8]; Toronto, ON.
University Campus Suffolk. Using PICO & PEO: developing your research question and search strategy [Internet]. Ipswich, UK: University Campus Suffolk; [2015] [cited 2018 Nov 8].
Methley AM, Campbell S, Chew-Graham C, McNally R, Cheraghi-Sohi S. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res. 2014;14:579.
Dobbins, M. (2017). Rapid review guidebook. Hamilton, ON: National Collaborating Centre for Methods and Tools.
Page MJ, McKenzie JE, Kirkham J, Dwan K, Kramer S, Green S, et al. Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. Cochrane Database Syst Rev. 2014(10):Mr000035.
Tricco AC, Antony J, Zarin W, Strifler L, Ghassemi M, Ivory J, et al. A scoping review of rapid review methods. BMC medicine. 2015;13:224.
Thank you!