shortening the distance to research evidence
TRANSCRIPT
Shortening the distance to research evidence
The librarian as knowledge translation activist
Raechel Damarell
Outline• Search filters
– Research– Implementation– Dissemination– Evaluation
• Flinders Filters: a new research initiative and opportunity for librarians
What is a search filter?• Tool to retrieve specific subset of literature • Much more than an expert search
– Developed experimentally following a strict (and usually transparent) research methodology
– Has a known level of performance in the database for which it was developed
• The InterTASC Information Specialists' Sub-Group Search Filter Resource
Why use a search filter?
Searching without a filter
Why use a search filter?Searching with a filter
Types of search filters• Methodological vs subject-based
– Cochrane HSSS; PubMed Clinical Queries
• 1st, 2nd, and 3rd generation – subjectively derived/effectiveness unknown– subjectively derived/effectiveness known– objectively derived/effectiveness known
Subject-based filters• Diagnostic imaging techniques• Heart failure• Nephrology• Return to work • Alcohol-impaired driving• Sleep• Ultrasound for DVT• Knowledge translation• Cardiovascular biology• Occupational origin of disease• Nurse staffing research• Palliative care
• Patient safety• Mental health• Spanish studies• Measurement properties of
measurement instruments• Administrative data studies• Primary health care• Occupation health
interventions• Evidence based nursing
Methodological filters• Adverse effects• Diagnostic studies• Economic evaluations• Epidemiological studies• Etiology• Guidelines• Health services research• Observational studies• Prognosis• Qualitative research• Quality of life• RCTs and other trials
• Systematic review• Therapy studies
Basic filter methodology• Establish an Expert Advisory Group
– Scoping conceptual boundaries– Preliminary consensus on potential search terms– Advice on Gold Standard Set
The Gold Standard Set• Set of representative citations of known
relevance indexed in the database of interest
• Target set: all citations in it should be retrieved by final filter
• Provides means of iteratively improving a search strategy
Gold Standard Set options• Journal ‘hand search’ (e.g. Palliative Care Filter)• Included references of:
– Clinical Practice Guidelines (Heart Failure)– Systematic Reviews (Lung Cancer)– All citations from range of specialty journals on topic (PHC)
• Others:– Delphi study method (Best Supportive Care)– Set retrieved by a search on the topic
Basic filter methodology• Establish an Expert Advisory Group • Division of Gold Standard Set
– Term Identification Set – Filter Development Set– Filter validation Set(s)
Performance measures
Sensitivity (or Recall)The number of relevant citations retrieved by the search filter from the gold standard as a proportion of the total number of relevant citations (100% = all relevant citations retrieved)
PrecisionThe number of relevant records retrieved as a proportion of the total number of records retrieved (100% = all citations retrieved relevant)
Specificity Number of irrelevant citations not retrieved (correct exclusions)
Basic filter methodology• Establish an Expert Advisory Group
– Scoping conceptual boundaries– Preliminary consensus on potential search terms– Advice on Gold Standard Set
• Division of Gold Standard Set– Term Identification Set; Filter Development Set; Filter
Validation Set(s)
• Implementation: making it available
Why PubMed?• Accessibility: Largest freely available
biomedical database • Currency: PubMed’s ‘in process’ and ‘as
supplied by publisher’ citations• Immediacy: Search filter can be saved as
hyperlink for one-click searching • Implementable: Can be embedded in any
webpage
The Palliative Care Filter• Developed in 2004 based on 1999-2001
dataset • Specificity = 99% but sensitivity <50% • Combined with 60+ topics of clinical
importance to palliative care clinicians• Subject of three published studies (4th
currently being written up)
What the users don’t see((advance care planning[mh] OR attitude to death[mh] OR bereavement[mh] OR terminal care[mh] OR hospices[mh] OR life support care[mh] OR palliative care[mh] OR terminally ill[mh] OR death[mh:noexp] OR palliat*[tw] OR hospice*[tw] OR terminal care[tw] OR 1049-9091[is] OR 1472-684X[is] OR 1357-6321[is] OR 1536-0539[is] OR 0825-8597[is] OR 1557-7740[is] OR 1552-4264[is] OR 1478-9523[is] OR 1477-030X[is] OR 0749-1565[is] OR 0742-969X[is] OR 1544-6794[is] OR 0941-4355[is] OR 1873-6513[is] OR 0145-7624[is] OR 1091-7683[is] OR 0030-2228[is]) AND Medline[sb]) OR ((advance care plan*[tw] OR attitude to death[tw] OR bereavement[tw] OR terminal care[tw] OR life supportive care[tw] OR terminally ill[tw] OR palliat*[tw] OR hospice*[tw] OR 1049-9091[is] OR 1472-684X[is] OR 1357-6321[is] OR 1536-0539[is] OR 0825-8597[is] OR 1557-7740[is] OR 1552-4264[is] OR 1478-9523[is] OR 1477-030X[is] OR 0749-1565[is] OR 0742-969X[is] OR 1544-6794[is] OR 0941-4355[is] OR 1873-6513[is] OR 0145-7624[is] OR 1091-7683[is] OR 0030-2228[is]) NOT Medline[sb]) AND English[la]
The Heart Failure Filter• Developed in collaboration with Centre for
Cardiovascular & Chronic Care (UTS) • Gold standard: Evidence based guidelines• Sensitivity: 97%; Precision: 75% • Methodology article published in
BMC Medical Research Methodology• Innovative methodology for PubMed translation
(article in progress)• Combined with 40+ topics of clinical importance to
heart failure/palliative care clinicians
The Heart Failure Filter• Developed in collaboration with Centre for
Cardiovascular & Chronic Care (UTS) • Gold standard: Evidence based guidelines• Sensitivity: 97%; Precision: 75% • Methodology article published in
BMC Medical Research Methodology• Innovative methodology for PubMed translation
(article in progress)• Combined with 40+ topics of clinical importance to
heart failure/palliative care clinicians
What the users don’t see((heart failure[tw] OR ventricular dysfunction, left[mh:noexp] OR cardiomyopathy[tw] OR left ventricular ejection fraction[tw]) AND Medline[sb]) OR ((heart failure[tw] OR left ventricular dysfunction[tw] OR cardiomyopathy[tw] OR left ventricular ejection fraction[tw] OR cardiac resynchronization[tw] OR LV dysfunction[tw] OR left ventricular systolic dysfunction[tw] OR left ventricular diastolic dysfunction[tw] OR cardiac failure[tw]) NOT medline[sb]) AND english[la]
Palliative Care Filter effectiveness studies
Study 1• Compiled more recent gold standard based on
included studies in 29 palliative care systematic reviews
• Found that Palliative Care Filter performed better in more recent dataset than in original one (˃ 5%)
Palliative Care Filter effectiveness studies
Study 2• Real life PubMed searches of pc clinicians for pc
literature (n=37) analysed • Each run in gold standard set and retrieval compared
with that of PCF• Results highlighted difficulties in searching:
– Very limited number of search terms used for complex topic– Incorrect search construction– Negative correlation between stated confidence and actual
performance (i.e. overconfidence is issue)– Search filter retrieved 68% of gold standard; average
clinician search retrieved 25%
Palliative Care Filter effectiveness studies
Study 3• Were palliative care clinicians aware of topic
searches built on the PCF?• Of those aware:
– Were they using them? If yes, what did they think? If no, why not?
• If not aware: – What did they think after using search? (intervention study)
• Follow up study– Were they still using searches after 6 weeks?
Flinders Filters
Flinders Filters
• Established in 2012 as Flinders co-funded research venture
• Committed to producing over 2 years:– Two health filters – Two non-health filters– Four research papers that make a
contribution to search filter methodology
Flinders Filters
Completed projects: • Primary Health Care Search Filter (PHCRIS)
Flinders Filters
PHC Search Filter launch
All welcome
Flinders Filters
New projects: • ‘Indigenous health’ and ‘Australian Aboriginal health’ filters
(Lowitja Institute)• Best Supportive Care (Duke University)• Dementia• Sexual health
Flinders Filters
Urgent issues for FF:• Non-health filters: What? Who with? Where
appropriate? • Finding time to write for publication • More librarians needed to join team
Thank you
Questions?