knowing how good our searches are
TRANSCRIPT
Knowing how good our searches are: an approach derived from search filter development methodology
Sarah Hayman
8th International Evidence Based Library and Information Practice ConferenceBrisbane, July 2015
Smart Searching
• Introduction• Search Filters at CareSearch and
Flinders Filters• Developing the learning modules• Evaluation of the learning modules
The Importance of Searching
• Need for evidence• Impact of missing evidence• Volume of information• Complexity of sources
Search Filters• CareSearch• Flinders Filters• Some Search filters worldwide
– ISSG Search Filters Resource– BMJ Clinical Evidence– Canadian Agency for Drugs and Technologies in Health (CA
DTH)– HL WIKI International– McMaster University. Health Information Research Unit– Scottish Intercollegiate Guideline Network (SIGN)
A search filter …• is a validated search strategy of known performance
effectiveness• is designed for and built in a particular bibliographic
database• can be methodology- or subject-based• can be expressed as a URL and embedded in a web
page for quick reliable access to evidence• may be sensitive or specific
Using Search Filters• Understand the nature and purpose of the search
filter• Critically appraise the search filter (tools available at
e.g. ISSG Search Filters Resource) • Look at the terms in the search filter and use as a
starting point for your search• Make search filters available to your users as useful
tools for accessing evidence reliably• Note that search filters may not be appropriate for the
comprehensive searching required for systematic reviews
Example: the Heart Failure Search Filter
As a URL:http://www.ncbi.nlm.nih.gov/pubmed?term=((heart+failure[tw]+OR+ventricular+dysfunction,+left[mh:noexp]+OR+cardiomyopathy[tw]+OR+left+ventricular+ejection+fraction[tw])+AND+Medline[sb])+OR+((heart+failure[tw]+OR+left+ventricular+dysfunction[tw]+OR+cardiomyopathy[tw]+OR+left+ventricular+ejection+fraction[tw]+OR+cardiac+resynchronization[tw]+OR+LV+dysfunction[tw]+OR+left+ventricular+systolic+dysfunction[tw]+OR+left+ventricular+diastolic+dysfunction[tw]+OR+cardiac+failure[tw])+NOT+medline[sb])+AND+english[la]
As a PubMed query:
Smart SearchingExpert Advisory Group (EAG)
Subject expertsGold Standard Set
Sample setTerm identification
Term identificationValidation
Testing
https://sites.google.com/site/smartsearchinglogical/home
The Modules
Usage and Evaluation of the Smart Searching modules
• Direct feedback sought from site• Web usage statistics (Google Analytics)• User survey conducted April, 2015
Google Analytics Report: Audience Overview (June 19, 2015)
Google Analytics Report: Location(June 19, 2015)
User Survey (April, 2015) - 50 responses
Occupation
Would you recommend this site to a colleague?
User Survey (April, 2015) - 50 responses
Do you think you would use this approach for testing?
Have you applied any of these techniques in your searching practice?
User Survey (April, 2015) - 50 responses
Some Representative Comments
Functioning of the SiteSystematic and methodical approach
Clear and easy to use (13 others similar to this)
Template applicable across disciplines
While the content is useful- the constant arrow moving would not be appealing to busy clinicians or medical librarians
Time constraintsTime is the major factor, followed closely by access to the subject experts
it can possibly save me some time as I spend a lot of time in my job training and assisting health researchers in building effective literature searches
Time restraints, level of information need does not usually require that level of sensitivity (several like this)
Very useful, however not sure if I would have time to test every search in a real life work situation.
User Survey (April, 2015) - 50 responsesSome Representative Comments
Value of the SiteReassessing the way I approach things
The more knowledge/ideas we share about improving search techniques the more beneficial it is to the profession
I can see the value in being able to 'qualify' and measure my searching outcomes
Informative...I bet there are other librarians who, just like me, are not utilizing these techniques properly.
Adding those extra dimensions increases the robustness of our searching and helps to systematise the things we do
I tend to be more intuitive than systematic with my searches […] Reporting would force me to ensure consistency!
Seems great as a refresher for me but will also be really useful for staff training purposes
I don't think that librarians test their search strategy and I feel it is an important tool to argue our competence and relevancy, especially in private enterprise
It gives a measure of effectiveness that speaks for itself...numbers are extremely hard to dispute!
Quality of the information and instruction; Quiz to reinforce what just learnt
User Survey (April, 2015) - 50 responses
Two Interesting Comments
Keep this aimed at professional librarians. I've never seen a profession so bent on becoming obsolete. Professional librarians/expert searchers have unique skills that we should be promoting to incoming librarians and teaching the new generation of searchers.
I was not happy to see that you state that librarians are not subject experts. After many years of searching on specialist topics I would consider myself to have more knowledge about the thesaurus terms used by databases than most subject 'experts'.
Conclusion• We developed this resource because we believe that searches are
important and should be accorded the respect of a scientific approach.• The resource appears to be well used (4,484 unique users in first year,
from 72 countries)• 29.55% of total sessions were from Australia in the first year• Top six countries represent approx. 85% of all users in the first year
(Australia, US, UK, Canada, Ireland, NZ)• Bounce rate lower and session times longer amongst Australian users• Responses to the survey were positive (81.63% would recommend to a
colleague)• Need more information about reasons for using – can largely only
speculate beyond what the survey responses suggested• We will maintain and aim to improve the resource – all suggestions and
feedback welcomed
AcknowledgementsThank you to:
• Colleagues at CareSearch and Flinders Filters (Raechel Damarell, Mikaela Lawrence, Yasmine Shaheem, Jennifer Tieman)
• Members of the Smart Searching Advisory Group• Health Libraries Australia and Medical Director (formerly HCN)
for the Health Informatics Innovation Award 2012 that supported the development of Smart Searching
Sarah HaymanFlinders Filters, Flinders University
