search strategies and data sources for adverse effects reviews...• adverse effects search terms or...

Centre for Reviews and Dissemination

Search strategies and data

sources for adverse effects

reviews

Su Golder [email protected]

Liliane Zorzela [email protected]

mailto:[email protected]

mailto:[email protected]


• Research referred to in this workshop was undertaken as part of an MRC fellowship

• The views expressed in this presentation are those of the author and not necessarily those of the MRC

• CRD is part of the National Institute for Health Research (NIHR) and is a department of the University of York, UK


Introductions

• Background

– Any information specialists?

– Reviewers?

• Experience of adverse effects reviews?


Format of the workshop

• Introduction (10 mins)

• Group work using real systematic reviews

(30 mins)

• Feedback and discussion (20 mins)

• What the research evidence tells us about

the most effective search strategies and

data sources (20-30 mins)


Why do adverse effects matter?

Definition

• ‘A harmful or undesirable outcome that occurs during or after the use of a drug or intervention for which there is at least reasonable possibility of a causal relation’ (Chou 2010)

Why adverse effects matter

• Unpleasant, often serious – hospitalisation, disability, death (USA: 4th to 6th leading cause of death) (Lazarou 1998)

• Worsen quality of life, make people stop treatment

• Cost (estimates of cost to UK NHS of £2 billion per year) (Compass 2008)


Why should we do systematic

reviews of adverse effects?

• Need to assess benefit/ harm balance

- Considering benefits alone leads to bias

• Detailed evaluation needed when:

- Narrow margin between benefit and harm

- Other effective treatments have different or unknown

safety profiles

- When adverse effects make people stop treatment


Why is searching for adverse

effects difficult?

• AEs often treated as

secondary or even tertiary

outcomes. Poor reporting

in titles and abstracts and

indexing

• Inconsistent terminology

and indexing

• False hits; ‘Relative Risk’,

‘Self-harm’, ‘Patient safety’,

‘adverse effects were not

considered’

• May wish to identify all adverse

effects. Hard to predict/plan (may

not know which adverse effects

searching for)

• Relevant adverse effects may

come from a range of study

designs, not just RCTs

• Many data sources: specialist

databases, unpublished data,

industry funded data, surveillance

data, tertiary sources


Group work

• Two scenarios

• Work out what data you want, and how

you are going to find it

• Small groups to discuss (one scenario

each)

• Should take 30 minutes

• Followed by open discussion


Group work

Scenario A: Review of a specific adverse effect

Long-term use of glitazones and fractures in type 2 diabetes

Scenario B: Review of all adverse effects (safety profile review)

Adverse effects of newer drugs (gabapentin, lamotrigine, levetiracetam, oxcarbazepine, tiagabine, topiramate, vigabatrin) for epilepsy in adults


Points to consider

• Search strategy design

- PICO (patient group, intervention,

comparison, outcome)

- Restrictions (language, date, study design)

• Where to search

- Which databases

- Other sources


Search strategy design

Scenario A: Review of a specific adverse effect

Long-term use of glitazones and fractures in type 2 diabetes

– P (patient group) - ?

– I (intervention) - ?

– C (comparison) - ?

– O (outcome) - ?


PICO

• P (patient group) - people with type 2 diabetes

• I (intervention) - glitazones

• C (comparison) - any

• O (outcome) - fractures


PICO – Patient Group

• Should we limit to type 2 diabetes?

OR

• Should we include fractures with glitazones in other groups?

• Some titles and abstracts may not mention the patient group (type 2 diabetes)


Example patient group terms

• Freetext • Diabet*, NIDDM, MODY, non insulin* depend*,

noninsulin* depend*, non insulin*depend*, noninsulin*depend*, T2DM, type II DM

• MeSH • exp Diabetes Mellitus, Type 2/

• EMTREE • non insulin dependent diabetes mellitus/


PICO - Intervention

• Intervention terms may not be included in title and

abstract

• Most sensitive search strategy (for breast cancer and

oral contraception) in Wieland et al 2005 did not include

intervention terms:

- “human” [MESH] AND journal article [pt] AND breast neoplasms

[majr:noexp] AND (risk [mh:noexp] OR risk factors [mh:noexp]

OR follow-up studies [mh:noexp] OR odds ratio [mh:noexp])


Example intervention terms

• Freetext – rosiglitazone, avandia, avandaryl, avaglim, avandamet,

glitazone*, thiazolidinedione*, tzd, ppar gamma agonist*, peroxisome proliferator activated receptor gamma agonist*, pioglitazone*, actos, actoplus, duetact, competact, glustin, nyracta, venvia

• MeSH – thiazolidinediones/

• EMTREE – 2,4 thiazolidinedione derivative/, exp glitazone derivative/


PICO - Comparator

• Should we include all possible comparators? (for

example, placebo, other drug interventions e.g.

glimepiride)

• Would all study designs have a comparator?

• Is it worth trying to include these terms in the search

strategy?


PICO - Outcome

• Should we search on the specific named

adverse effects terms?

• Should we use generic adverse effects

terms?


Example outcome terms

• Freetext

• Fracture*, bone, bmd

• MeSH

• exp Fractures, Bone/, bone density/

• EMTREE

• exp fracture/, bone density/


Restrictions

• Should we limit to specific study designs?

- RCTs?

- Observational studies?

• Date or language restrictions?


Example combinations in MEDLINE

A: “glitazones” terms AND “fractures” terms

- 251 records

- no need to AND with generic adverse effects terms

- may risk not retrieving all relevant papers

B: “glitazones” terms AND (“fractures” terms OR “adverse effects” terms)

- generates thousands of records (over seven thousand in MEDLINE and over thirteen thousand in EMBASE)


Search strategy design

Scenario B: Review of all adverse effects

Adverse effects of newer drugs (gabapentin, lamotrigine,

levetiracetam, oxcarbazepine, tiagabine, topiramate,

vigabatrin) for epilepsy in adults

• P (patient group) - ?

• I (intervention) - ?

• C (comparison) - ?

• O (outcome) - ?


PICO

• P (patient group) - adults with epilepsy

• I (intervention) - gabapentin,lamotrigine,

levetiracetam, carbazepine,

tiagabine, topiramate,

vigabatrin

• C (comparison) - any

• O (outcome) - any adverse effects


Example patient group terms

• Freetext

– epilep*, seizure*, convuls*

• MeSH

– exp epilepsy/

• EMTREE

– exp epilepsy/


Example intervention terms

• Freetext

- epitomax, etiracetam, gabapentin, gabitril, gbp, labileno, lamictal, lamotrigine, ltg, keppra, levetiracetam, neurontin, neurotonin, oxcarbazepine, oxocarbazepine, sabril, sabrilex, trileptal, tiagabine, tiabex, topiramate, topamax, topimax, vigabatrin

• MeSH

- vigabatrin/

• EMTREE

- gabapentin/, lamotrigine/, etiracetam/, oxcarbazepine/, tiagabine/, topiramate/, vigabatrin/


Searching with outcome terms

• Adverse effects terms

- Indexing terms (such as MeSH or EMTREE)

- Subheadings / qualifiers

- Textwords

- Search filters / hedges


Example generic adverse effects

indexing terms MEDLINE:

adverse drug reaction reporting systems/

drug toxicity/

abnormalities, drug induced/

drug monitoring/

drug hypersensitivity/

poisoning/

substance-related disorders/

product surveillance postmarketing/

postoperative complications/

intraoperative complications/

Many of these terms can be exploded to include narrower indexing terms

EMBASE:

adverse drug reaction/

drug toxicity/

drug safety/

drug monitoring/

drug hypersensitivity/

drug surveillance program/

intoxication/

side effect/

postmarketing surveillance/

postoperative complication/

periperative complication/


Example subheadings

MEDLINE

/adverse effects

/poisoning

/toxicity

/chemically induced

/contraindications

/complications

EMBASE

/side effect

/adverse drug reaction

/drug toxicity

/complication


How to use subheadings (1)

Free floating subheadings

• Subheadings can be searched for attached to any indexing term in MEDLINE and EMBASE

• Examples for OVID MEDLINE:

– ae.fs (adverse effects)

– co.fs (complications)

– po.fs (poisoning)

– de.fs (drug effects)


How to use subheadings (2)

MEDLINE

Example:

Aspirin/adverse effects

Aspirin is the MeSH term and

adverse effects is the

subheading

Example:

headache/chemically induced

Headache is the MeSH term

and chemically induced is the

subheading

EMBASE

Example:

Acetylsalicylic-acid/adverse-

drug-reaction

Acetylsalicylic-acid is the EMTREE

term and adverse-drug-reaction is

the subheading

Example:

headache/side effect

Headache is the EMTREE term and

side effect is the subheading


Example free text adverse effects

terms

Safe, safety, side effect*, undesirable effect*,

treatment emergent, tolerability, toxicity, adverse

drug reaction*, adrs, adverse effect*, adverse

drug effect*, adverse reaction*, adverse event*,

adverse outcome*, complication*, harm, harmful,

harms, risk


Problems with free text search

• Wide range of terms for adverse effects - so try to include as many relevant synonyms as possible

• general (toxicity, side effect, adverse effect, harms)

• specific (e.g. lethargy, tiredness, malaise)

• Specific terms may not be known in advance

• Free text search cannot find adverse effects not mentioned in the title or abstract (even though they appear in the full report)

• False hits; ‘Relative Risk’, ‘Self-harm’, ‘Patient safety’, ‘adverse effects were not considered’


Evaluated adverse effects search

strategies in MEDLINE • Badgett et al 1999

• (ae OR co OR po OR de).fs OR case report/ AND human/

• Golder et al 2006

• (ae OR co OR de).fs OR (safe OR safety OR side effect*

OR undesirable effect* OR treatment emergent OR

tolerability OR toxicity OR adrs OR (adverse adj2 (effect

OR effects OR reaction OR reactions OR event OR events

OR outcome OR outcomes)).ti,ab

• Tested in 27 systematic reviews. Sensitivity ranged from 72%

to 100%. (Golder et al 2012)


Adverse effects search filters

• In 2001, 23% of published reports that

contained harmful effects data had no

adverse effects textwords or indexing

terms (Derry et al 2001)

• In 2011 this figure had decreased to 8%

(Golder et al 2012)


Where to search?


Number of included references retrieved

by databases in glitazone review

0

5

10

15

20

25

30

35

40

45

ScienceCitation

Index (SCI)

BIOSISPreviews

EMBASE MEDLINE Scirus DerwentDrug File

PASCAL BritishLibraryDirect

ThomsonReutersIntegrity

TOXLINE ADISClinicalTrials

Insight

Missedreferences

Relevantreferencesretrieved


Quiz time

• What percentage of papers would have been missed in the glitazone review if the search had been limited to MEDLINE, EMBASE, CENTRAL and reference checking?

– A: 25%

– B: 8%

– C: 57%


Percentage of relevant references missed

in glitazone review

0%

10%

20%

30%

40%

50%

60%

70%

80%

MEDLINE MEDLINE and Reference Checking MEDLINE, EMBASE, CENTRAL andreference checking

Percentage ofincludedreferences missedwith searches

Percentage ofrelevantreferences missedif all papers on thedatabases hadbeen retrieved


Minimum combination of sources in

glitazone review

Identifies all 58 included references with search strategy for ‘glitazones’ and ‘fractures’

AHFS First Medscape DrugInfo

Science Citation Index Thomson Reuters Integrity*

EMBASE Conference Papers Index*

GSK website BIOSIS Previews

British Library Direct Handsearching

Reference checking

*either database


Availability of relevant references

Minimum combination of sources in which the 58

included references were available

BIOSIS Previews

British Library Direct

Medscape DrugInfo

Science Citation Index

Handsearching


Citation Searches

• Case reports or series of suspected new adverse effects

may lead you to more detailed studies as researchers

unlikely to design safety study without a ‘signal’ (Kuper

et al 2006)

• Especially useful if find relatively old studies

• Examples: Google Scholar, MEDLINE, PsycINFO,

SciFinder, Scopus, Web of Science


Take home message

• Including adverse effects in systematic reviews is important so that clinicians, patients and policy makers can make balanced decisions and minimise harm.

• Adverse effects search terms or adverse effects search filters can be useful particularly when large numbers of records would otherwise be retrieved.

• Searches of multiple databases sources and non-database sources are required in systematic reviews of adverse effects.

• Searching only MEDLINE may miss over half the relevant references.


Future

• More reviews are including adverse effects either as secondary

outcome (in addition to effectiveness) or as primary outcome

• Review of 849 systematic reviews suggests improvements in

searches (Golder et al 2013)

- More databases searched

- Fewer reviews restricted to searching only MEDLINE or to date

or language restrictions

- More transparent reporting of searches

• Better reporting in primary studies

- CONSORT Extension for Harms (Ioannidis et al 2004)

- Only 8% of published reports that contained harmful effects

data had no ‘adverse effects’ related textwords or indexing

terms (Golder et al 2012)


Guidance

Cochrane Handbook

- Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.

CRD’s Guidance

- Systematic Reviews: CRD’s guidance for undertaking reviews in health care. http://www.york.ac.uk/inst/crd/pdf/Systematic_Reviews.pdf

BMC Paper

- Loke YK, Price D, Herxheimer A. Systematic reviews of adverse effects: framework for a structured approach. BMC Med Res Methodol 2007;7:32.

http://www.cochrane-handbook.org/



http://www.york.ac.uk/inst/crd/pdf/Systematic_Reviews.pdf


Help and support

Cochrane Adverse Effects Methods Group

http://aemg.cochrane.org/

Discussion List

http://lists.cochrane.org/mailman/listinfo/aemg

Twitter

@CAEMG1

http://aemg.cochrane.org/




References

Badgett R, Chiquette E, Anagnostelis B, Mulrow C. Locating reports of serious adverse drug

reactions (PowerPoint presentation).[1999. http://medinformatics.uthscsa.edu/#FILTERS

Chou R, Aronsonb N, Atkinsc D, Ismailad AS, Santaguidad P, Smith DH, et al. AHRQ Series Paper 4:

Assessing harms when comparing medical interventions: AHRQ and the Effective Health-Care

Program J Clin Epidemiol 2010;63:502-12.

Compass. Adverse drug reactions wastes NHS £2BN reveals Compass. Compass; London; 2008.

[cited 2008 18 Sep]. Available from: http://www.compassonline.org.uk/news/item.asp?n=1551.

Derry S, Loke YK, Aronson JK. Incomplete evidence: the inadequacy of databases in tracing

published adverse drug reactions in clinical trials. BMC Med Res Methodol 2001;1:7.

Golder S, McIntosh HM, Duffy S, Glanville J, Developing efficient search strategies to identify reports

of adverse effects in MEDLINE and EMBASE. Health Info Libr J. 2006 Mar;23(1):3-12.

Golder S, Loke YK. Failure or success of electronic search strategies to identify adverse effects data.

JMLA 2012 ;100(2):130-4.

Golder S, Loke YK. The contribution of different information sources for adverse effects data. Int J

Technol Assess Health Care. 2012;28(2):133-7.

http://medinformatics.uthscsa.edu/#FILTERS

http://www.compassonline.org.uk/news/item.asp?n=1551


References

Golder S, Loke YK. The performance of adverse effects search filters in MEDLINE and EMBASE.

Health Info Libr J 2012;29(2):141-51.

Golder S, Loke YK, Zorzela L. Some improvements are apparent in identifying adverse effects in

systematic reviews from 1994 to 2011. J Clin Epidemiol 2013;66(3):253-60. Ioannidis JP, Evans SJ,

Gotzsche PC, O'Neill RT, Altman DG, Schulz K, Moher D. Better reporting of harms in randomized

trials: an extension of the CONSORT statement. Ann Intern Med 2004; 141(10):781-788

Kuper H, Nicholson A, Hemingway H. Searching for observational studies: what does citation tracking

add to PubMed? A case study in depression and coronary heart disease. BMC Med Res Methodol

2006;6:4.

Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug reaction in hospitalized patients: a

meta-analysis of prospective studies. JAMA 1998;279:1200-5.

Wieland S, Dickersin K. Selective exposure reporting and Medline indexing limited the search

sensitivity for observational studies of the adverse effects of oral contraceptives. J Clin Epidemiol.

2005 Jun;58:6:560-7.

search strategies and data sources for adverse effects reviews...• adverse effects search terms or...

Documents