do guidelines for treating chest disease in children use
TRANSCRIPT
Prayle, A.P. and Cox, Tessy and Smith, Sherie and Rycroft-Malone, Joanne and Thomas, Kim S. and Hughes, Dyfrig A. and Smyth, Alan R. (2017) Do guidelines for treating chest disease in children use Cochrane Reviews effectively?: a systematic review. Thorax . ISSN 1468-3296
Access from the University of Nottingham repository: http://eprints.nottingham.ac.uk/44147/8/thoraxjnl-2016-208790.full.pdf
Copyright and reuse:
The Nottingham ePrints service makes this work by researchers of the University of Nottingham available open access under the following conditions.
This article is made available under the Creative Commons Attribution licence and may be reused according to the conditions of the licence. For more details see: http://creativecommons.org/licenses/by/2.5/
A note on versions:
The version presented here may differ from the published version or from the version of record. If you wish to cite this item you are advised to consult the publisher’s version. Please see the repository url above for details on accessing the published version and note that access may require a subscription.
For more information, please contact [email protected]
brought to you by COREView metadata, citation and similar papers at core.ac.uk
provided by Nottingham ePrints
Do guidelines for treatingchest disease in children useCochrane Reviews effectively?A systematic review
ABSTRACTCochrane Reviews summarise best evidenceand should inform guidelines. We assessedthe use of Cochrane Reviews in the UKguidelines for paediatric respiratory disease.We found 21 guidelines which made 1025recommendations, of which 96 could beinformed by a Cochrane Review. In 38/96recommendations (40%), some or all of therelevant Cochrane Reviews were not cited. Welinked recommendations to 140 CochraneReviews. In 37/140 (26%) cases, the guidelinerecommendation did not fully agree with theCochrane Review. Guideline developers mayfail to use Cochrane Reviews or may makerecommendations which are not in line withbest evidence.
INTRODUCTIONClinical practice guidelines support optimaldecision making in medical care. Guidelinesshould use the best available evidence.1
Systematic reviews use transparent criteria(such as Grading of RecommendationsAssessment, Development and Evaluation(GRADE)) to evaluate the quality of evi-dence2 and so systematic reviews (whereavailable) should be the primary source ofevidence in guidelines. The CochraneCollaboration produces systematic reviews(‘Cochrane Reviews’) using a rigorous meth-odology. These are peer reviewed at theprotocol and review stage, and are updatedregularly.3 Previous work indicates thatguidelines do not make full use of CochraneReviews.4 5 This represents researchwastage, and may lead to suboptimalmedical care.
Respiratory disease in children iscommon—20% of children visiting theemergency department, with a medicalproblem, will have a respiratory illness.6
However, the paediatric respiratory evi-dence base is limited. Nearly half of chil-dren with respiratory disease receive amedication which is off-label orunlicensed.7 It is particularly importantthat guidelines for respiratory disease inchildren make the best use of this limitedevidence. We examined the use of evi-dence from Cochrane Reviews in guide-lines for respiratory disease in children.
We aimed to understand the use of thebest available evidence in the field ofpaediatric respiratory medicine. We sys-tematically examined the use of CochraneReviews in the UK clinical guidelines for
lower respiratory diseases in children andwe examined the agreement betweenthe guideline recommendations and theCochrane Reviews. We investigated theassociation between guideline commission-ing agency, the topic, the publication yearand the use of alternate high-quality evi-dence on whether Cochrane Reviews werecited, and whether their conclusions werefollowed.
METHODSWe identified all the respiratory guidelinesin the UK for lower respiratory tract diseasefor children via database and web searches.We simultaneously identified all theCochrane Reviews relevant to paediatricrespiratory medicine, via the Cochranelibrary. For each guideline, we included allrecommendations pertaining to an inter-vention for lower respiratory tract diseasein children. For each recommendation, weidentified if there was a Cochrane Reviewwhich could inform it, and which had beenpublished at least 1 year prior to the guide-line. We mapped each guideline recommen-dation to relevant Cochrane Reviews.For each linked guideline recommenda-
tion—Cochrane Review, we categorisedthe agreement between the guideline rec-ommendation and the Cochrane Reviewinto one of four categories: (i) totally, (ii)partially, (iii) not in agreement or (iv) astrong guideline recommendation wherethe Cochrane Review concluded thatthere was not enough evidence to draw aconclusion (see online supplementarytables S1 and S2 for definitions and exam-ples). Where guideline recommendationsdisagreed with the Cochrane Review, wecategorised the extent of the disagreement.The protocol (including study eligibilitycriteria and statistical analysis plan) wasproduced in advance of the data collection,is available at the University ofNottingham ePrints server and as onlinesupplementary files 2 and 3). Detailedmethods are provided in onlinesupplementary information.
RESULTSGuidelines and Cochrane ReviewsidentifiedWe included 21 guidelines and 236Cochrane Reviews (see figure 1). The 21guidelines made 1025 recommendations,of which 555 were for treatment of lowerrespiratory disease in children. We identi-fied relevant Cochrane Reviews for 96(17.3%) of these 555 recommendations.Of the 96 recommendations that could
use Cochrane Reviews, 28/96 (29%) didnot use any, and 10/96 (10%) did not useall the available Cochrane Reviews. There
were 140 instances where a CochraneReview could be linked to at least oneguideline recommendation. Of these, 103/140 (74%) were in agreement, 13/140(9%) were partially in agreement, 5/140(4%) disagreed and 19/140 (13%) werestrong recommendations but the CochraneReview did not draw a conclusion. FewCochrane Reviews in paediatric respiratorymedicine were able to draw a strong con-clusion, 96/283 (34%).
We summarise these data in figure 2. Aninteractive version of this figure allowing thereader to directly explore the data within aweb browser is available online here: https://www.nottingham.ac.uk/~mszap3/interactive_figure.html.
Further analysis is presented in theonline supplementary information, includ-ing a sensitivity analysis of our judgementsin categorising agreement and an analysisinvestigating the impact of commissioningagency, guideline topic, guideline year andguideline use of alternate high-quality evi-dence on the use of Cochrane Reviews.
DISCUSSIONWe found that 38/96 (40%) of guidelinerecommendations did not use all the rele-vant Cochrane Reviews. The majority ofguideline recommendations were in agree-ment with Cochrane Review recommen-dations. We present the data as aninteractive figure allowing the reader toexplore the links between CochraneReviews and guideline recommendations.
Our results are broadly in keeping withstudies in other fields such as smoking ces-sation and neonatal medicine which showthat guidelines do not make the best useof Cochrane Reviews. Silagy et al4 foundfour guidelines for smoking cessation (onefrom the UK). In the UK guideline, 16/22recommendations could have cited aCochrane Review but only 8 recommenda-tions did so. Brok et al5 studied the agree-ment between guidelines and CochraneReviews for newborns in Denmark.Compared with our study, they foundsimilar discrepancies between CochraneReviews and guideline recommendations—24% were not in agreement (of which6% partially agreed and 18% disagreed).
Our study is comprehensive, used an apriori protocol and categorisations wereconducted independently by two investi-gators. The study has limitations, includ-ing the subjectivity in decisions regardingagreement and disagreement. We expandon this in online supplementary file 1.
CONCLUSIONIn spite of the work of the Cochrane col-laboration, there are still many treatment
Thorax Month 2017 Vol 0 No 0 1
Research letter Thorax Online First, published on April 26, 2017 as 10.1136/thoraxjnl-2016-208790
Copyright Article author (or their employer) 2017. Produced by BMJ Publishing Group Ltd (& BTS) under licence.
group.bmj.com on July 14, 2017 - Published by http://thorax.bmj.com/Downloaded from
Figure 1 Flow diagram of the selection of guidelines and Cochrane Reviews.
2 Thorax Month 2017 Vol 0 No 0
Research letter
group.bmj.com on July 14, 2017 - Published by http://thorax.bmj.com/Downloaded from
Figure 2 Do Cochrane Reviews influence clinical practice guideline recommendations? Evidence network diagram to show the links betweenCochrane Reviews and Guideline recommendations. Each individual guideline recommendation is represented by a purple node, and each CochraneReview by a green node. A solid blue line connecting a guideline recommendation to a Cochrane recommendation indicates that the guideline citedthe Cochrane Review, and the two are in agreement. A broken line indicates that the guideline did not cite the Cochrane Review. A brown lineindicates that the Cochrane Review and guideline were not totally in agreement. Panel A shows all the links. Subsequent panels are subgrouped bydisease topic: B=asthma, C=cystic fibrosis, D=respiratory infections and E=respiratory aspects of critical care. An interactive web-based version ofthe evidence network diagram (which loads in all recent major browsers) is available at www.nottingham.ac.uk/~mszap3/interactive_figure.html andallows the reader to explore the underlying data further.
Thorax Month 2017 Vol 0 No 0 3
Research letter
group.bmj.com on July 14, 2017 - Published by http://thorax.bmj.com/Downloaded from
decisions where there is no systematicreview to inform guideline recommenda-tions. However, we have shown that, evenwhere a Cochrane Reviews exists, guidelinedevelopers may not make use of it or maymake recommendations contrary to thefindings of the review. This study demon-strates that only a minority of recommen-dations in clinical practice guidelines arebased on the highest quality evidence. Agreat deal of money, time and effort goesinto creating and updating CochraneReviews. Not using such evidence in guide-lines constitutes research waste.
Andrew P Prayle,1 Tessy Cox,1 Sherie J Smith,1
Joanne Rycroft-Malone,2 Kim S Thomas,3
Dyfrig A Hughes,4 Alan R Smyth1
1Division of Child Health, Obstetrics and Gynaecology,University of Nottingham, Nottingham, UK2School of Healthcare Sciences, Bangor University,Bangor, UK3Centre of Evidence Based Dermatology, University ofNottingham, Nottingham, UK4Centre for Health Economics and MedicinesEvaluation, Bangor University, Bangor, UK
Correspondence to Professor Alan R Smyth, Divisionof Child Health, Obstetrics and Gynaecology, Universityof Nottingham, E Floor East Block, Queens MedicalCentre, Derby Road, Nottingham NG7 2UH, UK; [email protected]
Author note The protocol is available for this study athttp://eprints.nottingham.ac.uk/3031/ (part 1) andhttp://eprints.nottingham.ac.uk/3032/ (part 2).
Twitter Follow Andrew Prayle @andrewprayle andAlan R Smyth @AlanRSmyth
Acknowledgements The authors thank all thosewho were involved in the development and authorship ofthe Cochrane Reviews and Guidelines used within thisstudy.
Contributors The study was conceived by ARS, withdevelopment by APP and TC. Final study design was
achieved by discussion of all authors. Data collectionand analysis: TC, APP and SJS. Data analysis, statisticsand programming: APP. Final manuscript written andapproved by all authors.
Funding This work did not receive specific funding.APP was funded by a NIHR Doctoral ResearchFellowship (DRF-2009-02-112), and is currently fundedby a NIHR ACL post. DAH receives financial supportfrom the MRC North West Hub in Trial MethodologicalResearch: MR/K025635/1. The funders and sponsorhad no involvement in the decision to publish, or thecontents of the publication. This report presentsindependent research funded by the National Institutefor Health Research.
Competing interests All authors have completed theUnified Competing Interests form at http://www.icmje.org/coi_disclosure.pdf (available on request from thecorresponding author) and declare: no support fromany organisation for the submitted work; no financialrelationships with any organisations that might have aninterest in the submitted work in the previous 3 years,no other relationships or activities that could appear tohave influenced the submitted work. Professor Smythreports personal fees from Vertex, personal fees fromGilead, personal fees from Roche, personal fees fromPTC Therapeutics, outside the submitted work. Inaddition, Professor Smyth has a patent AlkylQuinolones as Biomarkers of Pseudomonas aeruginosaInfection and Uses Thereof issued.
Provenance and peer review Not commissioned;externally peer reviewed.
Data sharing statement The raw data andcomputer code underlying this study to fully reproduceall the statistics and figures reported are available as adownloadable online supplementary file on GitHubhttps://github.com/andrewprayle/Do-guidelines-for-treating-chest-disease-in-children-use-Cochrane-reviews-effectively.
▸ Additional material is published online only. To viewplease visit the journal online (http://dx.doi.org/10.1136/thoraxjnl-2016-208790).
Open Access This is an Open Access articledistributed in accordance with the terms of the CreativeCommons Attribution (CC BY 4.0) license, whichpermits others to distribute, remix, adapt and buildupon this work, for commercial use, provided theoriginal work is properly cited. See: http://creativecommons.org/licenses/by/4.0/
To cite Prayle AP, Cox T, Smith SJ, et al. ThoraxPublished Online First: [please include Day Month Year]doi:10.1136/thoraxjnl-2016-208790
Received 19 April 2016Revised 1 March 2017Accepted 5 March 2017
Thorax 2017;0:1–4.doi:10.1136/thoraxjnl-2016-208790
REFERENCES1 Shekelle PG, Woolf SH, Eccles M, et al. Developing
guidelines. BMJ 1999;318:593–6.2 Guyatt GH, Oxman AD, Kunz R, et al. What is
“quality of evidence” and why is it important toclinicians? BMJ 2008;336:995–8.
3 Delaney A, Bagshaw SM, Ferland A, et al. The qualityof reports of critical care meta-analyses in theCochrane database of systematic reviews: anindependent appraisal. Crit Care Med2007;35:589–94.
4 Silagy CA, Stead LF, Lancaster T. Use ofsystematic reviews in clinical practice guidelines:case study of smoking cessation. BMJ2001;323:833–6.
5 Brok J, Greisen G, Madsen LP, et al. Agreementbetween Cochrane Neonatal reviews and clinicalpractice guidelines for newborns in Denmark: across-sectional study. Arch Dis Child Fetal Neonatal Ed2008;93:F225–9.
6 Sands R, Shanmugavadivel D, Stephenson T, et al.Medical problems presenting to paediatric emergencydepartments: 10 years on. Emerg Med J2012;29:379–82.
7 ’t Jong GW, Eland IA, Sturkenboom MC, et al.Unlicensed and off-label prescription ofrespiratory drugs to children. Eur Respir J2004;23:310–13.
4 Thorax Month 2017 Vol 0 No 0
Research letter
group.bmj.com on July 14, 2017 - Published by http://thorax.bmj.com/Downloaded from
A systematic reviewchildren use Cochrane Reviews effectively? Do guidelines for treating chest disease in
Kim S Thomas, Dyfrig A Hughes and Alan R SmythAndrew P Prayle, Tessy Cox, Sherie J Smith, Joanne Rycroft-Malone,
published online April 26, 2017Thorax
0http://thorax.bmj.com/content/early/2017/04/26/thoraxjnl-2016-20879Updated information and services can be found at:
These include:
References
#BIBL0http://thorax.bmj.com/content/early/2017/04/26/thoraxjnl-2016-20879This article cites 7 articles, 6 of which you can access for free at:
Open Access
http://creativecommons.org/licenses/by/4.0/use, provided the original work is properly cited. See: others to distribute, remix, adapt and build upon this work, for commercialthe Creative Commons Attribution (CC BY 4.0) license, which permits This is an Open Access article distributed in accordance with the terms of
serviceEmail alerting
box at the top right corner of the online article. Receive free email alerts when new articles cite this article. Sign up in the
CollectionsTopic Articles on similar topics can be found in the following collections
(251)Open access
Notes
http://group.bmj.com/group/rights-licensing/permissionsTo request permissions go to:
http://journals.bmj.com/cgi/reprintformTo order reprints go to:
http://group.bmj.com/subscribe/To subscribe to BMJ go to:
group.bmj.com on July 14, 2017 - Published by http://thorax.bmj.com/Downloaded from