quality of news media reports about the effects and costs ... · cepts. in norway, where i am...
TRANSCRIPT
![Page 1: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/1.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 1
Qualityofnewsmediareportsabouttheeffectsandcostsofhealthinterventions:SystematicreviewprotocolOxmanMWorkingpaper,April26,2018www.informedhealthchoices.org
![Page 2: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/2.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 2
Colophon
Title Qualityofnewsmediareportsabouttheeffectsandcostsofhealthinterventions:Systematicreviewprotocol
Authors Oxman, Matt 1,2,3
1. University of Oxford, United Kingdom 2. Faculty of Health Sciences,Oslo Metropolitan University, Norway 3. Centre for Informed Health Choices,Norwegian Institute of Public Health, Norway
Correspondingauthor
Matt [email protected] Telephone: +47 98636788 Oslo Metropolitan University, Faculty of Health Sciences Section for Research & Development Pilestredet Campus P.O. Box 4 St. Olavs plass N-0130 Oslo, Norway
Keywords Newsmedia,Systematicreview,Unreliableinformation,KeyConcepts
Citation Oxman,M.,Qualityofnewsmediareportsabouttheeffectsandcostsofhealthinterventions:Systematicreviewprotocol.IHCWorkingPaper.Oslo:InformedHealthChoices.ISBN: 978-82-8082-971-9
Articlecategory �AboutInformedHealthChoices�Keyconceptsandglossary�Learningresources�Systematicreviews�Developmentandevaluationoflearningresources�Claimevaluationtools�Editorialsandcommentaries�Grantapplicationsþ Other
Date April26,2018
![Page 3: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/3.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 3
Abstract
Background:Unreliableinformationabouthealthcare,combinedwiththeina-bilitytoassessthereliabilityofsuchinformation,canleadtouninformeddeci-sionsand,ultimately,wasteandunnecessarysuffering.Journalismisaparticu-larlyimportantsourceofhealthinformation.Therehavebeenseveralscientificstudiesofthequality(reliability)ofnewsmediareportsabouttheeffectsofhealthinterventions.Theredoesnotappeartobeasystematicreviewofthesestudies.Theplannedreviewisintendedtoinform:theproductionandcon-sumptionofhealthnews;aswellasfurtherresearch,includingthedevelopmentofinterventionstohelppeopleassesshealthcareinformationinthenews,sotheycanmakewell-informeddecisions.Objectives:1)Toassesscriteriausedtomeasurethequalityofnewsmediare-portsabouttheeffectsandcostsofhealthinterventions;2)toassessthequalityofsuchreports;and3)toexplorefactorsthatmightexplainvariationinquality.Eligibility:Wewillincludescientificstudiesofprint,broadcastandonlinenewsreports(thepopulation).Atleastoneexplicitcriterionmusthavebeenusedbytheresearcherstomeasurequality(thecondition).Itmustbepossibleandsen-sibletoframetheresultsforatleastonecriterionastheproportionofreportsthatsatisfiedit(e.g.theproportionofreportsinwhicheffectestimatesareab-solute,notonlyrelative).Therewillbenolimitsonlanguage,geographyortimeperiodofthereports(thecontext).Tobeincluded,studiesmustspecify:thesamplingframe;theselectioncriteria;andtheselectiontechnique.Methods:Tworeviewerswillextractdata,assessriskofbias,andmakejudge-mentsaboutInformedHealthChoices(IHC)KeyConceptscapturedbycriteriausedinscientificstudies.Ifnecessary,athirdreviewerwillhelpreachconsen-susonanyjudgement.Meta-analyseswillbeconductedtoestimatethepreva-lenceofnewsreportsthatsatisfyrespectivecriteria.Toexplorevariation,sub-groupanalyseswillbeconductedfor:medium(broadcastvs.other);timeperi-od(decade);financialmodel(commercialvs.non-commercial);journalistspe-cialisation(healthorsciencevs.other);andcountryincomelevel(low-incomevs.other).
![Page 4: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/4.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 4
Background
TheproblemUnreliablehealthinformationThisprotocolisforasystematicreviewthatisultimatelyintendedtohelpad-dressamajor,globalproblem.Theproblemhasthreeparts.Thefirstisunrelia-blehealthinformation.Ineffect,thereisendlessinformationabouttheeffectsofhealthinterventions,ahealthinterventionbeinganyactionintendedtoimproveormaintainthehealthofindividualsorgroups.Thisincludesinformationabout:“modern”,“academ-ic”,“conventional”or“western”medicine;“complementary”,“alternative”,“tra-ditional”or“natural”medicine;screening;surgeryanddevices;diet,exerciseandlifestyle;andsystemsandpolicies.Someofthisinformation,ifnotmostofit,isunreliableinonewayoranother.Theinformationcanbedirectlymislead-ing,suchastheexplicitclaimthataninterventioncausesanoutcomewhenitisonlyassociatedwiththeoutcome.Oritcanbemisleadingbyomission,suchasrelativeeffectestimateswithoutabsoluteestimates,particularlywhenthebase-lineriskissmall(Woloshin,etal.,2008).Researchershavefoundunreliablelayinformationabouttheeffectsofhealthinterventionsin:
• Patientmaterials(Coulter,etal.,1999);productlabels(UnitedStatesGovernmentAccountabilityOffice,2010),
• Varioustypesofwebsites(Culver,etal.,1997;Wolfe,etal.,2002;Glenton,etal.,2005;Spencer,etal.,2016)
• Advertisements(Sansgiry,etal.,1999;Frosch,etal.,2007;Frosch,etal.,2011;Faerber&Kreling,2013;Groven&Braitwaite,2016)
• Thenews(Appendix1)Informationinthenewsmediaisofparticularinterest,forreasonsdescribedlaterinthebackgroundsection.
![Page 5: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/5.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 5
Finally,researchreferencedinthefollowingsubsectionshowspeopleareuna-bletoassessinformationabouttheeffectsofhealthinterventions.Logically,ifpeopleareunabletoassesssuchinformation,theyaremorelikelytospreadthatwhichisunreliable.Conversely,peoplespreadingunreliableinformation,asdocumentedinthestudiesreferencedinthissubsection,suggeststheyareunabletoassesssuchinformation.InabilitytoassesshealthinformationThesecondpartoftheproblemisthatpeopleareunabletoassessthereliabilityofinformationabouttheeffectsofhealthinterventions.AspartoftheInformedHealthChoices(IHC)project(www.informedhealthchoices.org),theabilitytoassesssuchinformationandmakeinformeddecisionsabouthealthinterven-tionshasbeenbrokendownintotheabilitytoapplyspecificconceptscalledtheIHCKeyConcepts.Forexample,peopleneedtobeabletoapplytheconceptthatassociationisnotthesameascausation.Theoriginallistincluded32concepts(Austvoll-Dahlgren,etal.,2015).Thenewestiterationincludes36(Box1)(Chalmers,etal.,2018).TheabilitytoapplyIHCKeyConceptsispartofhealthliteracyandhasbeenmeasuredbothdirectlyandindirectly.Therehavebeenlargesurveysofhealthliteracy,inEuropebytheEuropeanHealthLiteracyProjectConsortium(HLS-EUConsortium,2013)andintheUnit-edStatesbytheNationalCenterforEducationStatistics(Kutner,etal.,2006).AccordingtotheEuropeansurvey,“Healthliteracyislinkedtoliteracyanden-tailspeople’sknowledge,motivationandcompetencestoaccess,understand,appraise,andapplyhealthinformationinordertomakejudgmentsandtakede-cisionsineverydaylifeconcerninghealthcare,diseasepreventionandhealthpromotiontomaintainorimprovequalityoflifeduringthelifecourse”(HLS-EUConsortium,2013,p.7).FortheAmericansurvey,thetermwasdefinedas:“Thedegreetowhichindividualshavethecapacitytoobtain,process,andunder-standbasichealthinformationandservicesneededtomakeappropriatehealthdecisions”(Kutner,etal.,2006,p.iii).TheabilitytoapplyIHCKeyConcepts(i.e.assessthereliabilityofinformationabouttheeffectsofhealthinterventionsandmakewell-informeddecisions)islogicallyincludedinbothdefinitions.However,neithersurveyappearstohaveobjectivelymeasuredthis.TheEuropeansurveymeasuredthesubjectiveeaseof47tasks.TasksrelatedtoapplyingIHCKeyConcepts,including“judgeiftheinformationaboutillnessinthemediaisreliable”,wererankedamongstthemostdifficult(HLS-EUConsortium,2013).
![Page 6: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/6.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 6
Box1:ShorttitlesfortheInformedHealthChoices(IHC)KeyConceptsRecognisinganunreliablebasisforaclaim
1.1. Treatmentscanharm.1.2. Anecdotesareunreliableevidence.1.3. Associationisnotthesameascausation.1.4. Commonpracticeisnotalwaysevidence-based.1.5. Newerisnotnecessarilybetter.1.6. Expertopinionisnotalwaysright.1.7. Bewareofconflictinginterests.1.8. Moreisnotnecessarilybetter.1.9. Earlierisnotnecessarilybetter.1.10. Hopemayleadtounrealisticexpectations.1.11. Explanationsabouthowtreatmentsworkcanbewrong.1.12. Dramatictreatmenteffectsarerare.
Understandingwhethercomparisonsarefairandreliable2.1. Comparisonsareneededtoidentifytreatmenteffects.2.2. Comparisongroupsshouldbesimilar.2.3. Peoples’outcomesshouldbeanalysedintheiroriginalgroups.2.4. Comparisongroupsshouldbetreatedequally.2.5. Peopleshouldnotknowwhichtreatmenttheyget.2.6. Peoples’outcomesshouldbeassessedsimilarly.2.7. Allshouldbefollowedup.2.8. Consideralltherelevantfaircomparisons.2.9. Reviewsoffaircomparisonsshouldbesystematic.2.10. Peerreviewandpublicationdoesnotguaranteereliableinformation.2.11. Allfaircomparisonsandoutcomesshouldbereported.2.12. Subgroupanalysesmaybemisleading.2.13. Relativemeasuresofeffectscanbemisleading.2.14. Averagemeasuresofeffectscanbemisleading.2.15. Faircomparisonswithfewpeopleoroutcomeeventscanbemisleading.2.16. Confidenceintervalsshouldbereported.2.17. Donotconfuse‘statisticalsignificance’with‘importance’.2.18. Donotconfuse‘noevidenceofadifference’with‘evidenceofnodifference’.
Makinginformedchoices3.1. Dotheoutcomesmeasuredmattertoyou?3.2. Areyouverydifferentfromthepeoplestudied?3.3. Arethetreatmentspracticalinyoursetting?3.4. Dotreatmentcomparisonsreflectyourcircumstances?3.5. Howcertainistheevidence?3.6. Dotheadvantagesoutweighthedisadvantages?
![Page 7: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/7.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 7
TheAmericansurveysuggestedaminorityofAmericanadults(12%)had“pro-ficient”healthliteracy(Kutner,etal.,2006,p.v).However,onlyasampleofthehealthliteracyquestionsusedinthatsurveyarepubliclyavailableandnoneoftheavailablequestionsmeasuretheabilitytoassessinformationaboutcausali-ty(www.nces.ed.gov/NAAL/sample.asp).Anothersurrogatemeasureofpeople’sabilitytoapplyIHCKeyConceptsistheirexpectationsabouteffects.Inasystematicreviewofpatients’expectations,themajorityofparticipantsoverestimatedbenefitsandunderestimatedharms(Hoffmann&DelMar,2015).Asystematicreviewofclinicians’expectationsshowedcliniciansdothesame(Hoffmann&DelMar,2017).Othersubstitutesforabilityareattitudesandbeliefs.AccordingtoastudyintheUnitedKingdom,patientsandgeneralpractitionersappeartoplaceadispropor-tionateamountoftrustinthereputationoftheorganisationconductingaclini-caltrial,thequalificationsoftheresearchers,andpeer-review,versusthemeth-ods,whicharewhatinfactdeterminethereliabilityoftheresults(TheAcademyofMedicalSciences,2016).Moreover,aboutathirdoftherespondents(37%)placedahighleveloftrustindatafrommedicaltrials,whileabouttwothirds(65%)placedahighleveloftrustintheexperiencesoffriendsandfamily,i.e.anecdotalevidence.TherehaveindeedbeenstudiesdirectlymeasuringtheabilitytoapplyKeyCon-cepts.InNorway,whereIambased,Oxmanetal.testedarandomsampleof626Norwegianadults(Oxman,etal.,2017).Aboutoneinfive(19%)showedtheywereabletodistinguishbetweenanassociationandcausation.OthersurveyssuggestNorwegianstudentsinpost-secondaryschool(Pettersen,2007)andsecondaryschool(Pettersen,2005)alsostrugglewithassessingclaimsabouttheeffectsoftreatments.AnotherproductoftheInformedHealthChoicesprojectistheCLAIMEvaluationToolsdatabase(www.informedhealthchoices.org/claim-evaluation-tools)(Chalmers,etal.,2018).TheCLAIMEvaluationToolsaremultiple-choiceques-tionsdevelopedspecificallytomeasurepeople’sabilitytoapplytheIHCKeyConcepts(Austvoll-Dahlgren,etal.,2016;Austvoll-Dahlgren,etal.,2017).Be-forethedevelopmentofthedatabase,inasystematicreviewoftoolsformeas-uringsaidability,Austvoll-Dahlgrenetal.identified215discreteinstrumentsorprocedures,capturingupto15oftheconcepts,whereoffourcaptured10ormore(Austvoll-Dahlgren,etal.,2016).Avalidatedsetof26questionsmeasuringtheabilitytoapply13oftheconcepts(i.e.twoquestionsperconcept)wereusedtomeasuretheeffectsofIHClearn-ingresources,intworandomisedtrials(Nsangi,etal.,2017;Semakula,etal.,
![Page 8: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/8.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 8
2017).Inthefirsttrial,thecontrolgroupwas4430Ugandanprimaryschoolchildren,afterlosstofollow-up(Nsangi,etal.,2017).Inthesecondtrial,afterlosstofollow-up,thecontrolgroupwas273parentsandguardiansofchildrenparticipatingintheinterventionorcontrolgroupofthefirsttrial(Semakula,etal.,2017).Lessthanhalfoftheparticipantsinthecontrolgroupforeithertrialansweredbothquestionscorrectlyforanyofthe13includedconcepts.UninformedhealthdecisionsThecombinationofunreliableinformationabouttheeffectsofhealthinterven-tionsandpeople’sinabilitytoassesssuchinformationcanresultinuninformeddecisions.Thisisthethirdpartoftheproblem.Peopleactingonunreliablein-formation,orfailingtoactonreliableadvice,caninturnleadtowasteandun-necessarysuffering.Thisiswhytheproblemisimportant.Inasystematicreview,lowhealthliteracywasfoundtobeassociatedwithworsehealthoutcomes(Berkman,etal.,2011).Granted,whatwasmeasuredintheincludedstudies,aswiththeaforementionedsurveysofhealthliteracy,maynotincludetheobjectiveabilitytoapplyIHCKeyConcepts.Peoplecertainlyaremakingpoordecisions.Systematicreviewshavefound:worldwideoveruseofmedicalservicesmorelikelytodoharmthangood(Brownlee,etal.,2017);underuseofeffectiveservices(Glasziou,etal.,2017);andovertestingandundertestinginprimarycare(O'Sullivan,etal.,2018).Meanwhile,fortunesarespenton“alternative”medicineanddietarysupple-mentswithoutreliableevidencethattheyarebeneficialorsafe(Optum,2013;Starr,2015).TheinabilitytoapplyIHCKeyConceptsislogicallyafactorinmanyofthesedecisions.ThisreviewWhydothereview?WhileIamawareofscientificstudiesofthequalityofnewsmediareportsabouttheeffectsofhealthinterventions(Appendix1),Iwasunabletofindasystemat-icreviewofsuchprimarystudiesinEpistemonikos(www.epistemonikos.org),TripPro(www.tripdatabase.com),orGoogleScholar(www.scholar.google.com),aslaidoutinAppendix2.NeitheramIawareofasystematicreviewofhowqualityhasbeenmeasuredinsuchstudies.Zeraatkaretal.conductedaliteraturesearchbeforedevelopingtheirownmeasurementtool,namedtheQualityIndexforhealth-relatedMediaReports(QIMR)(2017).However,theyonlyassessedvalidatedtools,andtheydidnotcomparecriteriatotheIHCKeyConcepts.
![Page 9: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/9.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 9
Byassessingthemethodsusedtoassessqualityineligiblestudies,theplannedreviewcaninformthedevelopmentanduseofmethodsinfurtherresearch.Byprovidinganoverviewofthequalityofnewsmediareportsabouttheeffectsofhealthinterventions,thereviewcaninformthegenerationofnewresearchquestions,forexamplethegeographicfocusofthosequestions.Intwoways,thereviewcanhelpaddresstheproblemofunreliableinformationabouttheeffectsofhealthinterventions,inabilitytoassessthatinformation,anduninformeddecisionsabouttheinterventions.First,theresultscanbeusedtoraiseawarenessabouttheproblem.Second,byshowinghownewsmediare-portsabouttheeffectsofhealthinterventionstendtobeunreliable(i.e.thespe-cificcriteriathatthereportstendnottosatisfy),theresultscaninformthepri-oritisationofIHCKeyConceptswhendevelopinginterventionstohelpjournal-istsproducereliablereportsandhelpconsumersassessinformationinthenews.Whyfocusonthenewsmedia?Thenewsmediaplayaspecialroleinthedisseminationofhealthinformation.IntheUnitedKingdom,despiteanincreaseintheuseofsocialmedia,themajor-ityofadultsstillfindoutaboutsciencemostregularlythroughnewsmedia,ac-cordingtoarepresentativesurveyof1,749adults(Castell,etal.,2014).Granted,theproportionofpeoplefindingoutaboutspecificallyhealthsciencethroughthenewsmaybedifferent.Meanwhile,ChewandEysenbachfoundthatin5395tweetsabouttheH1N1outbreakin2009,mostlinksweretoanewswebsite(23%)(Chew&Eysenbach,2010).Another12%linkedtonewsblogs,feedsornichenews,comparedto1.5%thatlinkedtogovernmentandpublichealthagencies.Whilethereareotherimportantsourcesoflayinformationabouttheeffectsofhealthinterventions,includingtheminthisreviewwouldbe“mixingapplesandoranges”.Inotherwords,thequalityofinformationinthosesourcesislikelytobesignificantlydifferent.Forexample,patientmaterialsarelikelytoincludemoreinformationaswellasmorereliableinformationthannewsreports,giventhatthematerialsarepreparedbyhealthprofessionalswhoaretryingtoinformpatients,notattractanaudience.Themethodsusedtoassessinformationfromdifferentsourcesmayalsobedifferent.Furthermore,includingsourcesbesidesthenewsmediacouldmakethereviewunwieldy.Rather,thisstudycanbeusedtoinformreviewsfocusingonothersources.AsstatedintheCochraneHandbookforSystematicReviewsofInterventions,there
![Page 10: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/10.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 10
areprosandconstobothbroadandnarrowreviewquestions,butwiththead-ventofoverviewsofsystematicreviews,“Itmayincreasinglybeconsideredde-sirabletoplanaseriesofreviewswitharelativelynarrowscope,alongsideanOverviewtosummarizetheirfindings”(Higgins&Green,2011).
![Page 11: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/11.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 11
Objectives
Primary
1. Assesscriteriausedtomeasurethequalityofnewsmediareportsabouttheeffectsandcostsofhealthinterventions
2. Assessthequalityofnewsmediareportsabouttheeffectsandcostsofhealthinterventions
Secondary
3. Explorefactorsthatmightexplainvariationinqualityofnewsmediare-portsabouttheeffectsandcostsofhealthinterventions
![Page 12: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/12.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 12
Eligibility
CoCoPopMunnetal.notethatquestionsaboutprevalencedonotfitthepopulation,in-tervention,comparatorandoutcome(PICO)framework,usedforsystematicre-viewsaboutefficacy(Munn,etal.,2015,pp.148-149).Thereisnointerventionorexposure,noristhereanoutcomeonwhichaninterventionorexposurecanhaveaneffect.Therefore,insteadofthePICOframework,Iwillusethemne-monicCoCoPop(condition,context,andpopulation)tosetinclusioncriteria,asMunnetal.recommend.Ihaveaddedcriteriaforstudydesign,asperthePre-ferredReportingItemsforSystematicreviewandMeta-AnalysisProtocols(PRISMA-P)statement(Moher,etal.,2015).PopulationInNewsAroundtheWorld:Content,PractitionersandthePublic,ShoemakerandCohenwrite:“[Thejournalist]typicallyconstructsamethodforfulfillingthedailyjobrequirements.Heorsherarelyhasanunderlyingtheoreticalunder-standingofwhatdefiningsomethingorsomeoneasnewsworthyentails”(Shoemaker&Cohen,2006,p.7).Inotherwords,evenjournalistswillstruggletotellyouwhatexactly“news”is.Thisincludesme.InAhistoryofnews,Stephensusesthedefinition:“Newinformationaboutasub-jectofsomepublicinterestthatissharedwithsomeportionofthepublic”(Stephens,2007,p.4).Intermsofthereview,applyingsuchabroaddefinitionwouldbeproblematicinseveralways.Forexample,advertisements,pressre-leasesandjournalarticlescouldallbeconsiderednews.Insteadofusingaspecificdefinition,Iwillbepragmatic.Iwillonlyconsiderstudiesofinformationthat:1)isinnewspapersormagazines(print),radio,podcastsortelevision(broadcast)ornewswebsites(online);and2)islabelledbytheauthorsas“news”oranycommonsynonym.Intermsofonline,Iwillfo-cusondedicatednewswebsites,i.e.notsocialmediaplatforms,althoughnewsreportsappearonsuchplatforms.Iwillonlyincludestudiesofreportsaboutthepotentialhealtheffects(negativeorpositivechangesinhealthoutcomes)andthemonetarycostsofhealthinter-ventions(anyactionintendedtoimproveormaintainthehealthofanindividualorgroup).Thereportscanallbeaboutthesameconditionorintervention,ortheycanbeamix.Ifanotherwiseeligiblestudy’ssampleincludesnewsreportsabouttheeffectsofhealthinterventionsaswellasothertypesofnewsreports,Iwillextractdata
![Page 13: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/13.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 13
onlyabouttheformer,ifpossible.Otherwise,Iwillleaveoutthestudyandlistitinatableofexcludedstudies.Inthesametable,Iwilllistanystudiesofthequalityofotherhealthinformationinnewsreports,besidesinformationabouttheeffectsofhealthinterventions.ConditionBy“quality”,Imeananattributethatiseitherconducivetoinformeddecisionsabouthealthinterventions(e.g.presentspotentialeffectsinabsolutenumbers)ormisleading(e.g.onlypresentspotentialeffectsinrelativenumbers),orthesumofsuchattributes.Theconditionofmypopulation—newsmediareports—iseitherhavingsatisfiedatleastoneexplicitcriterionforqualityornot(e.g.ei-therpresentingpotentialeffectsinabsolutenumbersornot).InpotentiallyeligibleprimarystudiesofwhichIamaware(Appendix1),re-searchershaveuseddifferentsetsofcriteriatomeasurequality.Zeraatkaretal.conductedaliteraturesearchbeforedevelopingtheirowntooltomeasurethequalityofreportingonhealthresearch,namedtheQualityIndexforhealth-relatedMediaReports(QIMR)(2017).Theyidentifiedoneothervalidatedtoolwiththesamepurpose,namedtheIndexofScientificQuality(ISQ)(Oxman,etal.,1993).Theyalsoidentifiedandassessedinstrumentsformeasuringthequalityofpatientinformation:EnsuringQualityInformationforPatients(EQIP)(Moult,etal.,2004)andDISCERN(Charnock,etal.,1999).Zeraatkaretal.con-cludedthatbothEQIPandDISCERNhavepoorcontentrelevancetohealthnewsreports.ItwouldbeproblematictorestrictthereviewtostudiesthatuseeithertheQIMRorISQtomeasurequality.Thiswouldexcludestudiesofreportswherethehealthinformationisnotbasedonresearch—e.g.informationbasedonan-ecdotalevidence.Moreover,theQIMRwaspublishedasrecentlyas2017,andinmostofthepotentiallyeligiblestudiesofwhichIamaware,neithertheQIMRortheISQwereusedtomeasurequality.Inseveralstudies,researchershaveusedcriteriabasedonthoseusedbyMoynihanetal.(Moynihan,etal.,2000)—e.g.Smithetal.(2005).Inotherstudies,researchershavedevelopedandusedtheirowntools,e.g.Marconetal.(2017).Acriterionmaybesensibleevenifthetoolwasnotvalidated,anditmaybesensibletogroupthecriterionwithothersformeta-analysis.ContextThepopulationforthereviewislimitedtonewsintheaforementionedmedi-ums:newspapersandmagazines:radio,podcastsandtelevision;andnewsweb-sites.Iwillnotsetanylimitsintermsofgeography,languageortimeperiod.StudydesignForastudytobeeligible,theresultsmustincludedichotomousorcategoricaldataforatleastoneexplicitcriterionusedtomeasurequality.Itmustbepossi-bletodichotomisethedataaseithersatisfyingthecriterionornot,fortworea-sons:toensuretheresultscanbeeasilyinterpreted,andtosynthesisefindingsacrossstudies.
![Page 14: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/14.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 14
Somenuance,suchasthenumberofreportsthat“partially”satisfyacriterion,maybelostinthissynthesis.Theimportanceofsuchdetailisquestionable;ifaqualityisimportantenoughtobemeasured,thecriterionusedtomeasurethatqualityshouldbeclearlysatisfied,assumingitisrelevanttothespecificreport.Studiesthatonlyprovideaglobalqualityscore,suchasastarrating,willbeex-cludedandlistedinthetableofexcludedstudies.Globalscoresaredifficulttointerpret,sincetheydonottellyouwhichpartsofanewsreportareproblemat-ic.Furthermore,theyaredifficulttocompareacrossstudiesusingdifferentscoringsystems.Tobeincluded,studiesmustspecify:thesamplingframe(i.e.wherethenewsreportsweresampledfrom);theselectioncriteriaforthereports;andtheselec-tiontechnique.Thisexcludescasestudiesofsinglereports.Iwilllistanycasestudiesthatwouldotherwisebeeligibleinthetableofexcludedstudies.
![Page 15: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/15.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 15
Methods
SearchstrategyTomakesuremysearchissystematicandtransparent,IwillbasemyapproachontheguidelinedevelopedbyKableetal.(2012).Asnotedinthedescriptionofmysearchforanexistingreview(Appendix2),whilemyquestionisinterdisci-plinary,Iconsideritunlikelythatasystematicreviewdesignedtoansweritwouldhavebeenpublishedinanacademicjournalfocusedonjournalism.How-ever,Iconsideritmorelikelythataneligibleprimarystudyhasbeenpublishedinsuchajournal.Therefore,inadditiontoPubMed(www.ncbi.nlm.nih.gov/pubmed),IwillsearchGoogleScholar(www.scholar.google.com)andScopus(www.scopus.com).IwillsearchforgreyliteratureinOpenGrey(www.opengrey.ey)andGreyLiteratureReport(www.greylit.org),aswellasthesesinProQuestDissertations&Theses(GlobalFulltextplusUKandIrelandabstracts)(www.proquest.com).Iwillnotplacelimitsonlanguageortimeperiodinanyofmysearches.Inthesearchforanexistingreview,Ionlyincludedtermsforthepopulation(newsmediareports)whensearchingEpistemonikosandTripPro,sinceIcouldfilterforsystematicreviews,whichreducedtheresultstoamanageablenum-ber.Intheupcomingsearchforprimarystudies,giventhereisnotareliablefil-terIcanuseforstudydesign,Iwillalsoincludetermsforthecondition(quality)ortermsrelatedtotheanalysis,toimprovethespecificityofthesearchstrategy.IwillidentifyadditionaltermsforthePubMedsearch,includingMedicalSubjectHeadings(MeSH)terms,bylookingupthestudiesinPubMedor,ifthestudyisnotinPubMed,inanotherdatabase,andextractingrelevantterms.TheaimistoensuremysearchcapturesallstudiesofwhichIamaware,aswellassimilarstudies.Iwillalsoaddtermsforspecificcategoriesofmedia(e.g.print).Iwillconducttestsearchestoinformpragmaticdecisionsabouteditingordroppingterms,withtheaimofmakingthesearchandscreeningasefficientaspossible,whileminimisingtheriskofmissingeligiblestudies.IwillusethePubMedsearchasthebasisforsearchesinotherdatabases.Besidessearchingdatabases,Iwillcheckthereferencelistsofeligiblestudies.InScopus,Iwillscreenstudiescitingthosethatareeligible,aswellasstudiescit-ingthedevelopmentoftheISQortheQIMR.Iwilltakeadvantageofmyprofes-sionalnetwork,conductingtargeteddisseminationoftheprotocolviaemailandbroaddisseminationviaTwitter,tocrowdsourceeligiblestudies.Finally,IwillcontacttheauthorsofeligiblestudiesandthedevelopersoftheISQandQIMR,toaskaboutunpublishedstudies.
![Page 16: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/16.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 16
Studylanguageisnotaneligibilitycriterion,buttobepragmatic,IwillonlysearchEnglishlanguagedatabases.ToscreenanystudieswithoutanEnglishabstractforeligibility,IwilluseGoogleTranslate(www.translate.google.com).Ifnecessary,Iwillseekhelpfromsomeonefluentinthelanguagee.g.anac-quaintanceoroneoftheauthors.Toextractdatafromanon-Englishpaper,Iwillagainseekhelp.Intheexcludedstudiestable,Iwilllistanyapparentlyeli-giblestudiesfromwhichIamunabletoextractdatabecauseofalanguagebar-rierorbecauseofinadequatereportingandinabilitytoobtainmissinginfor-mationfromtheauthors.AsecondreviewerandIwillindependentlyjudgeeligibilitybyreviewingeachabstractand,ifnecessary,(sectionsof)thefulltext.Appendix3istheformwewillusetoconfirmeligibility.Anydisagreementswillbediscussedbeforemov-ingontodataextractionandassessmentofriskofbias.Athirdreviewerwillbebroughtintoarbitrateifnecessary.DataextractionAsecondreviewerandIwillindependentlyextractdataaboutthepopulation,studydesign,criteriausedtoassessthequalityofthereports,andtheresults.Wewillagaindiscussanydisagreements,bringinginathirdreviewertoarbi-trateifnecessary.Appendix4istheformwewillusetoextractdataaboutpopulationanddesign.Appendix5istheseparatespreadsheetinwhichwewillentertheindividualcriteriaandresultsforeachcriterion.Ihavedeterminedwhatdatatoextractbasedonwhatisreportedinthepoten-tiallyeligiblestudiesofwhichIamaware(Appendix1),varioustools(Vandenbroucke,etal.,2007;TheJoannaBriggsInstitute,2014;Moher,etal.,2015;Munn,etal.,2015),myexpertise,andcommonsense.Appendix6con-tainsdummytablesforallthedatatobeextracted,aswellastheriskofbiasas-sessmentsandanalyses.PopulationAsecondreviewerandIwillextractdataaboutthefollowingvariables,intermsofthesamplesofnewsreports:mediumofpublication;geographicareaofpub-lication;timeperiodofpublication;categoryofinterventionreportedon;andwhethertheoutletiscommercialornot.Intermsofmedium,radioandpod-castswillbeonecategory,inadditiontothecategories:newspapers;magazines;television;andnewswebsites.Theninepredeterminedcategoriesofinterven-tionare:“modern”medicine(aka.“academic”,“conventional”or“Western”medicine);“alternative”medicine(aka.“complementary”,“traditional”or“natu-ral”medicine);screening;surgery;devices;diet;exercise;lifestyle;andsystemsandpolicies.StudydesignForeachstudy,wewillrecord:theinstrumentusedtomeasurequality;thestatedobjectives;thesamplingframe;theselectioncriteriaandtechnique;andanyreportedsubgroupanalyses.
![Page 17: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/17.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 17
Criteria,responseoptionsandresultsFinally,wewillrecordeacheligiblecriterionusedtoassessthequalityofre-ports,aswellastheresponseoptions,inaspreadsheetseparatefromtheformusedtocollectdataaboutthepopulationandstudydesign(Appendix5).Wewillentertheresultsforeachcriterionintothesamespreadsheet.Wheredataareavailableforsubgroupsofnewsreports,wewillrecordtheoverallresultsandtheresultsforeachsubgroup.AssessingriskofbiasInasystematicreviewoftoolsusedtoassessthequalityofprevalencestudiesandotherobservationalresearch,Shamliyanetal.identified46scalesand51checklist,noneofwhichdiscriminatedbetweenreportingversusmethodologi-calquality,orexternalversusinternalvalidity(2010).Throughcitationsearch-es,Iidentifiedseveraltoolsdevelopedafterthereview,ofwhichIhavechosentousethelatest,developedbyMunnetal.(2014),asastartingpoint,basedonfacevalidityandsimplicity.WhileMunnetal.donotrefertothereviewbyShamliyanandcolleagues,theycomparetheirtooltoonedevelopedbyHoyetal.(2012)whodoconsidersaidreview.ThecriticalappraisaltooldevelopedbyMunnetal.includestenitems,withtheresponseoptions“Yes”,“No”,“Unclear”and“Notapplicable”:
1. Wasthesamplerepresentativeofthetargetpopulation?2. Werestudyparticipantsrecruitedinanappropriateway?3. Wasthesamplesizeadequate?4. Werethestudysubjectsandthesettingdescribedindetail?5. Wasthedataanalysisconductedwithsufficientcoverageoftheidenti-
fiedsample?6. Wereobjective,standardcriteriausedforthemeasurementofthecondi-
tion?7. Wastheconditionmeasuredreliably?8. Wasthereappropriatestatisticalanalysis?9. Areallimportantconfoundingfactors/subgroups/differencesidentified
andaccountedfor?10. Weresubpopulationsidentifiedusingobjectivecriteria?
Ofthetenitems,threeassessriskofbiasandarerelevantandapplicabletothisreview:items2,6and7.Item2assessesriskofselectionbias,whileitems6and7measuredetectionbias(aka.Informationbias).FocusingonthesetwotypesofbiasisconsistentwiththeStrengtheningtheReportingofObservationalStudiesinEpidemiology(STROBE)statement(Vandenbroucke,etal.,2007,p.814)andmethodsusedbytheCochraneMethodologyReviewGrouptosystematicallyreviewtheprevalenceofaconditioninapopulationofinformation,asopposedtopeople(Welch,etal.,2010,p.7).AsecondreviewerandIwillindependentlyassessriskofbiasineachstudy,cri-terionbycriterion,giventhatsomecriteriausedtomeasurethequalityofnewsreportsrequiremoresubjectivejudgementthanothers.Theseassessmentswillbeusedtodescribethestrengthsandweaknessesofthestudies.Ifdeemednec-essary,theywillalsobeusedtoconductasensitivityanalysis.Itisunclearhow
![Page 18: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/18.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 18
agreaterriskofbiaswouldimpacttheresultsofstudiestobeincludedinthisreview,soIhavenohypothesisabouttheresultsofsuchananalysis.Intermsofselectionbias,ifthenewsreportswererandomlyorsequentiallysampledfromthetargetpopulation,wewillconsidertheriskofbiasaslow.Ifthereportswerepurposivelysampled,wewillconsidertheriskofbiasashigh.Ifanyothersamplingtechniquewasused,wewillconsidertheriskofbiasasmoderateorhigh.Intermsofdetectionbias,ifacriterionrequiresminimaljudgement(e.g.acrite-rionthatrisksbepresentedinabsolutenumbers),wewillconsidertheriskofbiasaslow.Ifthecriterionrequiressubstantialjudgement(e.g.whetherthereissufficientinformationaboutpotentialharms),tworesearchersassessedthere-ports,andtheywereblindedtothejournalistandpublication,wewillconsidertheriskofbiasasmoderate.Ifthecriterionrequiressubstantialjudgement,andonlyoneresearcherassessedthereportsorresearchersassessingthereportswerenotblindedtothejournalistandpublication,wewillconsidertheriskofbiasashigh.Iftheriskofeitherselectionordetectionbiasishigh,wewillconsidertheover-allriskofbiasashigh.Iftheriskofeitherismoderate,wewillconsidertheriskasmoderate.Appendix6includesadummytablefortheriskofbiasassess-ments.Appendix7istheformthatwewillusetoassessriskofbias.Itincludesaspaceforrecordinganyconflictsofinterests.Anysuchconflictswillbedis-cussedseparatelywheninterpretingtheresultsofthereview.AnalysesCriteriausedtomeasurequalityHowqualityhasbeenmeasuredwillbeassessedbycomparingcriteriatotheIHCKeyConcepts(Box1).Eacheligiblecriterionfromeachstudywillbeen-teredintoaspreadsheet,diagonallyacrossfromthefullconceptlist(Appendix8).AsecondreviewerandIwilljudgewhichconceptsarecapturedbyeachcri-terion,representingthejudgementswithmarksinthespreadsheet.Wewilldis-cussanyagreementsandbringinathirdresearchertoarbitrate,ifnecessary.TheIHCKeyConceptsdifferfromotherlistsofcriteriaformakingcausalinfer-encesintwoways(Chalmers,etal.,2018,pp.30-31).First,thelistisdifferentinthatitisboth:a)developedusingasystematic,transparentanditerativepro-cess;andb)intendedtohelpchildren,thegeneralpublicandhealthpractition-ersmakeinformeddecisions,notjusthealthresearchers.Second,unlikeothertoolssuchaschecklistsandtipsheets,itisnotaninterventioninandofitself,limitedtoaparticularpopulationorcontext,butacomprehensiveframeworkthatcanbeusedformappingskills,developinginterventionsandevaluatingin-terventions.UsingtheIHCKeyConceptlisttoassessthecriteriausedtomeasurequalityofthenewsreportshasseveraladvantages.First,itprovidesaframeworkforgroupingcriteriafromdifferentstudies,soresultscanmoreeasilybesynthe-sised.Second,itwillshedlightonrelevantqualitiesthathavenottypicallybeenmeasuredormeasuredatall.Conversely,itmayrevealconceptscapturedby
![Page 19: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/19.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 19
thecriteriamissingfromtheIHCKeyConceptlist.Finally,usingthelistcanin-formtheinterpretationofresultsfromthisreview,includingwhattheresultssuggestaboutwhatconceptsaremostimportantwhendevelopinginterven-tionsforimprovingreportingontheeffectsofinterventions,orhelpingpeopleassessinformationinsuchreports.AsecondreviewerandIwilljudgewhatcriteriausedindifferentstudiescanbegrouped.Oncemore,wewilldiscussanydisagreementsandbringinathirdre-viewertoarbitrateifnecessary.QualityofnewsreportsAsfarasthedatapermits,wewillmeta-analysetheproportionsofsamplesthateithersatisfiedorfailedtosatisfyrespectivecriteria,i.e.theprevalenceofre-portssatisfyingorfailingtosatisfyagivencriterion.SeedummytablesinAp-pendix6.Wewilldichotomisecategoricaldatawherepossibleandsensible.Furthermore,wemayreframedatathatalreadyisdichotomousassatisfyingornotsatisfyingagivencriterion.Notsatisfyingacriterionwillincludeanythingotherthancompletelysatisfyingit.Inotherwords,ifnewsreports“partially”satisfyacriterionoritisunclearwhetherthereportssatisfyit,wewillconsiderthereportsasnothavingsatis-fiedthecriterion.Ifacriterionhasbeendeemedinapplicabletoreports,wewillexcludethosereportsfromthemeta-analysisforthatcriterion.Inadditiontopotentiallygivinguslargersamplesformeta-analysis,thedichotomisationwillmakeinterpretingtheresultsmorestraightforward.Acrossstudies,Ianticipatereportswillgenerallyfailtosatisfymostcriteria,de-spitedifferencesinpopulationsandmethods.Therefore,Iexpectmeanpropor-tionswith95%confidenceintervalswillbemeaningful.Iwillprepareforestplotsandvisuallyanalysetheextenttowhichtherearemeaningfuldifferencesbetweenpointestimates,andIwillconductaChi-squaretesttoseeiftheheter-ogeneityislargerthanonewouldexpectbychance.Foragivencriterion,ifthevisualinterpretationandChi-squaretestsuggestitisappropriate,Iwillconductameta-analysisusingarandomeffectsmodel.Iwillalsoconductsixsubgroupanalyses,whendataareavailable.VariationinqualityAgain,Idonotexpectalotofvariationinthequalityofreportsacrossstudies.Totheextentthatthereisvariation,Idonotexpectalotofdataforexploringit.Thatsaid,thereareseveralvariablesthatcouldlogicallyexplainvariation,evi-dencesupportingsomeofthosehypotheses,andreasonstotestthem.Table1isanoverviewof:thevariablesforwhichwewillconductsubgroupanalyses;thesubgroups;thehypothesiseddifferences;andtherationalesforthehypothesesandforconductingtheanalyses.Thevariablesare:themediuminwhichareportwaspublished;thetimeperiodinwhichareportwaspub-lished;whetherthepublishingoutletwascommercial;whetherthejournalistwasspecialisedinhealthorscience;andtheincomelevelofthecountryinwhichtheoutletissituated.
![Page 20: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/20.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 20
Ontheonehand,Iwillbecautiousaboutextrapolatingfindingsifthereisnodi-rectevidenceforasubpopulation(e.g.broadcastreports)andthereisreasontobeuncertainabouttheapplicabilityoftheevidencefromothersubpopulations(e.g.printandonlinereports).Ontheotherhand,subgroupanalysesarefre-quentlymisleading,asshownbySunetal.(2014).Toavoidspuriousresults,wewilluseexplicitcriteriaforassessingthecredibil-ityofanydifference,basedonSunandcolleagues’criteria:
• Thedifferenceisofpracticalimportance.• Thedifferenceisbiggerthanwewouldexpectbychance.• Thedifferenceisinthehypothesiseddirection.• Thedifferenceisunlikelytobeexplainedbyotherknownvariables.• Thedifferenceisconsistentacrossstudieswithdifferentmethods.• Thedifferenceisconsistentacrossstudieswithdifferentoutcome
measures.• Thedifferenceisconsistentacrossstudieswithdifferentlevelsofriskof
bias.Foreachexplanatoryvariable,givensufficientdata,wewillconductunivariateregressionanalyseswiththevariable(e.g.medium)astheindependentvaria-ble,andwiththeproportionofsubpopulations(e.g.reportsfromrespectivedecades)thatsatisfythecriterionasthedependentvariable.Wewillalsocon-ductmultipleregressionanalyseswithalloftheexplanatoryvariables.Wewillnoteanywithin-studysubgroupanalyseswheretheresearchershaveusedap-propriatestatisticalmethodsorprovidedthenecessarydataforustodoanap-propriatestatisticalanalysisourselves.Wewillconsiderthoseanalyseswhenmakingjudgementsaboutthecredibilityofsubgroupdifferences.Ihavedecidedagainstconductingasubgroupanalysisforthetypeofreport,i.e.whetherthereportisafeatureorabreakingnewsreport.Featuresaretypicallylongerinwordsandpreparedonlongerdeadlines.Theymightlogicallybehigherqualitythanbreakingnewsreports.However,ananalysisbytypeofre-portwouldlikelybeconfoundedbymedium,financialmodel,andspecialisation.Furthermore,howresearchershavedefinedfeatureversusbreakingnews,andhowtheyhavemeasuredlengthofreportordeadline,maybesodifferentacrossstudiesthatcombiningthedatawouldnotmakesense.
![Page 21: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/21.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 21
Table1:SubgroupanalysesVariable Grouping Hypothesis RationaleandimportanceMedium Broadcastvs.other(print
andonline)Broadcastreportsarelesslikelytosatisfycriteria
Broadcastallowsforfewerwords,aswellaslesstimeforwriting,giventimespentrecordingandediting.Thiswouldlogicallyhaveanegativeimpactonthequalityofreports.Iamawareofempiricalevidencethatsupportsthishypothesis(Walsh-Childers,etal.,2016).Acredibledifferenceinthehypothesiseddirectionwouldimplyconsumersshouldpaylessattentiontobroadcastreports.
Timeperiod Decades(≥2000;1990-1999;1980-1989;etc.)
Theproportionofreportssatisfyingcriteriaislikelytobesimilaracrossdecades
Ontheonehand,newsreportsmayhaveimprovedsincetheadventof“evidence-basedmedicine”(EBM)(Evidence-BasedMedicineWorkingGroup,1992).Ontheother,therearesustained,systemicbarrierstoreliablehealthjournalismsuchaslackoftime(Larsson,etal.,2003;Pettersen,2005),whichEBMdoesnotaddress.Moreover,revenuelossesmayhavelednewsmediatoinvestlessinstrategiesthatmightimprovethequalityoftheirhealthnews,suchastrainingjournalistsandeditors,andhiringandretainingspecialisedhealthandsciencereporters.However,acredibleimprovementforoneormorecriteriawouldbeanimpetusforresearchtoexplainsaidimprovement,andthatresearchcouldinformthedevelopmentofinterventionstoimprovequalityoverall.
Financialmodel Commercialvs.non-commercial
Reportspublishedbycommercialoutletsarelesslikelytosatisfycrite-ria
Thefinancialinterestsofcommercialoutletsmayleadtosensationalism,toattractaudiences(Larsson,etal.,2003;Pettersen,2005;Wallington,etal.,2010).Moreover,companiesthatsellhealthinterventionsmaybeadvertisers,disincentivisingcriticalreporting.Acredibledifferenceinthehypothesiseddirectionwouldimplyconsumersshouldsupportnon-commercialoutletsandpaylessattentiontoreportspublishedbycommercialoutlets.
Specialisation Healthandsciencejour-nalistsvs.other
Reportspreparedbyhealthorsci-encejournalistsaremorelikelytosatisfycriteria
Journalistwhohavespecialisedinhealthorsciencearemorelikelytohavereceivedtrainingincriticalappraisal.Moreover,comparedtootherjournalists,theymayhavebeeninmorecontactwithscientists,andmaybemorefamiliarwiththelanguage,contentandstructureofresearchpapers.Thereisevidencethatjournalistswithmoreadvanceddegreesaremorelikelytousescientificjournalarticlesassources,andthatjournalistswithfeweryearsofexperiencearemorelikelytosayprovidingenter-tainmentisanimportantpriorityforhealthnews(Wallington,etal.,2010).Acredibledifferenceinthehypothesiseddirectionwouldimplyconsumersshouldpaymoreattentiontoreportspreparedbyspecialisedhealthandsciencejournalists.
Countryincomelevel
Low-incomecountriesvs.other(middleandhigh-incomecountries)
Theproportionofreportssatisfyingcriteriaislikelytobesimilaracrosscountrieswithdifferentincomelevels
Whilenewsmediainmiddleandhigh-incomecountriesmayhavemoreresources,journalistsinthosecountriesstillfacesys-temicbarrierstohigh-qualityreporting(Larsson,etal.,2003;Pettersen,2005;Wallington,etal.,2010).Supportforthishypoth-esiswouldsuggestresearchersshoulddevelopinterventionsthatcanimprovethequalityofreportinginlow-incomecountriesandmiddleorhigh-incomecountriesalike.Itwouldalsoimplythoseinterventionsaddressbarriersbesidelackofresources.
Newspapertype Broadsheetvs.tabloid Reportsinbroadsheetnewspapersaremorelikelytosatisfycriteria
Reportingintabloidnewspapersmaybelessdetailedandmoresensationalthanreportinginbroadsheetnewspapers,andIamawareofevidencethatthequalityofreportsabouttheeffectsofhealthinterventionsislowerintabloids(Entwistle&Hancock-Beaulieu,1992;Robinson,etal.,2013).Acredibledifferenceinthehypothesiseddirectionwouldimplyconsumersshouldpaymoreattentiontobroadsheetnewspapers,andlesstotabloids.
![Page 22: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/22.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 22
PilotHavingconductedthesearch,wewillpilotforms,spreadsheetsandtables(Ap-pendixes3through6)onthefirstfiveeligiblestudies.Wewilldiscussanyis-suesandmakeanyrevisionsdeemednecessarybeforecompletingdataextrac-tion,riskofbiasassessmentsandanalysesforremainingstudies.RatingthequalityoftheevidenceIhaveadjustedtheGradingofRecommendationsAssessment,DevelopmentandEvaluation(GRADE)approachtoratingthequalityofevidence(Guyatt,etal.,2008).TheGRADEapproachinvolvesconsideringfivefactorsforloweringthecertaintyofevidence:studylimitations,inconsistencyofresults,indirectnessofevidence,imprecision,andpublicationbias.Ihaveadjustedthese,giventhatGRADEwasdevelopedforresearchabouteffects,notprevalence.Appendix9istheformwewillusetoratethequalityoftheevidence.Thefirstadjustmentisthatwewillnotconsiderpublicationbias.Iamnotawareofanyresearchdocumentingpublicationbiasforprevalenceorincidencestudiesgenerally,norspecificallyinthisarea.Besides,Iexpecttoofewstudiesandsamplesthataretoosmallforittobemeaningfultoassesstheriskofpubli-cationbias.Second,wewillnotconsiderdirectnesssinceitisinapplicable.Iwill,however,becautiousaboutextrapolatingevidenceacrosssubpopulationsspecifiedinTable1(e.g.acrossreportsfromdifferenttimeperiods),ifthereisnoevidenceforoneofthesubpopulationsspecific(e.g.reportsfrombeforeaparticulardec-ade).Thisleavesconsideringriskofbias,imprecisionandinconsistency.Intermsofimprecision,wewillusequartilesasaruleofthumb;iftheconfidenceintervalsubstantiallycrossestwoormorequartiles,wewillconsidertheretobeim-portantimprecision,followingguidancedevelopedbytheCochraneEffectivePracticeandOrganisationofCare(EPOC)group(CochraneEffectivePracticeandOrganisationofCare,2017).GRADEfactorsforincreasingthecertaintyoftheevidencearenotrelevanthere.
![Page 23: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/23.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 23
Ethicalconsiderationsandconflictsofinterest
Thereviewdoesnotinvolveparticularethicalchallenges.IdeclarethatIhavenorelevantconflictsofinterest.
![Page 24: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/24.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 24
References
Ashorkhani,M.etal.,2012.Qualityofhealthnewsdisseminatedintheprintmediaindevelopingcountries:AcasestudyinIran.BMCPublicHealth,12(627).Austvoll-Dahlgren,A.etal.,2017.Measuringabilitytoassessclaimsabouttreatmenteffects:alatenttraitanalysisofitemsfromthe‘ClaimEvaluationTools’databaseusingRaschmodelling.BMJOpen,Volume7,p.e013184.Austvoll-Dahlgren,A.,Nsangi,A.&Semakula,D.,2016.Interventionsandassessmenttoolsaddressingkeyconceptspeopleneedtoknowtoappraiseclaimsabouttreatmenteffects:asystematicmappingreview.SystematicReviews,5(215).Austvoll-Dahlgren,A.etal.,2015.Keyconceptsthatpeopleneedtounderstandtoassessclaimsabouttreatmenteffects.JournalofEvidence-BasedMedicine,Volume8,pp.112-125.Austvoll-Dahlgren,A.etal.,2016.Measuringabilitytoassessclaimsabouttreatmenteffects:thedevelopmentofthe‘ClaimEvaluationTools’.BMJOpen,Volume7,p.e013184.Berkman,N.D.etal.,2011.Lowhealthliteracyandhealthoutcomes:Anupdatedsystematicreview.AnnalsofInternalMedicine,155(2),pp.97-107.Brownlee,S.etal.,2017.Evidenceforoveruseofmedicalservicesaroundtheworld.Lancet,390(10090),pp.156-168.Castell,S.etal.,2014.PublicAttitudestoScience2014,London:IpsosMORI.Chalmers,I.etal.,2018.KeyConceptsforInformedHealthChoices:aframeworkforhelpingpeoplelearnhowtoassesstreatmentclaimsandmakeinformedchoices.BMJEvidence-BasedMedicine,23(1),pp.29-33.Charnock,D.,Shepperd,S.,Needham,G.&Gann,R.,1999.DISCERN:aninstrumentforjudgingthequalityofwrittenconsumerhealthinformationontreatmentchoices.JournalofEpidemiologyandCommunityHealth,Volume53,pp.105-111.Chew,C.&Eysenbach,G.,2010.PandemicsintheAgeofTwitter:ContentAnalysisofTweetsduringthe2009H1N1Outbreak.PLoSONE,5(11),p.e14118.CochraneEffectivePracticeandOrganisationofCare,2017.ReportingtheeffectsofaninterventioninEPOCreviews.s.l.:CochraneEffectivePracticeandOrganisationofCare.Coulter,A.,Entwistle,V.&Gilbert,D.,1999.Sharingdecisionswithpatients:istheinformationgoodenough?.BMJ,Volume318,pp.318-322.
![Page 25: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/25.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 25
Culver,J.D.,Gerr,F.&Frumkin,H.,1997.MedicalinformationontheInternet:Astudyofanelectronicbulletinboard.JournalofGeneralInternalMedicine,12(8),pp.466-470.Entwistle,V.&Hancock-Beaulieu,M.,1992.HealthandmedicalcoverageintheUKnationalpress.PublicUnderstandingofScience,Volume1,pp.367-382.Evidence-BasedMedicineWorkingGroup,1992.Evidence-BasedMedicine:ANewApproachtoTeachingthePracticeofMedicine.JAMA,268(17),pp.2420-2425.Faerber,A.E.&Kreling,D.H.,2013.ContentAnalysisofFalseandMisleadingClaimsinTelevisionAdvertisingforPrescriptionandNonprescriptionDrugs.JournalofGeneralInternalMedicine,29(1),pp.110-118.Frosch,D.L.etal.,2007.CreatingDemandforPrescriptionDrugs:AContentAnalysisofTelevisionDirect-to-ConsumerAdvertising.AnnalsofFamilyMedicine,5(1),pp.6-13.Frosch,D.L.,May,S.G.,Tietbohl,C.&Pagán,J.A.,2011.Livinginthe“landofno”?Consumerperceptionsofhealthylifestyleportrayalsindirect-to-consumeradvertisementsofprescriptiondrugs.SocialScience&Medicine,Volume73,pp.995-1002.Glasziou,P.etal.,2017.Evidenceforunderuseofeffectivemedicalservicesaroundtheworld.Lancet,390(10090),pp.169-177.Glenton,C.,Paulsen,E.J.&Oxman,A.D.,2005.PortalstoWonderland:Healthportalsleadtoconfusinginformationabouttheeffectsofhealthcare.BMCMedicalInformaticsandDecisionMaking,5(7).Groven,K.S.&Braitwaite,J.,2016.Happily-ever-after:Personalnarrativesinweight-losssurgeryadvertising.HealthCareforWomenInternational,37(11),pp.1221-1238.Guyatt,G.H.etal.,2008.GRADE:whatis“qualityofevidence”andwhyisitimportanttoclinicians?.BMJ,Volume336,pp.995-998.Higgins,J.&Green,S.eds.,2011.Definingthescopeofareviewquestion(broadversusnarrow).In:CochraneHandbookforSystematicReviewsofInterventions.s.l.:TheCochraneCollaboration.HLS-EUConsortium,2013.ComparativereportofhealthliteracyineightEUmembersstates,s.l.:s.n.Hoffmann,T.C.&DelMar,C.,2015.Patients’ExpectationsoftheBenefitsandHarmsofTreatments,Screening,andTests:ASystematicReview.JAMAInternalMedicine,175(2),pp.274-286.Hoffmann,T.C.&DelMar,C.,2017.Clinicians’ExpectationsoftheBenefitsandHarmsofTreatments,Screening,andTests:ASystematicReview.JAMAInternalMedicine,177(3),pp.407-419.Hoy,D.etal.,2012.Assessingriskofbiasinprevalencestudies:modificationofanexistingtoolandevidenceofinterrateragreement.JournalofClinicalEpidemiology,Volume65,pp.934-939.
![Page 26: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/26.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 26
Kable,A.K.,Pich,J.&Maslin-Prothero,S.E.,2012.Astructuredapproachtodocumentingasearchstrategyforpublication:A12stepguidelineforauthors.NurseEducationToday,Volume32,pp.878-886.Kutner,M.,Greenberg,E.,Jin,Y.&Paulsen,C.,2006.TheHealthLiteracyofAmerica’sAdults:ResultsFromthe2003NationalAssessmentofAdultLiteracy,Washington,DC:NationalCenterforEducationStatistics.Larsson,A.,Oxman,A.D.,Carling,C.&Herrin,J.,2003.Medicalmessagesinthemedia–barriersandsolutionstoimprovingmedicaljournalism.HealthExpectations,Volume6,pp.323-331.Marcon,A.R.,Murdoch,B.&Caulfield,T.,2017.Fakenewsportrayalsofstemcellsandstemcellresearch.RegenerativeMedicine,12(7),pp.765-775.Moher,D.etal.,2015.Preferredreportingitemsforsystematicreviewandmeta-analysisprotocols(PRISMA-P)2015statement.Systematicreviews,4(1).Moult,B.,Franck,L.S.&Brady,H.,2004.EnsuringQualityInformationforPatients:developmentandpreliminaryvalidationofanewinstrumenttoimprovethequalityofwrittenhealthcareinformation.HealthExpectations,Volume7,pp.165-175.Moynihan,R.etal.,2000.Coveragebythenewsmediaofthebenefitsandrisksofmedications.TheNewEnglandJournalofMedicine,342(22),pp.1645-1650.Munn,Z.etal.,2015.Methodologicalguidanceforsystematicreviewsofobservationalepidemiologicalstudiesreportingprevalenceandcumulativeincidencedata.InternationalJournalofEvidence-BasedHealthcare,Volume13,pp.147-153.Munn,Z.,Moola,S.,Riitano,D.&Lisy,K.,2014.Thedevelopmentofacriticalappraisaltoolforuseinsystematicreviewsaddressingquestionsofprevalence.InternationalJournalofHealthPolicyandManagement,3(3),pp.123-128.Nsangi,A.etal.,2017.EffectsoftheInformedHealthChoicesprimaryschoolinterventionontheabilityofchildreninUgandatoassessthereliabilityofclaimsabouttreatmenteffects:acluster-randomisedcontrolledtrial.Lancet,390(10092),pp.374-388.Optum,2013.Effectivenessofhomeopathyforclinicalconditions:Evaluationoftheevidence,s.l.:Optum.O'Sullivan,J.W.etal.,2018.Overtestingandundertestinginprimarycare:asystematicreviewandmeta-analysis.BMJOpen,Volume8,p.e018557.Oxman,A.D.,Austvoll-Dahlgren,A.,Garrat,A.&Rosenbaum,S.,2017.Understandingofkeyconceptsrelevanttoassessingclaimsabouttreatmenteffects:asurveyofNorwegianadults.IHCWorkingPaper.Oslo:InformedHealthChoices.Oxman,A.D.etal.,1993.AnIndexofScientificQualityforHealthReportsintheLayPress.JournalofClinicalEpidemiology,46(9),pp.987-1001.Pettersen,S.,2005.CriticalThinkinginNorwegianUpperSecondaryBiologyEducation:TheCasesofComplementary-Alternative-MedicineandHealthClaimsintheMedia.Nordina,1(2),pp.61-71.
![Page 27: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/27.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 27
Pettersen,S.,2005.Norwegianhealthjournalists’abilitytoreportonhealthresearch:Aconcerntoscienceeducation?.Nordina,1(1),pp.5-16.Pettersen,S.,2007.HealthClaimsandScientificKnowledge:Astudyofhowstudentsofhealthsciences,theirteachers,andnewspaperjournalistsrelatetohealthclaimsinsociety.Oslo:UniversityofOslo.Robinson,A.,Coutinho,A.,Bryden,A.&McKee,M.,2013.AnalysisofhealthstoriesindailynewspapersintheUK.PublicHealth,Volume127,pp.39-45.Sansgiry,S.,Sharp,W.T.&Sansgiry,S.S.,1999.AccuracyofInformationonPrintedOver-the-CounterDrugAdvertisements.HealthMarketingQuarterly,17(2),pp.7-18.Semakula,D.etal.,2017.EffectsoftheInformedHealthChoicespodcastontheabilityofparentsofprimaryschoolchildreninUgandatoassessclaimsabouttreatmenteffects:arandomisedcontrolledtrial.Lancet,390(10092),pp.389-398.Shamliyan,T.,Kane,R.L.&Dickinson,S.,2010.Asystematicreviewoftoolsusedtoassessthequalityofobservationalstudiesthatexamineincidenceorprevalenceandriskfactorsfordiseases.JournalofClinicalEpidemiology,Volume63,pp.1061-1070.Shoemaker,P.J.&Cohen,A.A.,2006.Part1:TheoryandMethod.In:P.J.Shoemaker&A.A.Cohen,eds.NewsAroundtheWorld.NewYork:Routledge,pp.1-34.Smith,D.E.,Wilson,A.J.&Henry,D.A.,2005.Monitoringthequalityofmedicalnewsreporting:earlyexperiencewithmediadoctor.MJA,Volume183,pp.190-193.Spencer,E.A.,Mahtani,K.R.,Goldacre,B.&Heneghan,C.,2016.Claimsforfertilityinterventions:asystematicassessmentofstatementsonUKfertilitycentrewebsites.BMJOpen,Volume6,p.e013940.Starr,R.R.,2015.TooLittle,TooLate:IneffectiveRegulationofDietarySupplementsintheUnitedStates.AmericanJournalofPublicHealth,105(3),pp.478-485.Stephens,M.,2007.AHistoryofNews.3rdEditioned.NewYork:OxfordUniversityPress.Sun,X.etal.,2014.HowtoUseaSubgroupAnalysis:Users'GuidestotheMedicalLiterature.JAMA,311(4),pp.405-411.TheAcademyofMedicalSciences,2016.TheAcademyofMedicalSciences:MedicalInformationSurvey,London:TheAcademyofMedicalSciences.TheJoannaBriggsInstitute,2014.Thesystematicreviewofprevalenceandincidencedata.Adelaide:TheJoannaBriggsInstitute.UnitedStatesGovernmentAccountabilityOffice,2010.HerbalDietarySupplements:ExamplesofDeceptiveorQuestionableMarketingPracticesandPotentiallyDangerousAdvice,s.l.:UnitedStatesGovernmentAccountabilityOffice.
![Page 28: Quality of news media reports about the effects and costs ... · cepts. In Norway, where I am based, Oxman et al. tested a random sample of 626 Norwegian adults (Oxman, et al., 2017)](https://reader033.vdocuments.site/reader033/viewer/2022042016/5e74cb7ddc793c26fc71ef51/html5/thumbnails/28.jpg)
Quality of news media reports about the effects and costs of health interventions: Systematic review protocol 28
Vandenbroucke,J.P.etal.,2007.StrengtheningtheReportingofObservationalStudiesinEpidemiology(STROBE):Explanationandelaboration.Epidemiology,18(6),pp.805-835.Wallington,S.F.,Blake,K.,Taylor-Clark,K.&Viswanath,K.,2010.AntecedentstoAgendaSettingandFraminginHealthNews:AnExaminationofPriority,Angle,Source,andResourceUsagefromaNationalSurveyofU.S.HealthReportersandEditors.JournalofHealthCommunication,15(1),pp.76-94.Walsh-Childers,K.,Braddock,J.,Rabaza,C.&Schwitzer,G.,2016.OneStepForward,OneStepBack:ChangesinNewsCoverageofMedicalInterventions.HealthCommunication,33(2),pp.174-187.Welch,V.etal.,2010.Howeffectsonhealthequityareassessedinsystematicreviewsofinterventions(Review).CochraneDatabaseofSystematicReviews,Issue12.Wolfe,R.M.,Sharp,L.K.&Lipsky,M.S.,2002.ContentandDesignAttributesofAntivaccinationWebSites.JAMA,287(24),pp.3245-3248.Woloshin,S.,Schwartz,L.M.&Welch,H.G.,2008.Knowyourchances:understandinghealthstatistics.Berkeley:UniversityofCaliforniaPress.Zeraatkar,D.,Obeda,M.,Ginsberg,J.S.&Hirsh,J.,2017.Thedevelopmentandvalidationofaninstrumenttomeasurethequalityofhealthresearchreportsinthelaymedia.BMCPublicHealth,17(343).