gordon guyatt department of health research … guyatt department of health research methods...
TRANSCRIPT
![Page 1: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/1.jpg)
Gordon GuyattDepartment of Health Research MethodsEvidence and ImpactMcMaster University
![Page 2: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/2.jpg)
¡ Co-chair of the GRADE working group
¡ Have written about dangers of conflict of interest – therefore, when it comes to intellectual conflict of interest, I have intellectual conflict of interest
![Page 3: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/3.jpg)
¡ Current problems in health care
¡ What is a guideline?
¡ What makes a trustworthy guideline?§ Collect and summarize evidence§ Move from evidence to recommendations
![Page 4: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/4.jpg)
¡ Slow dissemination of innovations
¡ Related problems especially U.S§ Over diagnosis, overtreatment§ Low value care, waste§ Runaway costs
¡ Trustworthy guidelines could help
![Page 5: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/5.jpg)
Clinical guidelines - Not guidelines on how to do tests in the laboratory
![Page 6: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/6.jpg)
¡ Systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.
¡ Most guidelines deal with therapeutic issues
¡ Guidelines in diagnosis present special challenges
¡ GOBSAT or trustworthy
![Page 7: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/7.jpg)
¡ Right panel ¡ Deal with conflict of interest
¡ Clearly articulated question¡ Gather and summarize the best evidence
¡ Rate quality of evidence
¡ Consideration of values and preferences
¡ Rate strength of clear recommendations
![Page 8: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/8.jpg)
¡ Right panel ¡ Deal with conflict of interest
¡ Clearly articulated question¡ Gather and summarize the best evidence
¡ Rate quality of evidence
¡ Consideration of values and preferences
¡ Rate strength of clear recommendations
![Page 9: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/9.jpg)
¡ Experts§ Concern about conflict of interest
¡ Methodologists§ Concern about conflict of interest
¡ Front-line clinicians
¡ Patients§ Concern about representativeness
![Page 10: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/10.jpg)
¡ Right panel ¡ Deal with conflict of interest
¡ Clearly articulated question¡ Gather and summarize the best evidence
¡ Rate quality of evidence
¡ Consideration of values and preferences
¡ Rate strength of clear recommendations
![Page 11: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/11.jpg)
¡ Types§ Financial, Intellectual, Professional
¡ Possible approaches§ Ignore - GOBSAT§ Declare § Forbid § Manage
▪ Balance▪ Selectively forbid
![Page 12: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/12.jpg)
¡ 500 recommendations, 14 panels
¡ COI tension
¡ COI financial § Grant funding, speaker’s bureaus, § Consultancy, advisory board, ownership
¡ COI intellectual§ Authorship paper directly bearing recommendations
¡ Conflict, no participation make recommendation
![Page 13: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/13.jpg)
![Page 14: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/14.jpg)
¡ Right panel ¡ Deal with conflict of interest
¡ Clearly articulated question¡ Gather and summarize the best evidence
¡ Rate quality of evidence
¡ Consideration of values and preferences
¡ Rate strength of clear recommendations
![Page 15: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/15.jpg)
¡ Partial thromboplastin time accurate test (more or less) § Pre-op PTT in major surgery improves outcomes
¡ D-dimer accurate test (more or less)¡ Suspected DVT, high probability improves outcomes¡ Suspected DVT, low probability improves outcomes
¡ ELISA for HIT accurate test (more or less)¡ ELISA any pt suspected HIT improves outcomes¡ ELISA in patients with intermediate or high probability 4T
score improves outcomes
![Page 16: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/16.jpg)
¡ Very low prevalence§ False positive harm outweigh benefits
¡ Clinicians misinterpret test result¡ Clinicians just as good without the test¡ Test results don’t lead to change in
management¡ Clinicians don’t offer optimal management¡ Optimal management doesn’t improve
outcomes
![Page 17: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/17.jpg)
¡ PICO format¡ Pt: Suspect heparin induced thrombocytopenia
¡ Intervention: 4T + ELISA § Low 4T heparin; Intermediate or high on ELISA result
¡ Comparator: 4T § heparin for low, argatroban for intermediate and high
¡ Outcomes:§ Thrombosis § Bleeding§ Mortality§ Resource use (cost)
![Page 18: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/18.jpg)
¡ Right panel ¡ Deal with conflict of interest
¡ Clearly articulated question¡ Gather and summarize the best evidence
¡ Rate quality of evidence
¡ Consideration of values and preferences
¡ Rate strength of clear recommendations
![Page 19: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/19.jpg)
¡ Systematic reviews of best evidence
¡ What makes a systematic review systematic?§ Explicit eligibility criteria§ Comprehensive search§ Explicit consideration of evidence limitations
▪ Risk of bias, Imprecision, Inconsistency
§ Duplicate judgements (eligibility, RoB)§ Appropriate analysis
![Page 20: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/20.jpg)
Study designs RCTs
Studies inform mortality, morbidity, symptoms, quality of life, resource use
Randomised Trial orObservational Study
Accuracy Study
Target population
New test(s)Old test(s) New test(s) + Reference test
One
step
inference
Twostepinference
Managementdependingon results
Patient-important outcomes
TP + FP FN + TN TP FP FN TN
Assumptionsor indirect evidence aboutmanagementof patients correctly or incorrectly classified aspositive or negative with the newor old test(s)
Judgements about patient-important outcomeswith a new test and a reference test
Managementdependingon results
Managementdependingon results
Patient-important outcomes
TP + FP FN + TN
Managementdependingon results
Target population
![Page 21: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/21.jpg)
Study designs Linked EvidenceLook for diagnostic test accuracy studies
And then draw inferences from other evidence
Randomised Trial orObservational Study
Accuracy Study
Target population
New test(s)Old test(s) New test(s) + Reference test
One
step
inference
Twostepinference
Managementdependingon results
Patient-important outcomes
TP + FP FN + TN TP FP FN TN
Assumptionsor indirect evidence aboutmanagementof patients correctly or incorrectly classified aspositive or negative with the newor old test(s)
Judgements about patient-important outcomeswith a new test and a reference test
Managementdependingon results
Managementdependingon results
Patient-important outcomes
TP + FP FN + TN
Managementdependingon results
Target population
![Page 22: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/22.jpg)
Strategy4(4Tsscore)
PatientswithsuspectedHIT(n=1000)
Lowprob (n=558)
T(n=1)MB(n=0)
Totaladverseoutcomes:Thrombosis(n=40)Majorbleeds(n=30)
Argatroban Argatroban
Intermediateprob (n=360) Highprob (n=82)
HIT(n=1)
NotHIT(n=557)
HIT(n=50)
NotHIT(n=310)
HIT(n=52)
NotHIT(n=30)
Notx
T(n=18)MB(n=3)
T(n=9)MB(n=3)
T(n=3)MB(n=19)
T(n=9)MB(n=3)
T(n=0)MB(n=2)
![Page 23: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/23.jpg)
Strategy5(4Tsscore+PolyspecificELISA):Testpatientswithanintermediate/high4Tsscore
andtreatifpositive
PatientswithsuspectedHIT(n=1000)
Lowprob (n=558)
T(n=1)MB(n=0)
Totaladverseoutcomes:Thrombosis(n=48versus40)Majorbleeds(n=12versus30)
Intermediateprob (n=360) Highprob (n=82)
HIT(n=1)
NotHIT(n=557)
HIT(n=52)
NotHIT(n=30)
Notx
T(n=18)MB(n=3)
HIT(n=50)
NotHIT(n=310)
ELISA+(n=48)
ELISA-(n=2)
ELISA+(n=41)
ELISA-(n=269)
T(n=18)MB(n=6)
ELISA+(n=50)
ELISA-(n=2)
ELISA+(n=4)
ELISA-(n=26)
T(n=11)MB(n=3)
![Page 24: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/24.jpg)
¡ Right panel ¡ Deal with conflict of interest
¡ Clearly articulated question¡ Gather and summarize the best evidence
¡ Rate quality of evidence
¡ Consideration of values and preferences
¡ Rate strength of clear recommendations
![Page 25: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/25.jpg)
¡ Many available
¡ Cause of confusion, dismay
¡ GRADE (Grades of recommendation, assessment, development and evaluation)
¡ Guidance: BMJ 2004, 6 part series 2008
![Page 26: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/26.jpg)
>100organizationshaveadoptedGRADE
![Page 27: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/27.jpg)
AssessingQualityoftheEvidencebyOutcome
Table:'GRADE's'approach'to'rating'quality'of'evidence'(aka'confidence'in'effect'estimates)'For$each$outcome$based$on$a$systematic$review$and$across$outcomes$(lowest$quality$across$the$outcomes$critical$for$decision$making)
1.''Establish'initial'level'of'certainty'
' 2.''Consider'lowering'or'raising'
level'of'certainty'
' 3.''Final'level'of''
certainty'rating'
Study&design& Initial&certainty&&in&the&evidence&
' Reasons&for&considering&lowering&&or&raising&certainty&&
' Certainty&&in&the&evidence&&
across&those&considerations&''! 'Lower'if' ''" 'Higher'if*'
Randomized&trials! & High'certainty'
Risk'of'Bias!
Inconsistency!
Indirectness!
Imprecision'
Publication'bias'
Large'effect'
Dose'response'
All'plausible''confounding'&'bias'• would!reduce!a!demonstrated!effect!!
'''or'• would!suggest!a!spurious!effect!if!no!effect!was!observed!
High!⊕⊕⊕⊕'
! ' Moderate'⊕⊕⊕!'
Observational&studies! & Low'certainty'
Low'⊕⊕!!'
! ' Very'low'⊕!!!'
*upgrading criteria are usually applicable to observational studies only.
![Page 28: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/28.jpg)
¡ Right panel ¡ Deal with conflict of interest
¡ Clearly articulated question¡ Gather and summarize the best evidence
¡ Rate quality of evidence
¡ Consideration of values and preferences
¡ Rate strength of clear recommendations
![Page 29: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/29.jpg)
Whose values?: Patients
![Page 30: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/30.jpg)
¡ Right panel ¡ Deal with conflict of interest
¡ Clearly articulated question¡ Gather and summarize the best evidence
¡ Rate quality of evidence
¡ Consideration of values and preferences
¡ Rate strength of clear recommendations
![Page 31: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/31.jpg)
¡ strong recommendation§ benefits clearly outweigh risks/hassle/cost§ risk/hassle/cost clearly outweighs benefit
¡ What can downgrade strength?
¡ Low quality evidence
¡ Close balance between up and downsides
![Page 32: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/32.jpg)
¡ Variability in patient preference§ strong, almost all same choice (> 90%)§ weak, choice varies appreciably
¡ Interaction with patient§ strong, just inform patient§ weak, ensure choice reflects values
¡ Use of decision aid§ strong, don’t bother; weak, use the aid
¡ Quality of care criterion§ strong, consider; weak, don’t consider
![Page 33: Gordon Guyatt Department of Health Research … Guyatt Department of Health Research Methods Evidence and Impact McMaster University ¡Co-chair of the GRADE working group ¡Have written](https://reader031.vdocuments.site/reader031/viewer/2022030503/5ab0042f7f8b9a22118df1b0/html5/thumbnails/33.jpg)
¡ Look to trustworthy guidelines – reject GOBSAT
¡ Panel: experts, methodologists, clinicians, patients
¡ Deal effectively with conflict of interest
¡ Gather and summarize the best evidence§ GRADE
¡ Rate strength of recommendations§ GRADE