Elaboración de recomendaciones en las GPC

Sistema GRADE

Nicola MagriniNHS CeVEAS, Centre for the Evaluation of the Effectiveness of

Health Care, Modena, ItalyWHO Collaborating Centre for Evidence Based Research Synthesis

and Guideline Development

Contents of the presentation

• What are the defects of existing guidelines and systems of grading

• Why GRADE could help …• A three pillar method: the GRADE system to

evaluate quality of evidence and define the strength of a recommendation

• Three examples• Conclusions

Contents of the presentation

• What are the defects of existing guidelines and systems of grading

• Why GRADE could help …• A three pillar method: the GRADE system to

evaluate quality of evidence and define the strength of a recommendation

• Three examples• Conclusions

http://clinicalevidence.bmj.com/ceweb/about/knowledge.jsp 2011

Tendency of recent guidelines

Trends in guideline production(AHA guidelines, Tricoci JAMA 2009)

• Recommendations are increasing in size with every update (+48% form first version)

• Quality of evidence: only a minority of recommendations are based on good evidence (11%) and half (48%) on low quality evidence

• Recommendations with high quality evidence are mostly concentrated in class I (strong recommendation) but only 245 of 1305 class I recommendations have high quality evidence (median, 19%)

Guidelines reassessment …• … in ACC/AHA guidelines with at least 1 revision, the number of

recommendations increased 48% from the first guideline to the most recent version. If there is a main message in such guidelines, it is likely to be lost in the minutiae.

• Within a guideline document, individual recommendations also need to be prioritized.

• Finally, guidelines need flexibility. Recommendations should vary based on patient comorbidities, the health care setting, and patient values and preferences.

• Physicians would be better off making clinical decisions based on valid primary data.

Shaneyfelt TM, Centor RM.Reassessment of clinical practice guidelines

JAMA 2009

How to improve guideline quality

Present limitations:• Governance and composition of the guideline

committee (“what is to be decided is often already decided with the selection of the deciders”)

• Unanimity in guideline (not a natural component in research)

• Lack of independent review (outside the accepted procedures of scientific publications)

• Suboptimal management of Conflicts of interests

Sniderman AD, Furberg CD.Why guidelines making requires reform

JAMA 2009

Too many grading systems?Who is confused?

EvidenceRecommendation

B Class IC+ 1IV C

OrganizationAHAACCPSIGN

Recommendation for use of oral anticoagulation in patients with atrial fibrillation and rheumatic mitral valve disease

Contents of the presentation

• What are the defects of existing guidelines and systems of grading

• Why GRADE could help …• A three pillar method: the GRADE system to

evaluate quality of evidence and define the strength of a recommendation

• Three examples• Conclusions

Why using GRADE

GRADE is much more than a rating system • offers a transparent and structured process for

developing and presenting summaries of quality of evidence

• provides guideline developers with a comprehensive and transparent framework for carrying out the steps involved in developing recommendations

• specifies an approach to framing questions, choosing outcomes of interest and rating their importance, evaluating the evidence, and incorporating evidence with considerations of values and preferences of patients and society to arrive at recommendations

WHO guideline development processes

update 2010

1. Scoping the document: reasons for choosing the topic, problems with existing guidelines, variations and gaps,

2. Group composition (or consultations)

3. Conflict of interest

4. Formulations of the questions and choice of the relevant outcomes

5. Evidence retrieval, evaluation and synthesis (balance sheet, evidence

table)

6. Benefit/risk profile: integrating evidence with values and preferences,

equity and costs

7. Formulation of the recommendations

8. Implementation and evaluation of impact

9. Research needs or areas of further research

10. Peer-review process and updating

Title, responsible person, WHO Department - responsible of the clearance process, WHO Departments involved, CC

involved,

Standards for evidence: GRADE system

Reporting standard and process

Reporting standard and process

GRADE Working Group websiteand publications

www.gradeworkinggroup.org

Contents of the presentation

• What are the defects of existing guidelines and systems of grading

• Why GRADE could help …• A three pillar method: the GRADE system to

evaluate quality of evidence and define the strength of a recommendation

• Three examples• Conclusions

GRADE: a 3 pillars approach

1. Formulate the question, choose and rate your outcomes of interest and perform a systematic review (quality of evidence)

2. Risk benefit evaluation, consider patients values and preferences and also resource use and feasibility

3. Direction (positive/negative) and strength (strong/weak) of the recommendation

GRADE: a 3 pillars approach

1. Formulate the question, choose and rate the outcomes of interest and perform a systematic review (quality of evidence)

2. Risk benefit evaluation, consider patients values and preferences and also resource use and feasibility

3. Strength of the recommendation

Figure 1: Hierarchy of outcomes according to their patient-importance to assess the effect of enteral supplement nutrition for geriatric patientswith bed sores

Nutritional status 4

Importanceof endpoints

Microcirculationof the wound 1

2

Energy supply 3

5

Function 6

Quality of life 7

Healing of the 8bedsore

Mortality 9

Criticalfor decision making

Important, but not critical fordecision making

Not patient-important

Rating of outcomes

… example: patient with bed sores

WHO Recommendations for the Prevention of PPH, 2007

WHO Recommendations for the Prevention of PPH, 2007

Study design is important

Early systems of grading the quality of evidence focused almost exclusively on study design

Randomised trials provide, in general, stronger evidence than observational studies:–RCTs start at High Quality–Observational studies start at Low Quality

However, other factors may decrease or increase the quality of evidence

Quality assessment criteria: the big start

Factors that may decrease the quality of evidence

Study limitations (risk of bias) well established

– concealment– intention to treat principle observed– blinding– completeness of follow-up– Choice of comparator (standard/optimal

treatment)

more recent– early stopping for benefit– selective outcome reporting bias

Factors that may decrease the quality of evidence

Study limitations (risk of bias)Inconsistency among studiesIndirectness of evidence Imprecise results Reporting bias

Evidence synthesis (systematic review)

PICO

OutcomeOutcomeOutcomeOutcome

Formulate

question

Rate

importa

nce

Critical

Important

Critical

Not important

Create

evidence

profile with

GRADEpro

Summary of findings & estimate of effect for each outcome

Rate overall quality of

evidence across outcomes based on

lowest quality of critical outcomes

RCT start high, obs. data start

low

1. Risk of bias2. Inconsistency3. Indirectness4. Imprecision5. Publication

bias

Gra

de

dow

nG

rade

up

1. Large effect

2. Dose response

3. Confounders

Rate quality

of evidence

for each

outcomeSelect

outcomes

Very low

Low

ModerateHigh

Outcomes

across

studies

GRADE: a 3 pillars approach

1. Formulate the question, choose and rate your outcomes of interest and perform a systematic review (quality of evidence)

2. Risk benefit evaluation, consider patients values and preferences and also resource use and feasibility

3. Strength of the recommendation

Determining the benefit risk profile: positive/uncertain/unfavourable

Factors Impact on the strength of a recommendation

Balance between desirable and undesirable effects

Larger the difference between the desirable and undesirable effects, more likely a favourable benefit But differences can arise depending on the severuty of adverse events

Values and preferences

More variability in values and preferences, or more uncertainty in values and preferences, more likely an unfavourable profile.

Costs (resource use)

Higher the costs of an intervention – that is, the more resources consumed – less likely a favourable profile.

GRADE Step 2: risk benefit profile, values and preferences (1/3)

GRADE Step 2: risk benefit profile, values and preferences (2/3)

GRADE Step 2: risk benefit profile, values and preferences (3/3)

GRADE: a 3 pillars approach

1. Formulate the question, choose and rate your outcomes of interest and perform a systematic review (quality of evidence)

2. Risk benefit evaluation, consider patients values and preferences and also resource use and feasibility

3. Strength of the recommendation

Strength of recommendationThe degree of confidence that the

desirable effects of adherence to a recommendation outweigh the undesirable effects.

Desirable effects• health benefits• less burden• savings

Undesirable effects• harms• more burden• costs

Categories of recommendations

Although the degree of confidence is a continuum, we suggest using two categories: strong and weak.

Strong recommendation: the panel is confident that the desirable effects of adherence to a recommendation outweigh the undesirable effects.

Weak recommendation: the panel concludes that the desirable effects of adherence to a recommendation probably outweigh the undesirable effects, but is not confident.

Recommend

Suggest

Why Grade Recommendations?

Strong recommendations– strong methods – large precise effect – few down sides of therapy

Weak recommendations– weak methods– imprecise estimate– small effect– substantial down sides

Evidence synthesis (systematic review)

Making recommendations (guidelines)

PICO

OutcomeOutcomeOutcomeOutcome

Formulate

question

Rate

importa

nce

Critical

Important

Critical

Not important

Create

evidence

profile with

GRADEpro

Summary of findings & estimate of effect for each outcome

Rate overall quality of

evidence across outcomes based

on lowest quality of critical outcomes

Panel

RCT start high, obs. data start

low

1. Risk of bias2. Inconsistency3. Indirectness4. Imprecision5. Publication

bias

Gra

de

dow

nG

rade

up

1. Large effect

2. Dose response

3. Confounders

Rate quality

of evidence

for each

outcomeSelect

outcomes

Very low

Low

ModerateHigh

Formulate recommendations:• For or against (direction)• Strong or weak (strength)

By considering:Quality of evidenceBalance benefits/harms

Values and preferences

Revise if necessary by considering: Resource use (cost)

• “We recommend using…”• “We suggest using…”• “We recommend against

using…”• “We suggest against using…”

Outcomes

across

studies

Contents of the presentation

• What are the defects of existing guidelines and systems of grading

• Why GRADE could help …• A three pillar method: the GRADE system to

evaluate quality of evidence and define the strength of a recommendation

• Three examples• Conclusions

Recommendations using GRADE: Example 1

A flexible method:

quality of evidence independent from strength of recommendation

WHO avian flu guideline 2006

Schünemann HJ et al. Lancet Infect Dis 2007;7:21-31

For opioid agonist maintenance treatment, most patients should be advised to use methadone in adequate doses in preference to buprenorphine. – Strength of recommendation – Strong– Quality of evidence – High

WHO Guidelines for the Psychosocially Assisted Pharmacological Treatment of

Opioid Dependence (2009)

On average, methadone maintenance doses should be in the range of 60–120 mg per day. – Strength of recommendation – Strong– Quality of evidence – Low

Recommendations using GRADE: Example 2

Taking into account values and preferences … and local context

Values and preferences

Stroke guideline: patients with TIA clopidogrel over aspirin (Grade 2B).

Underlying values and preferences: This recommendation to use clopidogrel over aspirin places a relatively high value on a small absolute risk reduction in stroke rates, and a relatively low value on minimizing drug expenditures.

Values and preferences

peripheral vascular disease: aspirin be used instead of clopidogrel (Grade 2A).

Underlying values and preferences: This recommendation places a relatively high value on avoiding large expenditures to achieve small reductions in vascular events.

Recommendations using GRADE: Example 3

Weak recommendations …

a blurred vision or a clear one?

Recommendations and expected adoption rate

StrengthDefinition and implications

Expected adoption

rate

Strong positive

The drugs/interventions should offered to the vast majority of patients and could be used as an indicator of good quality of care > 60-70%

Weak positive

It has the wider range of uncertainty since it could mean only for a minority of patients (30%) or for a good proportion of them (50-60%). It is necessary to inform patients of the expected benefits and risks (and their magnitude), explore patients values and discuss potential alternative treatments

30-60%

Weak negative

In selected cases or a defined minority. The decision should go along with a detailed information to patient of the benefit risk profile (magnitude), patients values and expectations and the presentation of potential alternative treatments

5-30%

Strong negative

It should not be used neither routinely nor for a subgroup. Only in few very selected (and documented) cases can be used since the benefit/risk balance is negative/unknown and available alternative are preferable

< 5%

Contents of the presentation

• What are the defects of existing guidelines and systems of grading

• Why GRADE could help …• A three pillar method: the GRADE system to

evaluate quality of evidence and define the strength of a recommendation

• Three examples• Conclusions

GRADE … in short

• Have an overall view of the process (see WHO), a good-enough mandate and some governance of relevant CoI

• Make just a few (a reasonable number of) recommendations

• Use systematic reviews (if not available, review key, accessible evidence) – DO NOT meta-analyse if not done

• Use GRADE criteria for quality of evidence• Explain the reasons supporting the strength of

recommendations, including the benefit/risk profile and values and preferences

• … just be (more) transparent

Where am I?

You’re 30 metres

above the ground in a

balloon

You must be a

researcher

Yes. How

did you know?

Because what you told me is

absolutely correct but completely

useless

You must be a policy

maker

Yes, how did

you know? Because you

don’t know where you are, you

don’t know where you’re going, and

now you’re blaming me

from: Jonathan Lomas, 2008