clinical effectiveness

Clinical EffectivenessClinical Effectiveness

Dr. Shahram YazdaniAssociate Professor of SBUMS

Quote

Where is the life we have lost in livingWhere is the life we have lost in living

Where is the wisdom we have lost in knowledgeWhere is the wisdom we have lost in knowledge

Where is the knowledge we have lost in informationWhere is the knowledge we have lost in information

The Rock, TS Elliot

Concept Map

Data Information Knowledge Wisdom

In an Ideal WorldIn an Ideal World

►The most effective care for every The most effective care for every condition would be knowncondition would be known

►Every clinician would know the most Every clinician would know the most effective care for every patienteffective care for every patient

►Every clinician would practice the most Every clinician would practice the most effective care that she/he knowseffective care that she/he knows

In the Real WorldIn the Real World

►Much of what should be known is not Much of what should be known is not knownknown

►Much that is known, is not known by Much that is known, is not known by most cliniciansmost clinicians

►Clinicians often fail to practice what Clinicians often fail to practice what they know to be the most effective they know to be the most effective form of careform of care

Annually 3 million articlesAnnually 3 million articlesPublished in 30,000 journalsPublished in 30,000 journals

Creates a stack 750 meters tallCreates a stack 750 meters tallAfter removing the adsAfter removing the ads

Biomedical Science GrowthBiomedical Science Growth

Doubling time of biomedical scienceDoubling time of biomedical science is about 20 months in 2001is about 20 months in 2001

Biomedical Science GrowthBiomedical Science Growth

>1.5 million web pages are added daily, >1.5 million web pages are added daily,

doubling time of web is <8 monthsdoubling time of web is <8 months

Web GrowthWeb Growth

Traditional PracticeTraditional Practice

Huge, RawFragmentedInformation

ClinicalPractice

AIDS: 16,516AIDS: 16,516Breast cancer: 42,297Breast cancer: 42,297

Highway accidents: 43,458Highway accidents: 43,458Medical errors: 44,000-98,000Medical errors: 44,000-98,000

Annual death rates in USAAnnual death rates in USA

116 million extra visits to a physician per year 76 million additional prescriptions, 17 million emergency department visits, 8 million admissions to hospital, 3 million admissions to long-term care facilities199,000 additional deaths. $76.6 billion The total cost was

Cost of drug-related morbidity and mortality Cost of drug-related morbidity and mortality among outpatients in the United Statesamong outpatients in the United StatesDepression Depression 52% know at least 5 DSM3-R criteria52% know at least 5 DSM3-R criteria

59% know how long to treat59% know how long to treat 10-50% of depressed patients are treated10-50% of depressed patients are treated

60% of CHF patients on effective doses 60% of CHF patients on effective doses of ACE-inhibitorsof ACE-inhibitors3 year delay in widespread use of AZT3 year delay in widespread use of AZT

How Well Do We Care for Our Patients?

Richard Smith: about 15%Richard Smith: about 15%Kerr White: 15-20%Kerr White: 15-20%

Archie Cochrane: less than 10%Archie Cochrane: less than 10%

What Proportion of Healthcare What Proportion of Healthcare is Evidence-Based ?is Evidence-Based ?

Traditional PracticeTraditional Practice


ClinicalPractice

years since graduation

r = -0.54p<0.001

...

...

. ... . .... . ....

....

.....

...knowledgeof current best care

Modern PracticeModern Practice


ProcessedSynthesizedInformation

ClinicalPractice

ClinicalScientist

Evidence-BasedClinician

Health KnowledgeManagementUnits (KMU)

Evidence-BasedHealthcare

Centers

ModernEducation

What Proportion of What Proportion of Healthcare is Evidence-Healthcare is Evidence-

Based ?Based ?Setting Setting Type IType I Type IIType II No Evid.No Evid.

Cancer center (USA)Cancer center (USA) 24%24% 21%21% 55%55%

Tertiary surgical center (USA)Tertiary surgical center (USA) 14%14% 64%64% 22%22%

Primary care centers (Spain)Primary care centers (Spain) 38%38% 4%4% 58%58%

General medicine hospital (UK)General medicine hospital (UK) 53%53% 29%29% 18%18%

General psychiatric ward (UK)General psychiatric ward (UK) 65%65% 35%35%

Anesthesia (Australia)Anesthesia (Australia) 32%32% 65%65% 3%3%

Rese

arc

h

Rele

vant

Evid

en

ce

Articles:Articles:POEM:POEM: Patient Oriented Evidence that Matters Patient Oriented Evidence that MattersDOE:DOE: Disease Oriented Evidence Disease Oriented Evidence

Problems: Problems: Common:Common: conditions encountered at least conditions encountered at least every two weeks every two weeks Uncommon:Uncommon: conditions encountered between conditions encountered between one every two weeks and one every six monthsone every two weeks and one every six months

Slowson and ShaughnessySlowson and Shaughnessy

Concept Map

Examples of Hypothetical DOE and Examples of Hypothetical DOE and POEM studiesPOEM studies

Drug A lowers cholesterol

Drug A lowers cardiovascular mortality

Drug A decreases overall mortality

PSA screening detects prostate cancer most of The time and at an early stage

PSA screening decreasesmortality

PSA screening improvesQuality of life

Tight control of type 1 diabetes mellitus keeps FBS<140mg/dl

Tight control of type 1 Diabetes decrease Microvascular complications

Tight control of type 1 Diabetes decrease mortality And improve quality of life

DOE POEM

Rese

arc

h

Rele

vant

Evid

en

ce

Articles:Articles:POEM:POEM: Patient Oriented Evidence that Matters Patient Oriented Evidence that MattersDOE:DOE: Disease Oriented Evidence Disease Oriented Evidence

Problems: Problems: Common:Common: conditions encountered at least conditions encountered at least every two weeks every two weeks Uncommon:Uncommon: conditions encountered between conditions encountered between one every two weeks and one every six monthsone every two weeks and one every six months

Slowson and ShaughnessySlowson and Shaughnessy

Six month survey of 90 journals, which Six month survey of 90 journals, which identified 8047 articles and only 213 POEM:identified 8047 articles and only 213 POEM:

Over 97% of medical literature DOEOver 97% of medical literature DOEAbout About 2.6%2.6% of medical literature is of medical literature is POEMPOEM

% of relevant published articles % of relevant published articles

Concept Map

Rese

arc

h

Rele

vant

Evid

en

ce

Valid

Evid

en

ce

““Critical appraisalCritical appraisal is not just a fault finding is not just a fault finding

exercise. It is a process of reviewing a paper exercise. It is a process of reviewing a paper

to find information of value”. to find information of value”.

Crombie, 1996Crombie, 1996

Concept Map

0

20

40

60

80

100 Table ofcontentAbstract

Methods

Results

Discussion

Conclusion

Tables

Part of the article paid most attention to:

Low High Validity

Low

H

igh

C

lin

ical

Rel

evan

ce

High qualityrelevant

Validity VS. Clinical Relevance

Rese

arc

h

Rele

vant

Evid

en

ce

Valid

Evid

en

ce

Synth

esize

d

Evid

en

ce

Systematic ReviewSystematic ReviewComprehensive search of the relevant researchComprehensive search of the relevant researchExplicit selection criteriaExplicit selection criteriaCritical appraisal of the primary studiesCritical appraisal of the primary studiesIf quantitative methodology applied: meta-analysisIf quantitative methodology applied: meta-analysis

Systematic Reviews of Interventions:Systematic Reviews of Interventions:Evidence of benefit (positive effect)Evidence of benefit (positive effect)Evidence of harm (negative effect)Evidence of harm (negative effect)Evidence of no effect (no change)Evidence of no effect (no change)No evidence of effect (inadequate evidence)No evidence of effect (inadequate evidence)

Concept Map

Rese

arc

h

Rele

vant

Evid

en

ce

Valid

Evid

en

ce

Synth

esize

d

Evid

en

ce

CP

G

Evidence-Based Practice GuidelinesEvidence-Based Practice GuidelinesCritical analysis of primary evidenceCritical analysis of primary evidenceConsidering local conditionsConsidering local conditionsPromise of consistency and optimal carePromise of consistency and optimal careSource, methodology, accessibilitySource, methodology, accessibility

Concept Map

Rese

arc

h

Rele

vant

Evid

en

ce

Valid

Evid

en

ce

Synth

esize

d

Evid

en

ce

CP

G

ResearchEvidence

ClinicalExpertise

PatientPreferences

MakingClinical

Decisions

Bringing the Evidence at the Point of CareWe need evidence (about the accuracy of We need evidence (about the accuracy of diagnostic tests, the power of prognostic diagnostic tests, the power of prognostic markers, the comparative efficacy and safetymarkers, the comparative efficacy and safety of interventions, etc.)of interventions, etc.)

About 5 questions for every in-patient About 5 questions for every in-patient And 2 question for every 3 out-patients.And 2 question for every 3 out-patients.

Self-reports from 17 Grand RoundsSelf-reports from 17 Grand RoundsMedical studentsMedical students 90 min90 minHouse officersHouse officers 00SHOsSHOs 2020RegistrarsRegistrars 4545ConsultantsConsultants 4545

Median minutes/week spent reading about patientsMedian minutes/week spent reading about patients

The Concept Map

Clinical Practice GuidelineClinical Practice Guideline

►A systematically developed statement to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.

Clinical Practice GuidelineClinical Practice Guideline

►CPGs should define clinical review CPGs should define clinical review criteria, clinical indicators and criteria, clinical indicators and standards to allow those applying standards to allow those applying them to measure performance them to measure performance against the statements they contain.against the statements they contain.

ProtocolsProtocols

The term The term protocolprotocol, although in , although in widespread use, is viewed by many widespread use, is viewed by many clinicians as implying a prescriptive clinicians as implying a prescriptive quality, contrary to the spirit in quality, contrary to the spirit in which CPGs are designed (Scottish which CPGs are designed (Scottish Clinical Resource and Audit Group, Clinical Resource and Audit Group, 1993).1993).

22

FlowchartsFlowcharts

►A A flowchartflowchart is a sequential diagram is a sequential diagram employed to show the stepwise employed to show the stepwise procedures used in performing a procedures used in performing a task, as in an algorithm.task, as in an algorithm.

The Process of CPG The Process of CPG DevelopmentDevelopment

► Stage I. Selection of Topic & Formation of Work Group

► Stage II. Recommendations linked to the evidence

► Stage III. Considering modulating factors► Stage IV. Validity review and pilot testing► Stage V. Reporting► Stage VI. Dissemination► Stage VII. Implementation► Stage VIII. Review

Stage I. Selection of Topic & Stage I. Selection of Topic & Formation of Work GroupFormation of Work Group

►Factors to consider when deciding Factors to consider when deciding priorities for CPG Developmentpriorities for CPG Development1.1. Prevalence of conditionPrevalence of condition

2.2. Established variation in practiceEstablished variation in practice

3.3. Potential to change health outcomesPotential to change health outcomes

4.4. Potential to change cost outcomesPotential to change cost outcomes

5.5. Potential to change ethical, legal or Potential to change ethical, legal or social issuessocial issues

6.6. Cost of developing CPGCost of developing CPG

Stage I. Selection of Topic & Stage I. Selection of Topic & Formation of Work GroupFormation of Work Group

The character of a group relates to its size The character of a group relates to its size as well as its composition. as well as its composition.

The size of work groups in other programs The size of work groups in other programs of CPG development varies from of CPG development varies from fourfour (Royal (Royal College of Physicians) to College of Physicians) to fifteenfifteen (Agency for (Agency for Health Care Policy and Research). Health Care Policy and Research).

Striking a balance between stakeholder Striking a balance between stakeholder interest and efficient working is ultimately a interest and efficient working is ultimately a pragmatic decision. pragmatic decision.

Eight or nine Eight or nine members has been suggested members has been suggested as an effective number (Chassin, 1989; as an effective number (Chassin, 1989; Russell Russell et al, 1993).et al, 1993).

Stage II. Recommendations linked Stage II. Recommendations linked to the evidenceto the evidence

►An early task for guideline An early task for guideline developers is to weigh the soundness developers is to weigh the soundness and relevance of the direct and and relevance of the direct and indirect evidence. indirect evidence.

►This would have been generated by This would have been generated by processes of varying degrees of processes of varying degrees of scientific rigour, and by studies of scientific rigour, and by studies of different design and detail. different design and detail.


The approaches used to develop The approaches used to develop recommendations linked to this recommendations linked to this research evidence will vary according research evidence will vary according to the strength and quality of available to the strength and quality of available studies and may involve one or more of studies and may involve one or more of the following:the following:A.A. Expert opinionExpert opinionB.B. Unsystematic, ungraded literature reviewUnsystematic, ungraded literature reviewC.C. Unsystematic, graded literature reviewUnsystematic, graded literature reviewD.D. Systematic, graded literature reviewSystematic, graded literature reviewE.E. Meta-analysis.Meta-analysis.


►This work may be undertaken by:This work may be undertaken by: ““Analyst teams” (e.g. American College Analyst teams” (e.g. American College

of Physicians), of Physicians), Members of a work group, each taking Members of a work group, each taking

responsibility for a given area (e.g. responsibility for a given area (e.g. Royal College of Physicians)Royal College of Physicians)

Independent consultants conducting Independent consultants conducting systematic overviews or meta-analyses systematic overviews or meta-analyses (such as the Cochrane Centre).(such as the Cochrane Centre).


►Several scales have been devised Several scales have been devised that use preset criteria to rank the that use preset criteria to rank the strength of the evidence, and strength of the evidence, and therefore of the recommendationstherefore of the recommendations

Stage III. Modulating factorsStage III. Modulating factors

The consideration of the relationship of clinical The consideration of the relationship of clinical and non-clinical factors to the evidence-based and non-clinical factors to the evidence-based recommendations may involve the use of:recommendations may involve the use of:A.A. Peer groupsPeer groupsB.B. Consensus conferencesConsensus conferencesC.C. Delphi techniquesDelphi techniquesD.D. A combination of these.A combination of these.

Where the research evidence is strong, Where the research evidence is strong, consensus is more easily establishedconsensus is more easily established

It is inevitable that differences of opinion in It is inevitable that differences of opinion in interpreting the evidence will sometimes arise.interpreting the evidence will sometimes arise.

Stage IV. Validity review and pilot Stage IV. Validity review and pilot testingtesting

A CPG should specify the methods used in its A CPG should specify the methods used in its construction, including who was involved and construction, including who was involved and the weightings of the evidence upon which the weightings of the evidence upon which the recommendations are based. the recommendations are based.

An external peer review of the methodology, An external peer review of the methodology, as well as the content, of a CPG is desirable.as well as the content, of a CPG is desirable.

An appropriate pilot study would be required An appropriate pilot study would be required to establish the effectiveness and to establish the effectiveness and acceptability of a CPG. acceptability of a CPG.

Although a randomized controlled trial is the Although a randomized controlled trial is the ideal test of a CPG, time constraints may not ideal test of a CPG, time constraints may not always permit this.always permit this.

Stage V. ReportingStage V. Reporting

►The final product may have a range The final product may have a range of formats, for various target of formats, for various target audiences.audiences.

►These may include as patient These may include as patient information sheets, clinical information sheets, clinical algorithms (decision trees), audit algorithms (decision trees), audit tools, background texts, clinical tools, background texts, clinical ‘reminders’, and structured note ‘reminders’, and structured note formats.formats.

Stage VI. DisseminationStage VI. Dissemination

The distinction between implementation The distinction between implementation and dissemination strategies is often and dissemination strategies is often arbitrary. arbitrary.

The purpose of dissemination is to ensure The purpose of dissemination is to ensure that those who have an interest in the that those who have an interest in the CPG are aware of it, and understand it. CPG are aware of it, and understand it.

Dissemination can include the use of Dissemination can include the use of mass media, peer review journal mass media, peer review journal publication, targeted mailing, and publication, targeted mailing, and promotion by respected opinion leaders.promotion by respected opinion leaders.

Stage VII. ImplementationStage VII. Implementation

Although the extent to which a guideline is Although the extent to which a guideline is implemented is the only true measure of its implemented is the only true measure of its success, surprisingly little is understood about success, surprisingly little is understood about what enhances or inhibits implementation. what enhances or inhibits implementation.

Factors which may help include early and Factors which may help include early and thorough consultation (to foster ownership and thorough consultation (to foster ownership and increase the relevance of a CPG to clinical increase the relevance of a CPG to clinical reality), planned educational strategies and reality), planned educational strategies and clinical reminders, both outside and within the clinical reminders, both outside and within the consultation. consultation.

Potential obstacles to implementation include Potential obstacles to implementation include concerns about the implications of CPGs, concerns about the implications of CPGs, doubts over their relevance or feasibility, and doubts over their relevance or feasibility, and inadequate dissemination.inadequate dissemination.

Stage VIII. ReviewStage VIII. Review

Mechanisms for prompt feedback assist in Mechanisms for prompt feedback assist in the detection of inconsistencies in CPGs. the detection of inconsistencies in CPGs. To facilitate this process, CPGs should To facilitate this process, CPGs should specify:specify:I.I. The date of issueThe date of issue

II.II. The most recent published (or unpublished) The most recent published (or unpublished) evidence considered in formulating the evidence considered in formulating the recommendationsrecommendations

III.III. Relevant trials in progress, where findings Relevant trials in progress, where findings may effect the CPG contentmay effect the CPG content

IV.IV. A review or “sell by” date.A review or “sell by” date.

Importance of Implementation StrategyImportance of Implementation Strategy

► Field and Lohr make the important point that Field and Lohr make the important point that ‘guidelines do not implement themselves’‘guidelines do not implement themselves’ (1992). (1992).

► If guidelines are to be effective, their dissemination and If guidelines are to be effective, their dissemination and implementation must be vigorously pursued. implementation must be vigorously pursued.

► If not, the time, energy and cost devoted to the If not, the time, energy and cost devoted to the guidelines’ development will be wasted and potential guidelines’ development will be wasted and potential improvements in consumer health will be lost.improvements in consumer health will be lost.

Distributing Guidelines: No Distributing Guidelines: No EffectEffect

38

Implementation PanelImplementation Panel

► A multidisciplinary panel should oversee the various A multidisciplinary panel should oversee the various steps needed to disseminate and implement the steps needed to disseminate and implement the guidelines. guidelines.

► The panel, which may be the same as the panel The panel, which may be the same as the panel responsible for developing the guidelines, should also responsible for developing the guidelines, should also identify any barriers to the guidelines’ acceptance and identify any barriers to the guidelines’ acceptance and implementation and work with members of target implementation and work with members of target groups to develop ways of overcoming these barriers. groups to develop ways of overcoming these barriers.

39

Barriers to ChangeBarriers to Change

►Identifying barriers to change requires an Identifying barriers to change requires an understanding of understanding of sociologicalsociological and and psychologicalpsychological factors: it is essential that the guideline factors: it is essential that the guideline development panel has expertise in these areas; development panel has expertise in these areas; otherwise, inappropriate or ineffective methods otherwise, inappropriate or ineffective methods of dissemination and implementation may be of dissemination and implementation may be advocated.advocated.

40

CME and ChangeCME and Change

►Many studies have examined strategies for Many studies have examined strategies for continuing medical education continuing medical education (Davis et al. 1995)(Davis et al. 1995) and there is a considerable body of evidence on and there is a considerable body of evidence on which to draw. which to draw.

►The most striking finding is that the simple The most striking finding is that the simple dissemination of guidelines is likely to have no dissemination of guidelines is likely to have no impact at all on implementation impact at all on implementation (Oxman et al. (Oxman et al. 1995; Wise & Billi 1995).1995; Wise & Billi 1995).

41

Change InterventionChange Intervention

►Change will occur only if specific interventions Change will occur only if specific interventions designed to encourage it are used.designed to encourage it are used.

►The interventions most likely to induce change The interventions most likely to induce change are those that require the clinicians’ participation are those that require the clinicians’ participation in the change process (Wise & Billi 1995).in the change process (Wise & Billi 1995).

1. As Booklets2. In professional journals;3. In professional associations’ newsletters and magazines;4. In trade publications and industry newspapers;5. In the popular media;6. As brochures7. On the Internet and linked to websites appropriate for the target audience;8. As audio or video tapes; 9. On computer disks.

Awareness PreparationPractice Change

Reinforcement

Publishing the Guidelines

1. Posting out guidelines2. Using national, regional and local media;3. Publicity in trade publications and possibly writing articles for them;4. Publicity through professional associations and their publications 5. Publicity in professional journals;6. Publicity through consumer groups and their publications;7. Contact with undergraduate and postgraduate educators;


Reinforcement


Informing the target audience

8. Contact with undergraduate and postgraduate students;9. Publicity through institutions such as colleges, hospitals, 10. Discussion at conferences, seminars and professional meetings;11. Using ‘champions’ or local authorities to promote the guidelines or to be interviewed 12. Identifying ‘human interest’ stories for guidelines.


Reinforcement



1. Including in Undergraduate Medical Education2. Continuous Medical Education3. Educational Materials4. Seminars and Conferences5. Web Based Materials6. Interactive Educational Meetings


Reinforcement



Education

1. Including only technically efficient drugs for each problem in “national pharmacopoeia”2. “Insurance pharmacopoeia” according to allocative efficiency of interventions3. Considering “Pharmacopoeia in use” through sophisticated drug logistic strategies


Reinforcement



EducationAvailability AccessibilityAffordability

1. Perfect Practice Prize2. Naming 5 Star GPs in Professional Media3. Payment Bonuses4. Incentives for organizations within them CPGs are adopted and implemented5. Incentives for Provinces within them CPGs are mostly Implemented


Reinforcement




Incentive Strategies


Reinforcement



1. Setting Regulatory Clinical Standards2. Mandatory Registration of Patients with Disease of Interest in

Registration Books3. Performance Monitoring4. Clinical Audit5. Feedback Messages (according to audit results)6. Practice Reminders (eg on report of laboratory or radiology orders)



RegulatoryActivities


Reinforcement



7. Prescription Feedbacks8. Re-evaluation and Re-certification9. Contracts



RegulatoryActivities

Audit and FeedbackAudit and Feedback

Duration of EffectDuration of Effect

Thank You!Thank You!Any Question?Any Question?

[email protected][email protected]

clinical effectiveness

Documents

clinical indicators

clinical effectivenessdr

effective care

clinical review criteria

scottish clinical resource

primary care centers

appropriate health care

validity review