Dr Carl Thompson, University of York

Lecture 8 Lecture 8

clinical decision support clinical decision support systemssystems

What are they and how do I What are they and how do I know if they are any good?know if they are any good?

To introduce the major dimensions of To introduce the major dimensions of computerised clinical decision computerised clinical decision support systems (CDSSs)support systems (CDSSs)

Suggested appraisal criteria for CDSS Suggested appraisal criteria for CDSS

A scenarioA scenario

Chief nurse in a PCT. Chief nurse in a PCT. audit results reveal significant audit results reveal significant

variability, exception reports and low variability, exception reports and low satisfaction around guidelines for satisfaction around guidelines for monitoring and managing warfarin monitoring and managing warfarin therapy. therapy.

Whether specialist clinics and CDSS Whether specialist clinics and CDSS might be a better approach?might be a better approach?

Find a paper (Fitzmaurice 2000)Find a paper (Fitzmaurice 2000)

Some assumptions (humans Some assumptions (humans and decisions)and decisions)

Decisions are choices Decisions are choices Types of decisions merit types of Types of decisions merit types of

research evidence research evidence All require the combination of All require the combination of

information and injection of context information and injection of context to make knowledgeto make knowledge

Combining information is difficult and Combining information is difficult and error prone. error prone.

Some assumptions (CDSS)Some assumptions (CDSS)

Designed around the correct kind of Designed around the correct kind of knowledge for the problem facedknowledge for the problem faced

Must take into account natural Must take into account natural variations in patients and so must variations in patients and so must work with individual profiles and data work with individual profiles and data

Must offer tailored advice and Must offer tailored advice and actually inform decision makingactually inform decision making

Kinds of decision supportKinds of decision support

AUDIENCE: Public, professional and AUDIENCE: Public, professional and embeddedembedded

FunctionalistFunctionalist Information managementInformation management Focussing attentionFocussing attention Patient specific consultationsPatient specific consultations Clinical role (Dx, Tx)Clinical role (Dx, Tx)

Architectural approachArchitectural approach

CDSS architectureCDSS architectureDecision models

Quantitative qualitative

Neural networks

Bayesian,

Fuzzy sets

Truth tables, Boolean logic

Decision trees

Expert systems, reasoning models

CDSS quantitative CDSS quantitative

Decision Decision

ModelModel

TruthTruth

++ --

++ TPTP FNFN 100%100%

-- FPFP TNTN 100%100%

ROCROC

hypothyroidismhypothyroidism

T4T4 HypothyroiHypothyroid d

EuthyroidEuthyroid

5 or less5 or less 1818 11

5.1 – 75.1 – 7 77 1717

7.1 – 97.1 – 9 44 3636

9 or 9 or moremore

33 3939

totalstotals 32 32 9393

Decision thresholds…Decision thresholds…

T4T4 HypothyroiHypothyroidd

EuthyroidEuthyroid

5 or less 5 or less 18 A18 A 1 C1 C

>5>5 14 B14 B 92 D92 D

TotalsTotals 3232 9393

T4T4 HypothyroiHypothyroidd

EuthyroidEuthyroid

9 or less 9 or less 2929 5454

>9>9 33 3939

TotalsTotals 3232 9393

T4T4 HypothyroiHypothyroidd

EuthyroidEuthyroid

7 or less 7 or less 2525 1818

>7>7 77 7575

TotalsTotals 3232 9393

Cut pointCut point SensitivitySensitivity SpecificitySpecificity

55 0.560.56 0.990.99

77 0.78 0.78 0.810.81

99 0.910.91 0.420.42

1: T4 of 5 or less 2: T4 of 7 or less

3: T4 of 9 or lessImpact

rocroc

Cut Cut pointpoint

TPTP FPFP

55 0.560.56 0.010.01

77 0.780.78 0.190.19

99 0.910.91 0.580.58

interpretationinterpretation

•.90-1 = excellent (A) •.80-.90 = good (B) •.70-.80 = fair (C) •.60-.70 = poor (D) •.50-.60 = fail (F)

Qualitative approaches Qualitative approaches

Symbolic rule based reasoning (?Symbolic rule based reasoning (?Boolean logic)Boolean logic) Truth tablesTruth tables Flowcharts or algorythmsFlowcharts or algorythms

Expert systemsExpert systems Forward driven reasoningForward driven reasoning Backwards reasoning Backwards reasoning

Truth tablesTruth tablesRulRule e

E1E1 E2E2 E3E3 E4E4……

E10E10 DxDx

11 TT FF TT TT FF VTVT

22......

99

TT

TT

TT

TT

FF

dd

FF

dd

TT

TT

NRNR

SRSR

T = true

F = false

D = don’t care

E1 all RR intervals are regular

E2 All QRS complexes are identical

E3 QRS complexes longer than 120 msec

E4 P waves present

…

E10 PR intervals regular

algorithmicalgorithmicstart

E3

E1

E2

E10E4

VT

T

TTF

F

F

F

T

FT

Knowledge* based systems Knowledge* based systems

*NB may be called expert systems in Older literature

Knowledge Base

Inference Methods Patient

Database

KnowledgeAcquisition

explication Electronic Health Records

Practice guidelinesSecondary or primary

Evidence base

http://www.isabel.org.uk/http://www.isabel.org.uk/private/diagnostic/drsnav.htm#private/diagnostic/drsnav.htm#

How might CDSS help How might CDSS help innovation?innovation?

Stage Stage Barrier Barrier Possible BenefitPossible Benefit

Predispose Predispose Clinicians don’t know Clinicians don’t know about innovationabout innovation

Might help as learning Might help as learning tool tool

(staff unwilling to (staff unwilling to change)change)

ApathyApathy CDSS might attract CDSS might attract clinical clinical interest/generate interest/generate discussiondiscussion

Peer resistancePeer resistance DSS might help in DSS might help in marketing marketing

Patient resistancePatient resistance DSS may promote DSS may promote innovation to patientsinnovation to patients

‘‘we’re too busy’we’re too busy’ DSS enables NP to take DSS enables NP to take on new tasks freeing on new tasks freeing up medical timeup medical time

Conflicting financial Conflicting financial interest interest

Unlikely to helpUnlikely to help

ENABLINGENABLINGStage Stage Barrier Barrier Possible BenefitPossible Benefit

Enable innovation (staff Enable innovation (staff willing system is against willing system is against them)them)

Patient data not Patient data not complete, poorly complete, poorly presented presented

DSS issues problem-DSS issues problem-specific checklist or specific checklist or reminder to record reminder to record relevant data, relevant data, preinterprets complex preinterprets complex patient data, carries out patient data, carries out automatic case findingautomatic case finding

Poor access to detailed Poor access to detailed knowledgeknowledge

DSS acts as intelligent DSS acts as intelligent front-end to literature, front-end to literature, filtering knowledge filtering knowledge according to current according to current patient and problempatient and problem

Clinicians find it hard to Clinicians find it hard to synthesise patient data synthesise patient data and knowledgeand knowledge

DSS carries out complex DSS carries out complex calculation or logical calculation or logical reasoning to link relevant reasoning to link relevant patient data and clinical patient data and clinical knowledgeknowledge

Lack of skills Lack of skills DSS might help as a DSS might help as a learning or simulation learning or simulation tooltool

Lack of space, drugs, Lack of space, drugs, money… etcmoney… etc

Unlikely to help Unlikely to help

reinforcingreinforcingStage Stage Barrier Barrier Possible BenefitPossible Benefit

Reinforce innovation Reinforce innovation (staff need (staff need encouragement) encouragement)

Forgetting Forgetting Reminders for Reminders for clinicians clinicians

Mistakes caused by Mistakes caused by action slips, capture action slips, capture errors errors

Reminders and alerts Reminders and alerts to build a sfae to build a sfae operating operating environment; environment; preinterpreted preinterpreted patient data; patient data; problem-specific problem-specific workflow and record workflow and record formats to lessen formats to lessen errorserrors

Diminished Diminished motivation over time motivation over time

Reminders or alerts; Reminders or alerts; DSS can help support DSS can help support others (e.g. nurse others (e.g. nurse practitioners) to carry practitioners) to carry out routine tasks out routine tasks

Critical appraisal of CDSS - Critical appraisal of CDSS - validityvalidity

Were study participants randomised Were study participants randomised If not, did the investigators demonstrate If not, did the investigators demonstrate

similarity in all known determinants of similarity in all known determinants of prognosis – or adjust for differences in analysis?prognosis – or adjust for differences in analysis?

Was the control group uninfluenced by the Was the control group uninfluenced by the CDSS?CDSS?

Were interventions similar in the two Were interventions similar in the two groups ?groups ?

Was outcome assessed uniformly in the Was outcome assessed uniformly in the experimental and control groups?experimental and control groups?

Critical appraisal of CDSS – Critical appraisal of CDSS – results and applicationresults and application

What is the effect of the CDSS?What is the effect of the CDSS?

What elements of the CDSS are What elements of the CDSS are requiredrequired

Is the CDSS exportable to a new site?Is the CDSS exportable to a new site? Is the CDSS decision support system Is the CDSS decision support system

likely to be accepted in your setting?likely to be accepted in your setting? Do the benefits of the CDSS justify the Do the benefits of the CDSS justify the

risks and costs?risks and costs?