six sigma for clinicians what does it really mean?

SIX SIGMA FOR CLINICIANSWhat does it really mean?

SIX SIGMA IN HEALTHCARENew Orleans, Louisiana

March 3-4, 2005

Larry V. Staker MD, FACP

CMO Deseret Mutual

Salt Lake City, Utah

“If to do were as easy as to knowwhat were good to do, then chapelshad been churches and poor men’scottages, princes palaces.”

Merchant of VeniceWilliam Shakespeare

© Larry V. Staker MD

Clinicians Six Sigmaor

Clinical Practice Improvement

PEDAGOGY

(are there better ways to teach)

SIXSIGMA

EBM

STANDARDS

MEASUREMENT

OUTCOMES

ACCOUNTABILITY

$ENSE

1

2

3

4

6

5

1

METHODS

Start with Evidence Based Medicinefor

“CLINICAL PRACTICE IMPROVEMENT”

Evidence Based Medicine

SUMSwith disease without disease

test pos a b a+btest neg c d c+d

a+c b+d a+b+c+d

NUMBERS2 x 2 TABLE

SUMSwith disease without disease

test pos a b a+btest neg c d c+d

a+c b+d a+b+c+d

NUMBERS2 x 2 TABLE

UNDERSTANDING2 x 2 TABLES

SUM RATIOa b a+bc d c+d

a+c b+d T=(a+b)+(c+d)a/(a+c) d/(b+d)c/(a+b) b/(c+d)LR (+) LR (-) Likelihood

sens/(1-spec) (1-sens)/spec Ratios

(a+c)/T

NUMBER PERCENT / RATEa/(a+b)d/(c+d)

IMPROVING DIAGNOSTIC SKILLS The Basic Tool is a 2 x 2 Table

AImproving Diagnostic Skills

A TOOL FOR DETERMINING THE USEFULNESS OF A DIAGNOSTIC TEST (enter numbers in white boxes) © Larry V. Staker MD

SUMS RATIOSwith disease without disease

test pos 475.0 50.0 525 90.5% PPV 90.5%test neg 25.0 450.0 475 94.7% NPV 5.3%

500 500 1000 50.0% PTPSnOUT SENS

True (+) rate 95.0% 10.0% False (+) rate

False (-) rate 5.0% 90.0% True (-) rateSPEC SpIN

LR (+) LR (-)9.50 0.06

sens/(1-spec) (1-sens)/spec

NUMBERS PERCENTS / RATES / PROPORTIONS

Likelihood Ratios

Post Test Probability given positive result

(1/NPV) = Post Test Probability given negative result

Pre Test Probability of disease estimated from Hx / PE

THE BASIC TOOL FOR IMPROVING DIAGNOSTIC SKILLS

POST TEST PROBABILITY (pos)POST TEST PROBABILITY (neg)

PRE TEST PROBABILITY

LIKELIHOODRATIOS

A SOURCE OF INFORMATIONTO IMPROVE SKILLS

OF DIAGNOSIS

ISBN = 9-943126-74-6

DIAGNOSTIC STRATEGIESFor Common Medical Problems

Second Edition

Edited By: Edgar R Black MDPublisher: American College of Physicians

TEST SEN SPECETT and ST Seg ?

0.5 - 0.99 86.0% 77.0%1.0 - 1.49 65.0% 89.0%

1.50 - 1.99 42.0% 98.0%2.0 - 2.49 33.0% 99.0%

>= 2.5 20.0% 99.5%ETT and

Thallium 88.0% 91.0%SPECT 90.0% 72.0%

Adenosine 89.0% 83.0%Dipyridamole-o 87.0% 75.0%Dipyridamole-iv 90.0% 78.0%

Dobutamine 91.0% 86.0%Stress ECHO 81.0% 89.0%Dobutamine ECHO 81.0% 83.0%

THE OUTPUT OF THE TOOLpost-test probabilities and likelihood ratios

PreTOR PostTLR © Larry V. Staker MDDisease Pos Test

PreTProb Pre:1 Post:1 PTP Disease - Pos Test

75.0% 7.5 2.5 3.00 28.50 96.6% 21.6% CI 95 ( 95.5% 97.7% )10.0% 1 9 0.11 1.06 51.4% 41.4% CI 95 ( 48.3% 54.4% )20.0% 2 8 0.25 2.38 70.4% 50.4% CI 95 ( 67.5% 73.2% )

30.0% 3 7 0.43 4.07 80.3% 50.3% CI 95 ( 77.8% 82.7% )40.0% 4 6 0.67 6.33 86.4% 46.4% CI 95 ( 84.2% 88.5% ) PostTLR50.0% 5 5 1.00 9.50 90.5% 40.5% CI 95 ( 88.7% 92.3% ) >2560.0% 6 4 1.50 14.25 93.4% 33.4% CI 95 ( 91.9% 95.0% )70.0% 7 3 2.33 22.17 95.7% 25.7% CI 95 ( 94.4% 96.9% )

80.0% 8 2 4.00 38.00 97.4% 17.4% CI 95 ( 96.5% 98.4% )90.0% 9 1 9.00 85.50 98.8% 8.8% CI 95 ( 98.2% 99.5% )

PreTOR PostTLR © Larry V. Staker MDDisease Neg Test

PreTProb Pre:1 Post:1 PTP Disease - Neg Test

75.0% 7.5 2.5 3.00 0.17 14.3% 60.7% CI 95 ( 12.1% 16.5% )10.0% 1 9 0.11 0.01 0.6% 9.4% CI 95 ( 0.1% 1.1% )20.0% 2 8 0.25 0.01 1.4% 18.6% CI 95 ( 0.6% 2.1% )

30.0% 3 7 0.43 0.02 2.3% 27.7% CI 95 ( 1.4% 3.3% )40.0% 4 6 0.67 0.04 3.6% 36.4% CI 95 ( 2.4% 4.7% ) PostTLR50.0% 5 5 1.00 0.06 5.3% 44.7% CI 95 ( 3.9% 6.6% ) <0.2560.0% 6 4 1.50 0.08 7.7% 52.3% CI 95 ( 6.0% 9.3% )70.0% 7 3 2.33 0.13 11.5% 58.5% CI 95 ( 9.5% 13.5% )

80.0% 8 2 4.00 0.22 18.2% 61.8% CI 95 ( 15.8% 20.6% )90.0% 9 1 9.00 0.50 33.3% 56.7% CI 95 ( 30.4% 36.3% )

Test

Treat

Treat

PreTOdds

PreTOdds

BENEFIT OF NEGATIVE TEST

Observe

BENEFIT OF POSITIVE TEST

Observe

Test

CONFIDENCE INTERVALSLR rule: +>25; -<0.25

IMPROVING TREATMENT SKILLS The Basic Tool is a 2 x 2 Table

BChoosing Best Treatment

Enter numbers in white boxes © Larry V. Staker MD

SUMS MULT DIFF RECP RELATIONSHIPSTTG NTTG PROPORTIONS

experiment 140 20 160 87.5% ODDS

control 35 125 160 21.9% ODDS RATIOS

SUM 320

PERCENT 400.0% 0.0%MULTIPLICATION 17500 a*d c*b 700

DIFFERENCE 65.6% |EER-CER| ABI1.5 1/ABI NNT

RATIO 25 (a*d)/(c*b) ROR

BASIC TOOL FOR EVALUATING EFFECTIVENESS OF TREATMENT

NUMBERS

a/(a+b) TTG or EER

PERCENTS / RATES / PROPORTIONS

c/(c+d) NTTG or CER

RECIPROCAL

RB = EER/CER |ABI/CER| RBI

RBI, ABI, and NNT

THE OUTPUT OF THE TOOLevaluation standards of peer review journals

FormulasSE Ln Ln SE

p1=a/(a+b) 0.035 EER 50.0% 43.1% 56.9% EER Experiment Event Ratep2=c/(c+d) 0.029 CER 15.6% 10.0% 21.3% CER Control Event RateI EER-CER I 0.036 ARR 34.4% 27.3% 41.5% ARR Absolute Risk Reduction1/ARR NNT 2.9 2.4 3.7 NNT Number Needed to TreatEER/CER RR 320.0% 217.6% 470.7% RR Relative Risk

0.197 1.1632 CI95 LnRR 0.777 1.549 CI95 LnRR Natural Log RBI EER-CER I /CER RRR 220.0% 117.6% 370.7% RRR Relative Risk Reduction(a*d)/(c*b) ROR 5.4 3.2 9.0 ROR Relative Odds Ratio

0.260 1.686 CI95 LnOR 1.178 2.195 CI95 LnOR Natural Log OR

CI-95Standard Error Calculation Enumerative Statistical Analysis

RBI, ABI, and NNT or

RRR, ARR, and NNT

CONFIDENCEINTERVALS

PDSA

HEARING, SEEING and MEASURING

The Voice Of The Process

VOP

PDSA

STUDY

PLAN

DO

ACT

PROCESSIMPROVEMENT

PDSA: Minimize Variation

– There will always be some variation in a process

– But we can work to minimize variation around a mean or target

40

45

50

55

60

65

70

1 11 21 31 41 51 61 71 81

GOAL= reduce variation

Mean

UCL

LCL

RAPID CYCLE TESTING

Discovery Learning

P

D

S

A

P

D

S

A

P

D

S

A

P

D

S

A

Thomas W. Nolan Ph.D.

THE GAME OF IMPROVEMENTThe Work or

Process Base

The Measurement of Population Base

The Benchmark orEvidence Base

The Result or Outcome Base

© Larry V. Staker MD

TTG

1

2 4

3

5

SIX SIGMA

HEARING, SEEING, and MEASURING

Voice Of The CustomerDefects Reduction Error Free Yield

VOCSigma Metric

SIX SIGMA

1a

1b

1c2

3

4

5

PROCESSIMPROVEMENT

DEFINECORE PROCESSES

DEFINEKEY

CUSTOMERS

DEFINECUSTOMER

REQUIREMENTS

MEASURECURRENT

PERFORMANCE

ANALYZE

IMPROVE

CONTROLINTEGRATE

EXPANDP

D

C

A

SIX SIGMA: Customer ServiceThe process shown here is stable.

But why does it need to be improved?

} CustomerNeed

Time

LSL

USL

UCL

LCL

SIX SIGMA and PDSA

STUDY

PLAN

DO

ACT

1a

1b

1c2

3

4

5

PROCESSIMPROVEMENT

IDENTIFYCORE PROCESSES

IDENTIFYKEY

CUSTOMERS

DEFINECUSTOMER

REQUIREMENTS

MEASURECURRENT

PERFORMANCE

ANALYZE

IMPROVE

CONTROLINTEGRATE

EXPAND

Although we may do a good job of teaching the best medical practice or treatment available today, we do a poor job of teaching ourselves how to decide when what we learnedin the past is no longer good enoughand needs to be changed.

2

STANDARDS

NATIONALLY RECOGNIZED

“Evidence-Based”

CLINICAL STANDARDS

ATP III - Standards

STANDARDS FORDIABETES CARE

1. HbA1c <= 7.0

2. LDLC <= 100

3. BP <= 135/85

4. Eye Exam Every year

5. Foot Exam Every visit

6. Microalbumin / Creat Once a year

USING STANDARDS

1. Find an acceptable evidence-based Standard

2. Hold yourself to that Standard

3. Measure performance by that Standard

4. Evaluate and grade against the Standard (TTG)

5. Make $ense using the Standard

6. Negotiate Pay for Performance (P4P) from TTG

3

MEASUREMENT

Start by teaching the use of simpleMeasurement Tools

WHAT AND HOW WE MEASURE

SATISFACTION

COST QUALITY

PHYSICIANPERFORMANCE

WORK

WHAT AND HOW WE MEASURE

PHYSICIANPERFORMANCE

COST QUALITY

SATISFACTION

WORK

WHAT AND HOW TO MEASURE

OUTCOMESCOST QUALITY

SATISFACTION

© Larry V. Staker MD

UNDERSTANDING VARIATION

SPECIAL CAUSE VARIATION SPECIAL CAUSE VARIATION

COMMON CAUSE VARIATION

© Larry V. Staker MD

HOW WE MEASURETime Sequence Data Display

TIME

KPVMEDIAN

SPECIAL CAUSE VARIATION

SPECIAL CAUSE VARIATION

COMMON CAUSE VARIATION

UCL

LCL

COMMON CAUSE VARIATION

Intraoccular Traumatic Test

Rapid Feedback of Information

Time Ordered Sequence

LINE AND SPEC CHART

LEARNING TO USE

LINE CHART

KPV PLOTTED IN TIME ORDERED SEQUENCE

60

80

100

120

140

160

180

200

60

80

100

120

140

160

180

200

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

81 98 110 99 110108120 96 90 91 85 108 99 92 107102 83 92 125 98 102102109 95 116102106 98 80 130113

FB

S

dayfbs

SPECIFICATION CHARTPATIENT

60

80

100

120

140

160

180

200

60

80

100

120

140

160

180

200

1 2 3 4 5 6 7 8 9 10111213141516171819202122232425262728293031

FB

S

TREATMENT GOAL (TG)

UPPER SPECIFICATION LIMIT (USL)

LOWER SPECIFICATION LIMIT (LSL)

day

TREATMENT TO GOAL

SUPERIMPOSE

SPECIFICATION CHARTS ON LINE RUN OR CONTROL CHARTS

and use

THE INTEROCULAR TRAUMATIC TEST

( ITT )

Joseph Berkson MD, PhD. Mayo Clinic

© Larry V. Staker MD

LINE AND SPEC CHART

KPV PLOTTED IN TIME ORDERED SEQUENCE

60

80

100

120

140

160

180

200

60

80

100

120

140

160

180

200

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

81 98 110 99 110108120 96 90 91 85 108 99 92 107102 83 92 125 98 102102109 95 116102106 98 80 130113

FB

S

TREATMENT GOAL

USL

LSL

dayfbs

SPECIFICATION CHART Diabetes Mellitus

PATTERN: INTERVENTION:

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31

60

80

100

120

140

160

180

200

3

4

5

6

7

8

9

10

11

DAY OF MONTH

FG (mg/dl) FG (mmol/l)

USL

CL

LSL

SPECIFICATION CHART Diabetes Mellitus

PATTERN: INTERVENTION:

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31

60

80

100

120

140

160

180

200

3

4

5

6

7

8

9

10

11

DAY OF MONTH

FG (mg/dl) FG (mmol/l)

USL

CL

LSL

SPECIFICATION CHART Diabetes Mellitus

PATTERN: INTERVENTION:

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31

60

80

100

120

140

160

180

200

3

4

5

6

7

8

9

10

11

DAY OF MONTH

FG (mg/dl) FG (mmol/l)

USL

CL

LSL

SPECIFICATION CHART Diabetes Mellitus

PATTERN: INTERVENTION:

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31

60

80

100

120

140

160

180

200

3

4

5

6

7

8

9

10

11

DAY OF MONTH

FG (mg/dl) FG (mmol/l)

USL

CL

LSL

SPECIFICATION CHART Diabetes Mellitus

PATTERN: INTERVENTION:

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31

60

80

100

120

140

160

180

200

3

4

5

6

7

8

9

10

11

DAY OF MONTH

FG (mg/dl) FG (mmol/l)

SPECIFICATION CHART Diabetes Mellitus

PATTERN: INTERVENTION:

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31

60

80

100

120

140

160

180

200

3

4

5

6

7

8

9

10

11

DAY OF MONTH

FG (mg/dl) FG (mmol/l)

USL

CL

LSL

Peak Flow in AsthmaRun Chart - Ashtm a

0

50

100

150

200

250

300

350

400

450

500

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

Day

Pe

ak

Flo

w

Peak Flow in AshtmaRun Chart - Ashtm a

0

50

100

150

200

250

300

350

400

450

500

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

Day

Pe

ak

Flo

w

Peak Flow in AsthmaASTHMA - Control Chart (X)

UCL=381.85

LCL=290.65

CEN=336.25

UCL=395.48

LCL=202.02

CEN=298.75

UCL=459.85

LCL=387.65

CEN=423.75

0

50

100

150

200

250

300

350

400

450

500

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

Day

4

OUTCOMES

Doing what you do better

The best sources of clear learningobjectives in clinical medicine are theproblems presented by our own patients.

160 Diabetic Patients

Three Rapid Improvement Cycles

1. Patient self monitoring

2. Improved process of care

3. Use of best medication

DATA COLLECTIONPN DATE FBS HbA1c10251 4/8/1992 132 20.510063 4/8/1992 339 19.410163 4/20/1992 251 10.710075 4/23/1992 368 12.310719 4/23/1992 219 11.310251 5/6/1992 381 15.310025 6/1/1992 92 7.110719 6/2/1992 180 9.910063 6/3/1992 91 15.910248 6/9/1992 378 16.410251 6/10/1992 369 15.510075 6/29/1992 303 13.110163 7/1/1992 256 10.710491 7/9/1992 147 9.610075 7/23/1992 220 11.710248 8/14/1992 149 15.110251 8/26/1992 348 15.410191 9/10/1992 276 15.410719 9/10/1992 184 9.1

POPULATION BASED DATA

1992 - 1994

50 150 250 3500

20

40

60

80

100

0

20

40

60

80

100

BLOOD SUGAR

NU

MB

ER

COUNT : 631 MEDIAN : 168MEAN : 189MODE : 126STDEV : 84

NORMAL RANGE

© Larry V. Staker MD

DIABETES SPEC CHARTPATIENT

60

80

100

120

140

160

180

200

60

80

100

120

140

160

180

200

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

FB

S

MONTH

© Larry V. Staker MD

DIABETES SPEC CHART

KW D M N I D

0

50

100

150

200

250

300

350

400

0

50

100

150

200

250

300

350

400

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

374 219 180 184 182 265 198 190 191 173 153 144 133 143 132 150 136 129 120 111 124 141 120 149 118 141 130 131 128 120 133

FB

S

SPC

© Larry V. Staker MD

DIABETES SPEC CHART

N N D M I D

0

50

100

150

200

250

300

350

400

0

50

100

150

200

250

300

350

400

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

264274225231228166126139148105141 83 136151117116121126101121145122144113 69 122145126159181139

FB

S

SPC

© Larry V. Staker MD

DIABETES SPEC CHART D B D M B I D S

0

50

100

150

200

250

300

350

400

0

50

100

150

200

250

300

350

400

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

219228275252276246133182140176147138110 98 118140110146120128138 92 90 118126109122132103112117

FB

S

SPC

© Larry V. Staker MD

DIABETES SPEC CHART JF D M BIDS

60

80

100

120

140

160

180

200

60

80

100

120

140

160

180

200

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

81 98 110 99 110108120 96 90 91 85 108 99 92 107102 83 92 125 98 102102109 95 116102106 98 80 130113

FB

S

OCTOBER

© Larry V. Staker MD

FBS IN PATIENTS WITH DIABETES

0

50

100

150

200

250

YR 92 93 94 95 96217 189

176 171 166

DISPLAY OF MEAN OF ALL FBS DONE EACH YEAR

0

50

100

150

200

250

© Larry V. Staker MD

HbA1c IN PATIENTS WITH DIABETES

REF RANGE: 4.8% - 7.8% ION EXCHANGE METHOD

NOTE: 1.0% CHANGE IN HbA1c = 30 MG/DL CHANGE IN FBS

DISPLAY OF MEAN OF ALL Hb A1c DONE EACH YEAR

0

2

4

6

8

10

12

YR 92 93 94 95 96

11.5 11.2

9.5 9.4 8.9

0

2

4

6

8

10

12

© Larry V. Staker MD

HYPOTHESIS TESTINGHo: μA = μB or Ho: σA = σB

CYCLE 2 CYCLE 3 CYCLE 4 OUTCOME

Z Z Z Z Z Z Z Z Z93-92 94-93 95-94 96-95 97-96 98-97 99-98 00-99 00-950.999759 1.000000 0.674829 0.518497 1.000000 1.000000 1.000000 0.594465 1.000000

T T T T T T T T T93-92 94-93 95-94 96-95 97-96 98-97 99-98 00-99 00-950.013251 0.000005 0.628359 0.841768 0.000010 0.000205 0.000086 0.863190 0.000116

F F F F F F F F F93-92 94-93 95-94 96-95 97-96 98-97 99-98 00-99 00-950.252622 0.083474 0.529625 0.664346 0.000938 0.003996 0.244735 0.253290 0.000291

Improvement 1995-2000Hold Gains

BASELINE CYCLE 1

DCCT Published NEJM - 1993 Use of DM Spec and Run Charts

Improve DM Care

Use Best Meds

1992 1993 1994 1995 1996 1997 1998 1999 2000

Percent TTG (Yeild) 13.3% 20.4% 38.6% 41.4% 37.7% 59.2% 73.4% 60.8% 63.8%Sigma 0.4 0.7 1.2 1.3 1.2 1.7 2.1 1.8 1.9

N 1000 1000 1000 1000 1000 1000 1000 1000 1000D or DPTO 866.667 795.580 613.757 585.586 623.188 407.692 266.254 391.534 361.963

eRATE or %d or %NTTG 86.7% 79.6% 61.4% 58.6% 62.3% 40.8% 26.6% 39.2% 36.2%Yield Probability 0.00

Number of Projects 9

Nth Root of Yield Probability 0.40Estimated Ave TTG (Yeild) 40.3%

DPMO MILLION DPKO THOUSAND %TTG %NTTG SIGMABASELINE PERFORMANCE

597483 1000000 597.483 1000 40.25% 59.75% 1.3TEN FOLD IMPROVEMENT

59748 1000000 59.748 1000 94.03% 5.97% 3.1

298742 1000000 298.742 1000 70.13% 29.87% 2.0TWO FOLD IMPROVEMENT

This tool calculates SIGMA for multiple clinical outcomes from input of percent treated to goal (TTG). It also allows forcasting or projection of overall DPKO, %TTG, eRATE (%NTTG) and SIGMA for ten fold and two fold improvement.

HbA1c <= 8.0

If you know the percent of patients treated to goal use this tool.

© Larry V. Staker MD

ESTIMATED NORMALIZED TTG and CALCULATION OF OVERALL SIGMAEnter Percent TTG (as a number like 75.5) in fields colored green

EVALUATION OF OVERALL SIX SIGMA PERFORMANCE

1992 1993 1994 1995 1996 1997 1998 1999 2000

Percent TTG (Yeild) 8.1% 6.6% 20.1% 26.1% 22.5% 36.5% 50.2% 36.5% 38.7%Sigma 0.1 0.0 0.7 0.9 0.7 1.2 1.5 1.2 1.2

N 1000 1000 1000 1000 1000 1000 1000 1000 1000D or DPKO 918.519 933.702 798.942 738.739 775.362 634.615 498.452 634.921 613.497

eRATE or %d or %NTTG 91.9% 93.4% 79.9% 73.9% 77.5% 63.5% 49.8% 63.5% 61.3%Yield Probability 0.00

Number of Projects 9

Nth Root of Yield Probability 0.23Estimated Ave TTG (Yeild) 22.8%

DPMO M DPKO K %TTG %NTTG SIGMABASELINE PERFORMANCE

772255 1000000 772.255 1000 22.77% 77.23% 0.8TEN FOLD IMPROVEMENT

77225 1000000 77.225 1000 92.28% 7.72% 2.9

386127 1000000 386.127 1000 61.39% 38.61% 1.8TWO FOLD IMPROVEMENT

This tool calculates SIGMA for multiple clinical outcomes from input of percent treated to goal (TTG). It also allows forcasting or projection of overall DPKO, %TTG, eRATE (%NTTG) and SIGMA for ten fold and two fold improvement.

HbA1c <= 7.0

If you know the percent of patients treated to goal use this tool.

© Larry V. Staker MD

ESTIMATED NORMALIZED TTG and CALCULATION OF OVERALL SIGMAEnter Percent TTG (as a number like 75.5) in fields colored green

EVALUATION OF OVERALL SIX SIGMA PERFORMANCE

5

ACCOUNTABILITY

A PERSONAL GRADING SYSTEM

“HOW AM I DOING?”

The Sigma Metric: for Motorola

Percent DPMO 30.23% 697,672 1

69.15% 308,537 2

93.32% 66,807 3

99.38% 6,210 4

99.977% 233 5

99.9997% 3.4 6

ProcessCapability

Defects perMillion Opportunities

(distribution shifted ±1.5 s )

Increase in Sigmarequires exponentialdefect reduction

Error Free Yield

MANUFACTURING

DPMO

1

10

100

1000

10000

100000

1000000

1 2 3 4 5 6

DPMO

DPMO6976723085376680762102333.4

Lo

g S

cale

Sigma

Exponential Defect Reduction

The Sigma Metric: for Doctors

Percent DPHO GRADE

30.23% 69.7672 1 F

69.15% 30.8537 2 C

93.32% 6.6807 3 A

99.38% 0.6210 4

99.977% 0.0233 5

99.9997% 0.00034 6

ProcessCapability

Defects perHundred Opportunities

(distribution shifted ±1.5 s )

Increase in Sigmarequires exponentialdefect reduction

Error Free Yield or

TTG

CALCULATION OF SIGMA

SIGMA = NORMSINV (1 - ( #defects / #observations)) + 1.5

The “Sigma Metric” allows reliable comparison of improvement

6

$ENSE

MAKING THE BUSINESS CASE

“Breakeven and ROI”

RETURN ON INVESTMENT

A Profitability Ratio

Net Profit (from Profit and Loss Statement)________________________________________

Net Worth (from Balance Sheet)

Return On Investment- the profitability ratio –

value = ?? 0.10 ??

$ (Savings from best care)

$ (Investment in Equipment + Care)_______________________________

Types of Economic Evaluations

• Cost Comparison Analysis

• Cost Benefit Analysis

• Cost Effectiveness Analysis

• Cost Utility Analysis

• Cost Outcomes Analysis

Cost Comparison Analysis

• Comparison of costs and of two or more alternative therapies that have identical outcomes

• Examples– Generic versus brand name

– Different routes of administration

COST COMPARISON ANALYSIS

$800414.0CAD47.8%0.9295.0%19.060.234.7683.0%81.0%Dobutamine ECHO

$400414.0CAD46.1%0.8596.7%29.450.217.3689.0%81.0%Stress ECHO

$1,200414.0CAD29.5%0.4296.3%26.000.106.5086.0%91.0%Dobutamine

$1,200414.0CAD33.9%0.5194.2%16.360.134.0978.0%90.0%Dipyridamole-iv

$1,200414.0CAD40.9%0.6993.3%13.920.173.4875.0%87.0%Dipyridamole-o

$1,200414.0CAD34.6%0.5395.4%20.940.135.2483.0%89.0%Adenosine

$1,200414.0CAD35.7%0.5692.8%12.860.143.2172.0%90.0%SPECT

$1,000414.0CAD34.5%0.5397.5%39.110.139.7891.0%88.0%Thallium

        ETT and

414.0CAD76.3%3.2299.4%160.000.8040.0099.5%20.0%>= 2.5

414.0CAD73.0%2.7199.2%132.000.6833.0099.0%33.0%2.0 - 2.49

414.0CAD70.3%2.3798.8%84.000.5921.0098.0%42.0%1.50 - 1.99

414.0CAD61.1%1.5795.9%23.640.395.9189.0%65.0%1.0 - 1.49

414.0CAD42.1%0.7393.7%14.960.183.7477.0%86.0%0.5 - 0.99

 $300  NEGNEGPOSPOS  ETT and ST Seg ↓

CostTestICD9DxPTProbPTLRPTProbPTLRLR -LR +SPECSENTEST

Incremental Cost EffectivenessPresentation of Results

CA - CB

EOA - EOB

=

Cost for an additionalunit of effectiveness

Example:

CA - CB

EOA - EOB

=

$3194A - $2617B

45.6A - 42.9B

= $214 to gain anadditional unit ofeffectiveness with A

IMPROVING SKILLS OF MANAGING COST

CEvaluation of Cost of Care

TEST SEN SPEC LR + LR - PostTLR PostTProb PostTLR PostTProb Dx ICD9 Cost/TestETT and ST Seg ? POS POS NEG NEG

0.5 - 0.99 86.0% 77.0% 3.74 0.18 5.61 84.9% 0.27 21.4% CAD 414.0 $3001.0 - 1.49 65.0% 89.0% 5.91 0.39 8.86 89.9% 0.59 37.1% CAD 414.0 $300

1.50 - 1.99 42.0% 98.0% 21.00 0.59 31.50 96.9% 0.89 47.0% CAD 414.0 $3002.0 - 2.49 33.0% 99.0% 33.00 0.68 49.50 98.0% 1.02 50.4% CAD 414.0 $300

>= 2.5 20.0% 99.5% 40.00 0.80 60.00 98.4% 1.21 54.7% CAD 414.0 $300ETT and

Thallium 88.0% 91.0% 9.78 0.13 14.67 93.6% 0.20 16.5% CAD 414.0 $1,000SPECT 90.0% 72.0% 3.21 0.14 4.82 82.8% 0.21 17.2% CAD 414.0 $1,200

Adenosine 89.0% 83.0% 5.24 0.13 7.85 88.7% 0.20 16.6% CAD 414.0 $1,200Dipyridamole-o 87.0% 75.0% 3.48 0.17 5.22 83.9% 0.26 20.6% CAD 414.0 $1,200Dipyridamole-iv 90.0% 78.0% 4.09 0.13 6.14 86.0% 0.19 16.1% CAD 414.0 $1,200

Dobutamine 91.0% 86.0% 6.50 0.10 9.75 90.7% 0.16 13.6% CAD 414.0 $1,200Stress ECHO 81.0% 89.0% 7.36 0.21 11.05 91.7% 0.32 24.3% CAD 414.0 $500Dobutamine ECHO 81.0% 83.0% 4.76 0.23 7.15 87.7% 0.34 25.6% CAD 414.0 $800

COST MINIMIZATION and COST EFFECTIVENESS ANALYSES

LR rule: +>25; -<0.25

IMPROVING SKILLS OF MANAGING COST

TOOL FOR EVALUATION OF COST AND EFFECTIVENESS OF TREATMENT TO GOAL (TTG) [Gives Cost of Quality] © Larry V. Staker MDEnter Numbers in Red Boxes:

POPULATION WITH A DISEASE PROPORTION $300 cost visits/yrTTG NTTG n p $1,000 cost Rx/yr

experiment a 140 20 b 160 a+b 87.5% EER or %TTG $500 cost tests/yr

control c 25 135 d 160 c+d 15.6% CER or %NTTG $1,800 TOTALPanel Size Totals: 165 155 320 (a+b)+(c+d) 5.3% Prevalence

3000 a+c b+d0.5 2.7

FormulasSE Ln Ln SE

p1=a/(a+b) 0.026 EER 87.5% 82.4% 92.6% EER Experiment Event Ratep2=c/(c+d) 0.029 CER 15.6% 10.0% 21.3% CER Control Event RateI EER-CER I 0.027 ARR 71.9% 66.6% 77.2% ARR Absolute Risk Reduction1/ARR NNT 1.4 1.3 1.5 NNT Number Needed to TreatEER/CER RR 560.0% 388.8% 806.5% RR Relative Risk

0.186 1.7228 CI95 LnRR 1.358 2.088 CI95 LnRR Natural Log RBI EER-CER I /CER RRR 460.0% 288.8% 706.5% RRR Relative Risk Reduction(a*d)/(c*b) ROR 37.8 20.1 71.2 ROR Relative Odds Ratio

0.323 3.632 CI95 LnOR 2.999 4.266 CI95 LnOR Natural Log OR

COST ANALYSISOUTCOME

$2,504Δ SIGMA: $704

Standard Error Calculation Enumerative Statistical AnalysisCI-95

Cost of Poor Quality

THE SIX SIGMA METHOD

Healthcare Delivery Systems

Larry V. Staker MD, FACP

The Acronym is DMAIC

1

2

3

4

5

DEFINE

MEASURE

ANALYZE

IMPROVE

CONTROL

SIX SIGMA

1a

1b

1c2

3

4

5

PROCESSIMPROVEMENT

DEFINECORE PROCESSES

DEFINEKEY

CUSTOMERS

DEFINECUSTOMER

REQUIREMENTS

MEASURECURRENT

PERFORMANCE

ANALYZE

IMPROVE

CONTROLINTEGRATE

EXPANDP

D

C

A

SIX SIGMA

hearingseeing

and measuring

The Voice Of The CustomerVOC

SIPOC - Process Analysis

SUPPLIERS

CUSTOMERS

OutputsInputs PROCESS

SIX SIGMA: Minimize Defects

Percent DPMO 30.23% 697,672 1

69.15% 308,537 2

93.32% 66,807 3

99.38% 6,210 4

99.977% 233 5

99.9997% 3.4 6

ProcessCapability

Defects perMillion Opportunities

(distribution shifted ±1.5 s )

Increase in Sigmarequires exponentialdefect reduction

Error Free Yield

The Sigma Metric: for Doctors

Percent DPHO GRADE

30.23% 69.7672 1 F

69.15% 30.8537 2 C

93.32% 6.6807 3 A

99.38% 0.6210 4

99.977% 0.0233 5

99.9997% 0.00034 6

ProcessCapability

Defects perHundred Opportunities

(distribution shifted ±1.5 s )

Increase in Sigmarequires exponentialdefect reduction

Error Free Yield or

TTG

SIX SIGMA: Customer ServiceThe process shown here is stable.

But why does it need to be improved?

} CustomerNeed

Time

LSL

USL

UCL

LCL

SIX SIGMA

DATA DRIVEN

Voice Of The CustomerDefects Reduction Error Free Yield

VOCSigma Metric

REFERENCES

Staker LV. Practice Based Learning For Improvement: The pursuit of clinical excellence. Texas Medicine; V96, N10, Oct 2000, page 53-60.

Staker LV. Changing Clinical Practice by Improving Systems: The pursuit of clinical excellence through practice-based measurement for learning and improvement. QualityManagement in Health Care; V9, N1, Fall 2000, page 1-13.

REFERENCE

Carey, Raymond G. Improving Healthcare with Control Charts: Basic and Advanced SPC Methods and Case Studies. ASQ Quality Press, Milwaukee, WI. September, 2002, 194 pages. ISBN 0-87389-562-2

Chapter 10, pages 159-183 by Larry V. Staker MDThe Use of Run Charts and Control Charts in the Improvement of Clinical Practice.