Dichotomous Tests

Thomas B. Newman, MD, MPH

September 27, 2012

Thanks to Josh Galanter and Michael Shlipak

1

Overview Clarifications, chapter 1, chapter 2

material Definitions: sensitivity, specificity, prior

and posterior probability, predictive value, accuracy

2 x 2 table method Likelihood ratios - WOWO Probability and odds FP/FN confusion Test/treat thresholds 2

Clarifications

EBD errata on book website SLUBI= Self limited undiagnosed

benign illness – not a term I use with parents

3

Definitions: Sensitivity and Specificity

4

Disease status

Has disease

No disease

Total

Test Result

Positive

A B A + B

Negative

C D C + D

Total A + C B + D A + B + C + D

Sensitivity = A/ (A+C)

Specificity = D/ (B+D)

P.I.D. = Positive in Disease

N.I.H.= Negative in Health

Definitions: Positive and Negative Predictive value, Accuracy

5

Disease status

Has disease

No disease

Total

Test Result

Positive

A B A + B

Negative

C D C + D

Total A + C B + D A + B + C + D

PPV=A/(A+B)NPV=D/(C+D)

Accuracy = (A + D)/(A + B + C + D) = (A+D)/N Accuracy demonstration: screening for brain tumors

Definitions: Pretest (prior) and post-test (posterior) probability

6

Disease status

Has disease

No disease

Total

Test Result

Positive

A B A + B

Negative

C D C + D

Total A + C B + D A + B + C + D

Pretest probability =

ONLY IF SAMPLING IS “CROSS-SECTIONAL”!

(A+C)/(A+B+C+D)

Posttest probability = A/(A+B) or C/(C+D)

“Cross-sectional” sampling

7

Disease status

Has disease

No disease

Total

Test Result

Positive

A B A + B

Negative

C D C + D

Total A + C B + D A + B + C + D

PPV=A/(A+B)NPV=D/(C+D)

Subjects are sampled randomly or consecutively, so that the proportion with disease (pretest probability, prevalence) is clinically meaningful

“Case-control” sampling

8

Disease status

Has disease

No disease

Total

Test Result

Positive

A B A + B

Negative

C D C + D

Total A + C B + D A + B + C + D

PPV=A/(A+B)NPV=D/(C+D)

Subjects with and without disease are sampled separately

Proportion with disease is determined by investigator

Disease status

Has disease

No disease

Total

Test Result

Positive

A B A + B

Negative

C D C + D

Total A + C B + D A + B + C + D

Prevalence vs Pretest probability

Pretest probability is the more general term

For screening tests, pretest probability = prevalence

For diagnostic tests, pretest probability incorporates history and physical exam items

9

Post-test probability vs. Predictive value Posttest probability after a + test is the

same as positive predictive value Posttest probability after a – test is

1– negative predictive value

10

2 2 Table Method

Research vignette

“Tom, you need to call this mother. She’s really upset.”

11

Choroid Plexus Cyst

12

Fill in table

13

Pretest probability 0.0003 Sensitivity 33% Specificity 98.5%

Disease status

Trisomy 18

No Trisomy 18

Total

ChoroidPlexus Cyst

Present

Absent

Total

Likelihood Ratios

14

Likelihood ratios

A ratio of likelihoods:P(Result|Disease)P(Result|No Disease)

WOWO = With Over WithOut

Pretest odds x LR = Posttest odds (Prior odds x LR = Posterior odds)

15

What Tests Do•Their results change the probability of disease

Negative test Positive test

Reasurance TreatmentOrder a Test

•A good test moves us across action thresholds.

0% 100%

HIV+HIV-

16

Likelihood of Disease Depends on 2 Things

1. Where you started from (low, medium, high risk)

2. Length and direction of the “arrow”

Basic paradigm: – What we thought before test result

what we think now

17

Likelihood ratio Effect of test result

Very small (0.01) Greatly decreases P(disease)

Less than 1 (0.5) Decreases P(disease)

One No effect on P(disease)

More than 1 (2) Increases P(disease)

Very big (100) Greatly increases P(disease)

18

Likelihood Ratios Advantages

– Calculation of post-test probability easier (especially when disease is rare)

– Capture information for multi-level and continuous tests (next week)

Disadvantages– If either pretest or posttest probability is high

(~> 10%) you need to use odds (or a slide rule or calculator)

19

Switch to board

LR for the choroid plexus cyst example– Dichotomous test def of LR

Probability and odds

20

Can Use Slide Rule

21

False-negative confusion

Sensitivity of rapid strep test is 85% Therefore, false negative rate is 15% 15% is too high, so always culture to

confirm negative rapid strep tests

22

What’s wrong?Strep No Strep Total

Rapid Test + TP FP TP+FPRapid Test - FN TN TN+FN

TP+FN FP+TN

2 definitions of “false negative rate”– Def #1: 1-sensitivity = FN/(TP+FN). This one is easier because

it’s (assumed to be) constant.– Def #2: 1 - negative predictive value = FN/(FN+TN). This one is

harder because it depends on prior probability, but it is the one that should determine clinical decisions.

23

If prior probability of strep = 20% and specificity is 98%

False negative rate (def #2) = 15/407 = 3.7% NNC (number needed to culture) = 1/.037 = 27

to identify 1 false negative rapid test. (Pre-test probability of 20%)

At some prior probability of strep, culture after negative quick test is not indicated.

Strep No Strep TotalRapid test + 85 8 95Rapid test - 15 392 407Total 100 400 500

24

25

Sensitivity 85% Specificity 98% Prior probability = 20% Rapid test is NEGATIVE LR =

Try it with slide rule!

Similar examples:

Sensitivity of UA for UTI is only 80%, therefore always culture after a negative UA

Sensitivity of CT scan for subarachnoid hemorrhage is only 90%, therefore always do LP after a negative CT

False positive confusion is similar: 1-specificity vs. 1-positive predictive value

26

Test/Treat Thresholds

No test TreatTest

27

“X-Graph”

28

New “X-Graph”

29

Questions?

30