ruling in or out a disease tests to rule out a disease you want very few false negatives high...
TRANSCRIPT
Ruling in or out a disease
Tests to rule out a disease
You want very few false negatives High sensitivity
Thus, if you get a negative test, it is likely a true negative
Mnemonic: SnOUT (High SeNsitivity rules OUT disease)
Example: D-dimer for DVT/PE
Test to rule in a disease
You want very few false positives High specificity
A positive test is likely to be a true positive
Mnemonic: SpIN (High Specificty rules IN disease)
Example: Pathology for malignancy
Prior and posterior disease probabilities
Prior and Posterior Probabilities
What you thought before + New Information = What you think now “What you thought before” = Prior (pre-test) probability
Disease prevalence Probability of disease given patient’s presentation
“New Information” = Test result “What you think now” = Posterior (post-test) probability
For positive dichotomous test, positive predictive value For negative dichotomous test, 1 – negative predictive value For multilevel and continuous tests, see chapter 4
Serial tests
2x2 table method for updating prior
probabilities• How to populate a 2 x 2 table• Example: Serological testing for TB
How to populate a 2 x 2 table
Given sensitivity, specificity, prevalence (prior probability)
Calculate positive predictive value, negative predictive value, etc.
Scenarios Applying test characteristics derived in one population to
another Applying test characteristics derived from a case / control
study Applying independent tests serially*
Assume sensitivity and specificity are intrinsic to test and independent of population
* Coming in chapter 8, “Multiple tests and multivariable decision rules”
How to populate a 2 x 2 table
Use prevalence to calculate D+ and D- totals
Use sensitivity and specificity to calculate A, B, C, and D
Use A, B, C, and D to calculate positive and negative predictive values
Example: Serological Test for TB
Anda-TB IgG ELISA test for anti-A60
antibodies
Sensitivity 76%
Specificity 92%
Prevalence Uganda: 30% SFGH: 5%
TB No-TB
Total
Positive
Negative
Total 300 700 1000 Have TB
1000 x 30% = 300
Don’t have TB 1000 – 300 = 700
Example: Serological Test for TB
Anda-TB IgG ELISA test for anti-A60
antibodies
Sensitivity 76%
Specificity 92%
Prevalence Uganda: 30% SFGH: 5%
TB No-TB
Total
Positive 228
Negative 72
Total 300 700 1000 Test positive if they have
TB 300 x 76% = 228
Test negative if they have TB 300 – 228 = 72
Example: Serological Test for TB
Anda-TB IgG ELISA test for anti-A60
antibodies
Sensitivity 76%
Specificity 92%
Prevalence Uganda: 30% SFGH: 5%
TB No-TB
Total
Positive 228 56
Negative 72 644
Total 300 700 1000 Test negative if healthy
700 x 92% = 644
Test positive if healthy 700 – 644 = 56
Example: Serological Test for TB
Anda-TB IgG ELISA test for anti-A60
antibodies
Sensitivity 76%
Specificity 92%
Prevalence Uganda: 30% SFGH: 5%
TB No-TB
Total
Positive 228 56 284
Negative 72 644 716
Total 300 700 1000 Positive Predictive Value
228/284 = 80.3%
Negative Predictive Value 644/716 = 89.9%
Positive Predictive Value 228/284 = 80.3%
Negative Predictive Value 644/716 = 89.9%
Example: Serological Test for TB
Uganda (prevalence 30%)
SFGH (prevalence 5%)
TB No-TB
Total
Positive 38 76 114
Negative 12 874 886
Total 50 950 1000
TB No-TB
Total
Positive 228 56 284
Negative 72 644 716
Total 300 700 1000 Positive Predictive Value
38/114 = 33.3%
Negative Predictive Value 12/886 = 98.6%
Sampling Scheme Matters
Cross sectional study Case control study
Positive Predictive Value 228/284 = 80.3%
Negative Predictive Value 644/716 = 89.9%
TB No-TB
Total
Positive 380 40 420
Negative 120 460 580
Total 500 500 1000
TB No-TB
Total
Positive 228 56 284
Negative 72 644 716
Total 300 700 1000 Positive Predictive Value
380/420 = 90.5%
Negative Predictive Value 460/580 = 79.3%
Odds and Probabilities
• Mmmm… pizza• Converting probabilities to odds and vice
versa
Mmmm… pizza
Imagine that you want to divide a pizza evenly between you and a friend
1 to 1 ratio of pizza between the two of you
Each gets 50% of the pizza
Mmmm… pizza
3 to 2 ratio of pepperonis between the two slices
One person gets 3/5 (60%) of the pepperonis
One person gets 2/5 (40%) of the pepperonis
Odds and Probabilities
Odds
‘Odds’ refers to the ratios of the two portions
Odds of 1 to 1 can be expressed as 1:1 or 1
Odds of 3 to 2 can be expressed as 3:2 or 1.5
Odds of 2 to 3 can be expressed as 2:3 or 0.67
Probabilities
‘Probabilities’ refers to the proportion of each portion to the whole
Probability of ½ is 0.5
Probability of 3/5 is 0.6
Probability of 2/5 is 0.4
Odds and Probabilities
Odds
Odds can range from 0 to ∞
To convert from probabilities: Odds = p / (1 – p)
Odds > probability
As p → 0 Odds ≈ probabilities
Probabilities
Probabilities can range from 0 to 1
To convert from odds: p = odds / (1 + odds)
Odds at low probabilities
Converting odds to Probability and vice versa
Odds of pizza is 0.33 (1:3) P = odds / (1 + odds) P = 0.33 / (1 + 0.33) P = 0.25 (¼)
Probability of pepperoni is 0.2 (⅕) Odds = p / (1 – p) Odds = 0.2 / (1 – 0.2) Odds = 0.25 (1:4)
Likelihood ratios for dichotomous tests
• Likelihood ratios• Example: Fetal fibronectin and preterm
labor
Likelihood ratios
Converts prior odds to posterior odds given a test result
Prior odds (A + C) / (B + D)
Posterior odds Positive test: A / B Negative test: C / D
Likelihood ratio is multiplier to go from prior odds to posterior odds
D+ D- Total
Positive A B A + B
Negative
C D A + C
Total A + C B + D NLR+ = sens/(1 – spec)
LR- = (1 – sens) / spec
Generalizing likelihood ratios
It is possible to have more than two results in a test Multilevel and continuous tests (chapter 4)
You can use likelihood ratios for these
LR+ and LR- don’t make sense
LRresult = P(result|D+) / P(result|D-)
LR method for updating prior probabilities
Step 1: Convert pretest probability to pretest odds
Step 2: Calculate appropriate likelihood ratio
Step 3: Multiply pretest odds by appropriate likelihood ratio to get post-test odds
Step 4: Convert post-test odds back to probability
Example: FFN for Preterm Labor
Fetal fibronectin: extracellular glycoprotein produced in the decidua and chorion Presence in vaginal secretions between 24 and 34
weeks gestation associated with preterm delivery ACOG recommends against screening asymptomatic
women “May be useful in patients at risk for preterm birth”
(within 7 days)
Example: FFN for Preterm Labor
Clinical scenario: 35 year old woman at 30 weeks gestation and history of preterm births, complains of “uterine tightening” Should she be given tocolytics and steroids for fetal
lung maturation
Fetal Fibronectin Sensitivity: 76% Specificity: 82% Prior probability of preterm birth within 7 days: 8% Sanchez-Ramos et al. Obstet Gynecol 2009;114:631–40
Example: FFN for Preterm Labor
Step 1: Convert prevalence to odds Odds = p/(1 – p)
0.08/(1 – 0.08) = .087
Remember when probability is small, odds ~ probability
Example: FFN for Preterm Labor
Step 2: Calculate likelihood ratios LR+ = sensitivity / (1 – specificity) LR- = (1 – sensitivity) / specificity
LR+ = 0.76 / (1 – 0.82) = 0.76 / 0.18 = 4.2
LR- = (1 – 0.76) / 0.82 = 0.24 / 0.82 = 0.29
Example: FFN for Preterm Labor
Step 3: Multiply pretest odds by appropriate likelihood ratio Post-test odds = pretest-odds x LR
For positive test: Post-test odds = 0.087 x 4.2 Post-test odds = 0.37
For negative test Post-test odds = 0.087 x 0.29 Post-test odds = 0.025
Example: FFN for Preterm Labor
Step 4: Convert post-test odds back to probability p = odds / (1 + odds)
For positive test: Post-test probability = 0.37 / (1 + 0.37) Post-test probability = 27%
For negative test Post-test probability = 0.025 / (1 + 0.025) Post-test probability = 2.48%
Example: FFN for Preterm Labor
Positive predictive value 61/227 = 26.9%
Negative predictive value 754/773 = 97.5%
preterm
not preterm
Total
Positive 61 166 227
Negative
19 754 773
Total 80 920 1000Sensitivity:
76%Specificity: 82%
Treatment and Testing Thresholds
• Quantifying costs and benefits• Testing thresholds for a perfect but risky
or expensive test
Quantifying costs and benefits
A “wrong” clinical decision carries cost Treatment of individuals without disease
Cost of therapy Discomfort, side effects, etc.
Failure to treat individuals with disease Pain and suffering Lost productivity Additional cases
Quantifying costs and benefits
Benefits of tests Increases probability of making the “right” decision Reduces costs from “wrong” decision
Costs of tests Cost of the test itself Discomfort and complications from performing the test May still lead to “wrong” decision (imperfect
sensitivity, specificity)
Quantifying costs and benefits
C = Cost of unnecessary treatment
B = Benefit forgone by failure to treat
T = Testing cost Cost of test itself, pain
and discomfort associated with procedure
Does not include cost of errors
Treatment Thresholds
Balance costs Cost of treatment x
probability of unnecessary treatment (no disease)
Net benefit forgone x probability of failure to offer treatment (probability of disease)
Cost of test
Treatment Thresholds
Treat: Probability of disease is high and/or Cost of treatment is low
Don’t treat: Probability of disease is low and/or Cost of failure to treat is low and/or Cost of unnecessary treatment is
high Cancer chemotherapy
Test: Test offers substantial
improvement in diagnosis and/or cost of test is low
Cost of treatment
Cost of treatment x probability of unnecessary treatment (no disease) C x (1 – P)
Cost of no treatment
Net benefit forgone x probability of failure to offer treatment (probability of disease) B x P
Lowest Cost Option
Black line is lowest cost option
Cost of treatment > Cost of no treatment Don’t treat (red)
Cost of treatment < Cost of no treatment Treat (green)
Cost of treatment = Cost of no treatment Treatment threshold (PTT)
Extending to dichotomous tests
Why you would do a test
Cost of treatment and/or failure to treat is high
Test will reduce misdiagnosis Test has good
performance characteristics
Test is used when diagnosis is ambiguous Pre-test probability is in
the middle
Why you wouldn’t do a test
The test is expensive
The test is unlikely to help The test’s performance
characteristics are limited You are already sure or
nearly sure of the diagnosis Pre-test probability is
very high or very low
Cost of a perfect test
If the test is perfect, the only cost to consider is the cost of the test T T is constant at all
probabilities
Lowest Cost Option
Black line is lowest cost option
Cost of no treatment < Cost of test Don’t treat (red)
Cost of treatment < Cost of test Treat (green)
Cost of test < Cost of empiric treatment/no treatment Test (yellow)
Lowest Cost Option
Treat/No Treat threshold P = C / (C + B)
Test/Treat threshold P = 1 – T/C
No Treat/Test threshold P = T/B
Example: EGFR in NSCLC
EGFR by PCR and fragment analysis for non-small cell lung cancer (NSCLC) Cost of test: $400 Assume sensitivity and specificity = 100% Prevalence of EGFR mutation (19% in males to 26% in females)
C = Cost of erlotinib $1300/mo x 3 months + risk of rash, diarrhea, occasional
interstitial pneumonitis ~ $4,000
B = Benefits of erlotinib in EGFR positive patients 3.2 months of progression free survival – cost of drug ~ $11,000
Example: EGFR in NSCLC
Imperfect tests
Must consider residual probability of being wrong
Lots of complex algebra (covered in the book)
Can also use Excel spreadsheet Available on course website
http://rds.epi-ucsf.org/ticr/syllabus/courses/4/2011/10/06/Lecture/tools/Treatment_Testing_Thresholds_Galanter.xls
Example
Summary
A dichotomous test is a test with two possible outcomes
We covered the definitions of sensitivity, specificity, prevalence, PPV, NPV, & accuracy
What you thought before + New information = What you think now
Probabilities can be updated using a 2 x 2 table
Odds are the ratio of two portions, Probabilities the proportion from the whole
Likelihood ratios convert prior odds to posterior odds given a test result
Treatment and testing thresholds allow you to estimate the lowest cost option