statistical evaluation of diagnostic tests
DESCRIPTION
STATISTICAL EVALUATION OF DIAGNOSTIC TESTS. Describing the performance of a new diagnostic test. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/1.jpg)
STATISTICAL EVALUATION STATISTICAL EVALUATION OF DIAGNOSTIC TESTSOF DIAGNOSTIC TESTS
![Page 2: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/2.jpg)
Describing the performance of a new diagnostic testDescribing the performance of a new diagnostic test
Physicians are often faced with the task of evaluation
the merit of a new diagnostic test. An adequate critical
appraisal of a new test requires a working knowledge
of the properties of diagnostic tests and the
mathematical relationships between them.
![Page 3: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/3.jpg)
The gold standard test: Assessing a new diagnostic
test begins with the identification of a group of
patients known to have the disorder of interest, using
an accepted reference test known as the gold
standard.
Limitations:
1) The gold standard is often the most risky,
technically difficult, expensive, or impractical of
available diagnostic options.
2) For some conditions, no gold standard is
available.
![Page 4: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/4.jpg)
The basic idea of diagnostic test interpretation is to
calculate the probability a patient has a disease under
consideration given a certain test result. A 2 by 2 table
can be used for this purpose. Be sure to label the table
with the test results on the left side and the disease
status on top as shown here:
Test Disease
Present Absent
Positive True Positive False Positive
Negative False Negative True Negative
![Page 5: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/5.jpg)
The sensitivity of a diagnostic test is the
probability that a diseased individual will have a
positive test result. Sensitivity is the true positive
rate (TPR) of the test.
Sensitivity = P(T+|D+)=TPR
= TP / (TP+FN)
diseased all
test positive withdiseased
![Page 6: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/6.jpg)
The specificity of a diagnostic test is the
probability that a disease-free individual will have a
negative test result. Specificity is the true negative
rate (TNR) of the test.
Specificity=P(T-|D-) = TNR
=TN / (TN + FP).
free-disease all
test negative withfree-disease
![Page 7: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/7.jpg)
False-positive rate: The likelihood that a nondiseased patient has an abnormal test result.
FPR = P(T+|D-)=
= FP / (FP+TN)
free-diseased all
test positive withfree-disease
![Page 8: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/8.jpg)
False-negative rate: The likelihood that a diseased patient has a normal test result.
FNR = P(T-|D+)=
= FN / (FN+TP)
diseased all
test negative withdiseased
![Page 9: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/9.jpg)
Pretest Probability is the estimated likelihood of
disease before the test is done.
It is the same thing as prior probability and is often
estimated. If a defined population of patients is being
evaluated, the pretest probability is equal to the
prevalence of disease in the population. It is the
proportion of total patients who have the disease.
P(D+) = (TP+FN) / (TP+FP+TN+FN)
![Page 10: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/10.jpg)
Sensitivity and specificity describe how well the test
discriminates between patients with and without
disease. They address a different question than we
want answered when evaluating a patient, however.
What we usually want to know is: given a certain test
result, what is the probability of disease? This is the
predictive value of the test.
![Page 11: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/11.jpg)
Predictive value of a positive test is the proportion
of patients with positive tests who have disease.
PVP=P(D+|T+) = TP / (TP+FP)
This is the same thing as posttest probability of
disease given a positive test. It measures how well
the test rules in disease.
![Page 12: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/12.jpg)
Predictive value of a negative test is the
proportion of patients with negative tests who do not
have disease. In probability notation:
PVN = P(D-|T-) = TN / (TN+FN)
It measures how well the test rules out disease. This
is posttest probability of non-disease given a
negative test.
![Page 13: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/13.jpg)
Evaluating a 2 by 2 table is simple if you are methodical in your approach.
Test Disease
Present Absent
Positive TP FP Total positive
Negative FN TN Total negative
Total with disease
Total with- out disease
Grand total
![Page 14: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/14.jpg)
Bayes’ Rule MethodBayes’ Rule MethodBayes’ rule is a mathematical formula that may be used as an alternative to the back calculation method for obtaining unknown conditional probabilities such as PVP or PVN from known conditional probabilities such as sensitivity and specificity.
FPRDpTPRDp
TPRDpTDPPVP
))(1()(
)()(
FNRDpTNRDp
TNRDpTDPPVN
))(1()(
)()(
)()()()(
)()()(
ABPAPABPAP
ABPAPBAP
General form of Bayes’ rule is
Using Bayes’ rule, PVP and PVN are defined as
![Page 15: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/15.jpg)
Example The following table summarizes results of a study to evaluate the dexamethasone suppression test (DST) as a diagnostic test for major depression. The study compared results on the DST to those obtained using the gold standard procedure (routine psychiatric assessment and structured interview) in 368 psychiatric patients.
1. What is the prevalence of major depression in the study group?
2. For the DST, determine
a-Sensitivity and specificity
b-False positive rate (FPR) and false negative rate (FNR)
c-Predictive value positive (PVP) and predictive value negative (PVN)
![Page 16: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/16.jpg)
DST
Result
Depression Total
+ -
+ 84 5 89
- 131 148 279
Total 215 153 368
Sensitivity = P(T+|D+)=TPR=TP/(TP+FN)=84/215=0.391
Specificity=P(T-|D-)=TNR=TN / (TN + FP)=148/153=0.967
FPR = P(T+|D-)=FP/(FP+TN)=5/153=0.033
FNR = P(T-|D+)=FN/(FN+TP)=131/215=0.609
PVN = P(D-|T-) = TN / (TN+FN)=148/279=0.53
PVP=P(D+|T+) = TP / (TP+FP)=84/89=0.944
P(D+) =215/368 =0.584
![Page 17: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/17.jpg)
FNR=1-Sensitivity=1-0.391=0.609
FPR=1-Specificity=1-0.967=0.033
![Page 18: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/18.jpg)
ROC (Receiver Operating Characteristic ) CURVEROC (Receiver Operating Characteristic ) CURVE
The ROC Curve is a graphic representation of the
relationship between sensitivity and specificity for a
diagnostic test. It provides a simple tool for applying the
predictive value method to the choice of a positivity
criterion.
ROC Curve is constructed by plottting the true positive rate
(sensitivity) against the false positive rate (1-specificty) for
several choices of the positivity criterion.
![Page 19: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/19.jpg)
ROC Curve
Diagonal segments are produced by ties.
1 - Specificity
1,00,75,50,250,00
Sensitivity
1,00
,75
,50
,25
0,00
![Page 20: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/20.jpg)
Disease-free Diseased
Test negative Test positive
TP
FP
FN
TN
2 1
1
11
ix
z
2
22
ix
z
Positivity criterion
xi
![Page 21: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/21.jpg)
Example: One of the parameters which are evaluated for the diagnosis of CHD, is the value of “HDL/Total Cholesterol”. Consider a population consisting of 67 patients with CHD, 93 patients without CHD. The result of HDL/Total Cholesterol values of these two groups of patients are as follows.
CHD+
Hdl/Total Cholestrol
CHD-
Hdl/Total Cholestrol
0,29
0,26
0,39
0,16
.
.
.
0,25
0,36
0,30
0,20
.
.
.
![Page 22: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/22.jpg)
To construct the ROC Curve, we should find sensitivity and specificity for each cut off point. We have two alternatives to find these characteristics.
• Cross tables
• Normal Curve
Descriptive Statistics
HDL/Total Cholestrol
,2926 ,066 ,16 ,52,2301 ,048 ,06 ,34
GROUPCHD-CHD+
Mean SD Min Max
![Page 23: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/23.jpg)
If HDL/Total Cholestrol is less than or equal to 0,26, we classify this group into diseased.
64 15 79
68,8% 22,4% 49,4%
29 52 81
31,2% 77,6% 50,6%
93 67 160
100,0% 100,0% 100,0%
Count
Count
Count
0,26>
0,26<=
RATIO
Total
- +
CHD
Total
SensitivitySpecificity
![Page 24: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/24.jpg)
Best cutoff point
Cutoff TPR FPR0,000 0,000 0,0000,093 0,015 0,0000,129 0,030 0,0000,142 0,045 0,0000,156 0,060 0,0000,158 0,075 0,0000,162 0,075 0,0110,168 0,104 0,0110,171 0,119 0,0110,173 0,119 0,0220,175 0,119 0,032
. . .
. . .
. . .0.26 0.78 0.31
. . .
. . .0,393 1,000 0,9350,402 1,000 0,9460,407 1,000 0,9570,420 1,000 0,9680,446 1,000 0,9780,493 1,000 0,9891,000 1,000 1,000
92 59 151
98,9% 88% 94%
1 8 9
1,1% 12% 5,6%
93 67 160
100% 100% 100%
0,171<
0,171>=
RATIO
Total
- +
CHD
Total
Let cutoff=0,171
![Page 25: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/25.jpg)
ROC Curve
1 - Seçicilik
1,0,9,8,7,6,5,4,3,2,10,0
Se
ns
itiv
ity
1,0
,9
,8
,7
,6
,5
,4
,3
,2
,1
0,0
1-Specificity
Cutoff=0.26
TPR=0.78
FPR=0.31
TNR=0.69
FNR=0.22
![Page 26: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/26.jpg)
Area Under the Curve
Test Result Variable(s): ORAN
,778 ,036 ,000 ,708 ,849Area Std. Error
aAsymptotic
Sig.b
Lower Bound Upper Bound
Asymptotic 95% ConfidenceInterval
The test result variable(s): ORAN has at least one tie between thepositive actual state group and the negative actual state group. Statisticsmay be biased.
Under the nonparametric assumptiona.
Null hypothesis: true area = 0.5b.
![Page 27: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/27.jpg)
If Patients with CHD and without CHD are normally distributed, we can easily find sensitivity and specificity from the area under these normal curves. Sensitivity and specificity are calculated for each different cotoff points
CHD+ CHD-
Cutoff=0,28
TP
FP
FN
TN
0,23 0,29
![Page 28: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/28.jpg)
ZCHD+=(0.28-0.23)/0.048=1.04
TPR=0.5+0.3508=0.8508
FNR=1-TPR=0.1492
If we take cut off point=0.28, the characteristics of test are:
ZCHD-=(0.28-0,29)/0.066=-0.15
TNR=0.5+0.0596=0.5596
FPR=1-TNR=0.4404
Cutoff=0,28
0,23 0,29
CHD+ CHD-
![Page 29: STATISTICAL EVALUATION OF DIAGNOSTIC TESTS](https://reader035.vdocuments.site/reader035/viewer/2022062423/5681432e550346895daf9ef7/html5/thumbnails/29.jpg)
Cutoff TPR FNR TNR FPR0,10 0,00 1,00 1,00 0,000,15 0,05 0,95 0,98 0,020,20 0,27 0,73 0,91 0,090,25 0,66 0,34 0,73 0,270,28 0,85 0,15 0,56 0,440,30 0,93 0,07 0,44 0,560,35 0,99 0,01 0,18 0,820,45 1,00 0,00 0,00 1,00
0,000,100,200,300,400,500,600,700,800,901,00
0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1
FPR
TP
R