laboratory training for field epidemiologists sensitivity and specificity predictive values positive...

30
EP ID E M IC A LE R T A N D RESPO NSE Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

Upload: savannah-satchwell

Post on 16-Dec-2015

218 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Sensitivity and specificityPredictive values positive and negative

Interpretation of results

Sep 2007

Page 2: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Outline Outline

• Intrinsic characteristics of a test– Sensitivity

– Specificity

• Performance of a test in a population– Predictive value of a positive test

– Predictive value of a negative test

Page 3: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

SensitivitySensitivity

Sensitivity = True positives / Affected personsEstimate the 95% confidence interval

Persons testing positive(True positives)

Persons testing negative(False negatives)

Affected persons(Positive by gold standard)

The sensitivity of a test in the ability of the test to identify correctly affected individuals

Proportion of persons testing positive among affected individuals

Page 4: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Estimating the sensitivity of a testEstimating the sensitivity of a test

• Identify affected individuals with a gold standard

• Obtain a wide panel of samples that are representative of the population of affected individuals– Recent and old cases

– Severe and mild cases

– Various ages and sexes

• Test the affected individuals

• Estimate the proportion of affected individuals that are positive with the test

Page 5: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Estimating the sensitivity of a rapid test for Estimating the sensitivity of a rapid test for leishmaniasis leishmaniasis

• Identify persons with leishmaniasis with a gold standard– Parasitologically proven infection

• Obtain a wide panel of samples that are representative of the population of individuals with leishmaniasis– Recent and old cases

– Severe and asymptomatic cases

– Various ages and sexes

• Test the persons with leishmaniasis

• Estimate the proportion of persons with leishmaniasis that are positive with the rapid test

Page 6: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Sensitivity of a rapid test for leishmaniasis Sensitivity of a rapid test for leishmaniasis

Patients with leishmaniasis

Rapid testTrue positive 148

False negative 2

150

Sensitivity = 148 / (150) = 98%95% confidence interval: 94%-99%

Page 7: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

What factors influence What factors influence the sensitivity of a test?the sensitivity of a test?

• Characteristics of the affected persons? YES: Antigenic characteristics of the pathogen in the area

(e.g., if the test was not prepared with antigens reflecting the population of pathogens in the area, it will not pick up infected persons in the area)

• Characteristics of the non-affected persons? NO: The sensitivity is estimated on a population of affected persons

• Prevalence of the disease? NO: The sensitivity is estimated on a population of affected persons

Sensitivity is an INTRINSIC characteristic of the test

Page 8: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

SpecificitySpecificity

Specificity = True negatives / Non-affected personsEstimate the 95% confidence interval

Persons testing negative(True negatives)

Persons testing positive(False positives)

Non-affected persons(Negative by gold standard)

The specificity of a test in the ability of the test to identify correctly non-affected individuals

Proportion of person testing negative among non affected individuals

Page 9: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Estimating the specificity of a testEstimating the specificity of a test

• Identify non affected individuals – Negative with a gold standard

– Unlikely to be infected

• Obtain a wide panel of samples that are representative of the population of non-affected individuals– Diverse unaffected population: Difficult to find. Ideally, those

individuals that would need to be tested but not infected

• Test the non-affected individuals

• Estimate the proportion of non-affected individuals that are negative with the test

Page 10: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Estimating the specificity of a rapid test for Estimating the specificity of a rapid test for leishmaniasis leishmaniasis

• Identify persons without leishmaniasis– Persons without sign and symptoms of the infection

– Persons at low risk of infection, negative with gold standard

• Obtain a wide panel of samples that are representative of the population of individuals without leishmaniasis– Persons from neighbouring villages having similar characteristics

but with no transmission and no infections

• Test the persons without leishmaniasis

• Estimate the proportion of persons without leishmaniasis that are negative with the rapid test

Page 11: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Specificity of a rapid test for leishmaniasis Specificity of a rapid test for leishmaniasis

Individuals without

leishmaniasis

TestFalse positive 12

True negative 188

200

Specificity = 188 / 200 = 94% 95% confidence interval: 90%-96%

Page 12: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

What factors influence What factors influence the specificity of a test?the specificity of a test?

• Characteristics of the affected persons? NO: The specificity is estimated on a population of non affected

• Characteristics of the non-affected persons? YES: The diversity of antibodies to various other antigens in the

population may affect cross reactivity (e.g., If malaria is endemic, polyclonal hyper gammaglobulinemia may increase the proportion of false positives)

• Prevalence of the disease? NO: The specificity is estimated on a population of non affected

Specificity is an INTRINSIC characteristic of the test

Page 13: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Identifying the cut-off to use with a test on Identifying the cut-off to use with a test on the basis of panel analysis: Ideal case the basis of panel analysis: Ideal case

Cut-off

0

5

10

15

20

25

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Possible values of the test

Num

ber

of

test

s

Sick

Well

Page 14: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Identifying the cut-off to use with a test on Identifying the cut-off to use with a test on the basis of panel analysis: Real case the basis of panel analysis: Real case

Cut-off

0

5

10

15

20

25

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Possible values of the test

Num

ber

of

test

s

Sick

WellTrue

negatives

False negatives

True positives

False positives

Page 15: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

To whom sensitivity and specificity matters To whom sensitivity and specificity matters most?most?

• Look at denominators!– Panels of affected individuals

– Panels of negative individuals

• To laboratory specialists!

Page 16: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Outline Outline

• Intrinsic characteristics of a test– Sensitivity

– Specificity

• Performance of a test in a population– Predictive value of a positive test

– Predictive value of a negative test

Page 17: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

How is the test doing in a real population?How is the test doing in a real population?

Status of persons

Affected Non-affected

TestPositive True + False + A+B

Negative False - True - C+D

A + C B+D A+C+B+D

• The test is now used in a real population

• This population is made of – Affected individuals

– Non-affected individuals

• The proportion of affected individuals is the prevalence

Page 18: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Predictive value of a positive testPredictive value of a positive test

Persons affected(True positives)

Persons not affected(False positives)

Persons testing positive(Positive by test)

Predictive value of a positive test = True positives / Persons testing positive

Estimate the 95% confidence interval

The predictive value of a positive test is the probability that an individual testing positive is truly affected

Proportion of affected persons among those testing positive

Page 19: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Predictive value of a positive testPredictive value of a positive test

Status of persons

Affected Non-affected

TestPositive A B A+B

Negative C D C+D

A + C B+D A+C+B+D

PVP = A / (A+B)This is only valid for the sample of specimens tested

Page 20: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

What factors influence What factors influence the predictive value positive of a test?the predictive value positive of a test?

• Sensitivity? YES: To some extend.

• Specificity? YES: The more the test is specific, the more it will be negative for

non affected persons. Thus, when the test is positive, it is probably truly positive (All non affected were correctly identified as testing negative).

• Prevalence of the disease? YES: Low prevalence: The test will pick up more false positives

YES: High prevalence: The test will pick up more true positives

Page 21: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Predictive value positive of a test according to Predictive value positive of a test according to prevalence and specificityprevalence and specificity

0102030405060708090

100

0 10 20 30 40 50 60 70 80 90 100

Prevalence (%)

PVP % 70%80%90%95%

Specificity

Page 22: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Predictive value of a negative testPredictive value of a negative test

Persons non affected(True negatives)

Persons affected(False negatives)

Persons testing negative(Negative by test)

Predictive value of a negative test = True negatives / Persons testing negative

Estimate the 95% confidence interval

The predictive value of a negative test is the probability that an individual testing negative is truly non-affected

Proportion of non-affected persons among those testing negative

Page 23: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Predictive value of a negative testPredictive value of a negative test

Status of persons

Affected Non-affected

TestPositive A B A+B

Negative C D C+D

A + C B+D A+C+B+D

PVN = D / (C+D)This is only valid for the sample of specimens tested

Page 24: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

What factors influence What factors influence the predictive value negative of a test?the predictive value negative of a test?

• Sensitivity? YES: The more the test is sensitive, the more it captures affected

persons. Thus, when the test is negative, it is probably truly negative (all affected were captured among the positive).

• Specificity? YES: But to a lesser extend.

• Prevalence of the disease? YES: Low prevalence: The test will pick up more true negatives

YES: High prevalence: The test will pick up more false negatives

Page 25: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Predictive value negative of a test according Predictive value negative of a test according to prevalence and sensitivityto prevalence and sensitivity

Sensitivity

0

10

20

30

40

50

60

70

80

90

100

Prevalence (%)

PVN %

70%80%90%95%

Page 26: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Relation between predictive values and Relation between predictive values and (1) sensitivity and (2) specificity (1) sensitivity and (2) specificity

PVP Se.P

Se.P (1 Sp)(1 P)

PVN Sp(1- P)

Sp(1- P) (1 Se).P

Page 27: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Positive (PPV) and negative (NPV) predictive values Positive (PPV) and negative (NPV) predictive values of a test according to the prevalence (95% of a test according to the prevalence (95%

sensitivity and specificity)sensitivity and specificity)

0

20

40

60

80

100

Prevalence (%)

Pre

dic

tive

va

lue

(%

)

PVP

PVN

0 50 75 10025

Page 28: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

To whom predictive values matters most?To whom predictive values matters most?

• Look at denominators!– Persons testing positive

– Persons testing negative

• To clinicians and epidemiologists!

Page 29: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Summary Summary

• Sensitivity and specificity matter to laboratory specialists– Studied on panels of positives and negatives

– Look into the intrinsic characteristics of the test:

• Capacity to pick affected

• Capacity to pick non affected

• Predictive values matter to clinicians and epidemiologists– Studied on homogeneous populations

– Look into the performance of the test in real life:

• What to make of a positive test

• What to make of a negative test

Page 30: Laboratory Training for Field Epidemiologists Sensitivity and specificity Predictive values positive and negative Interpretation of results Sep 2007

E P I D E M I C A L E R T A N D R E S P O N S E Laboratory Training for Field Epidemiologists

Developed by:

The Department of Epidemic and Pandemic Alert and Response of the World Health Organization with the assistance of:

European Program for Field Epidemiology Training

Canadian Field Epidemiology Programme

Thailand Ministry of Health

Institut Pasteur

Interpretation of results