1 risks and benefits of home-use hiv test kits richard forshee, ph.d. u.s. food and drug...
Post on 20-Dec-2015
216 views
TRANSCRIPT
11
Risks and Benefits of Risks and Benefits of Home-Use HIV Test KitsHome-Use HIV Test Kits
Richard Forshee, Ph.D.Richard Forshee, Ph.D. U.S. Food and Drug AdministrationU.S. Food and Drug AdministrationCenter for Biologics Evaluation and Center for Biologics Evaluation and ResearchResearchOffice of Biostatistics and EpidemiologyOffice of Biostatistics and Epidemiology
BLOOD PRODUCTS ADVISORY COMMITTEEBLOOD PRODUCTS ADVISORY COMMITTEE96th Meeting, November 16-17, 200996th Meeting, November 16-17, 2009
Bethesda Marriott HotelBethesda Marriott Hotel5151 Pooks Hill Rd., Bethesda, MD 208145151 Pooks Hill Rd., Bethesda, MD 20814
22
AcknowledgmentsAcknowledgments
Mark Walderhaug, Hong Yang, and Mark Walderhaug, Hong Yang, and Arianna SimonettiArianna SimonettiOffice of Biostatistics and EpidemiologyOffice of Biostatistics and Epidemiology
Elliot Cowan and Hilary HoffmanElliot Cowan and Hilary HoffmanOffice Of Blood Research & ReviewOffice Of Blood Research & Review
Bernard Branson, Arielle Lasry, and Bernard Branson, Arielle Lasry, and Stephanie SansomStephanie SansomCenters for Disease Control and Centers for Disease Control and PreventionPrevention
33
Purpose of Model 1Purpose of Model 1
Design a tool to estimate the public Design a tool to estimate the public health benefits and risks of a home-health benefits and risks of a home-use HIV test kit under different test use HIV test kit under different test characteristicscharacteristics
If benefits and risks could be If benefits and risks could be quantified, it would be possible to quantified, it would be possible to define a region of sensitivity and define a region of sensitivity and specificity where the benefits exceed specificity where the benefits exceed the risksthe risks
44
Purpose of the Model 2Purpose of the Model 2
Benefits and risks of different test Benefits and risks of different test outcomes are not comparable, so outcomes are not comparable, so professional judgment is requiredprofessional judgment is required
The tool will help decision-makers The tool will help decision-makers make informed judgments about make informed judgments about the likely tradeoffsthe likely tradeoffs
We seek the committee’s input on We seek the committee’s input on these issuesthese issues
55
Public Health Benefits Public Health Benefits and Risks Considered by and Risks Considered by ModelModel BenefitsBenefits
– True Positive resultsTrue Positive results– True Negative resultsTrue Negative results
RisksRisks– False Negative resultsFalse Negative results– False Positive resultsFalse Positive results– Failed Tests (invalid results or user error)Failed Tests (invalid results or user error)
Benefits and risks are in the context of Benefits and risks are in the context of persons persons who would not otherwise be who would not otherwise be testedtested
66
Qualitative Summary of Qualitative Summary of Public Health Benefits and Public Health Benefits and Risks of Different Test Risks of Different Test Outcomes for a Home-Use Outcomes for a Home-Use HIV Test KitHIV Test Kit
77
True PositiveTrue Positive
Allows earlier medical intervention and Allows earlier medical intervention and entry into careentry into care
Knowledge of individual HIV status allows Knowledge of individual HIV status allows for behavior modification to prevent HIV for behavior modification to prevent HIV transmission, and allows for partner transmission, and allows for partner notificationnotification
Knowledge of HIV prevalence can allow Knowledge of HIV prevalence can allow for better targeting of public health for better targeting of public health resources as cases come to medical resources as cases come to medical attentionattention
88
True NegativeTrue Negative
Peace of mindPeace of mind Assistance in appropriate Assistance in appropriate
targeting of public health targeting of public health resourcesresources
99
False PositiveFalse Positive
Unnecessary personal anxietyUnnecessary personal anxiety Additional testing requiredAdditional testing required Will people seek additional Will people seek additional
testing?testing?
1010
False NegativeFalse Negative
False reassurance False reassurance Unsuspected transmission of virus Unsuspected transmission of virus
and continued high risk behaviorand continued high risk behavior Delayed medical interventionDelayed medical intervention False negative results may False negative results may
include tests taken during window include tests taken during window periodperiod
1111
Failed Test, Person is Failed Test, Person is HIV+HIV+
Another test performedAnother test performed– Delay in diagnosisDelay in diagnosis– Additional testingAdditional testing
No further testingNo further testing– False reassuranceFalse reassurance– Unsuspected transmission of virus Unsuspected transmission of virus
and continued high risk behaviorand continued high risk behavior– Delayed medical interventionDelayed medical intervention
1212
Failed Test, Person is Failed Test, Person is HIV-HIV-
Another test performedAnother test performed– Possibility for personal anxietyPossibility for personal anxiety– Delay in determining statusDelay in determining status– Additional testingAdditional testing
No additional testingNo additional testing– Personal anxietyPersonal anxiety– Status remains unknownStatus remains unknown
1313
Subpopulations Subpopulations ModeledModeled1.1. Low Risk Heterosexuals (LRH):Low Risk Heterosexuals (LRH):
0 or 1 partner in previous year0 or 1 partner in previous year2.2. High Risk Heterosexuals (HRH): High Risk Heterosexuals (HRH):
previous yearprevious year3.3. Men Who Have Had Sex With Men Men Who Have Had Sex With Men
(MSM): (MSM): previous yearprevious year
4.4. Injectable Drug Users (IDU):Injectable Drug Users (IDU):previous yearprevious year
1414
Time Frame of the Time Frame of the ModelModel
Model estimates annual rates for 2nd Model estimates annual rates for 2nd and succeeding years after and succeeding years after introduction by considering currently introduction by considering currently active at-risk group (with risky active at-risk group (with risky behaviors in past 12 months) as behaviors in past 12 months) as regular users of the test kitregular users of the test kit
Use of test kit might be higher or lower Use of test kit might be higher or lower in the 1st year after introductionin the 1st year after introduction
1515
Global InputsGlobal Inputs
1.1. Percent of test kits that fail (5%)Percent of test kits that fail (5%)
2.2. Sensitivity of test kits that did NOT failSensitivity of test kits that did NOT fail
3.3. Specificity of test kits that did NOT failSpecificity of test kits that did NOT fail
These are treated as hypothetical and These are treated as hypothetical and will be estimated across a range of will be estimated across a range of plausible values. Sensitivity does not plausible values. Sensitivity does not include window period cases.include window period cases.
1616
Sensitivity Values Sensitivity Values ModeledModeled
MeanMean
99.5%99.5%97.0%97.0%95.0%95.0%92.0%92.0%90.0%90.0%
1717
Specificity Values Specificity Values ModeledModeled
MeanMean
99.5%99.5%99.0%99.0%98.5%98.5%98.0%98.0%97.5%97.5%
1818
Sub-population Sub-population Specific InputsSpecific Inputs
1.1. Size of sub-populationSize of sub-population2.2. Percent of sub-population that is Percent of sub-population that is
untested (previous year)untested (previous year)3.3. Percent of untested persons who Percent of untested persons who
are HIV+are HIV+4.4. Percent of untested persons who Percent of untested persons who
would use a home-use HIV test would use a home-use HIV test (highly uncertain)(highly uncertain)
1919
Sub-population Sub-population InputsInputs
Presenting means onlyPresenting means only
Uncertainty was included in Uncertainty was included in modelmodel
2020
Sub-population SizeSub-population Size
3,840,000
1,580,000
16,200,000
167,200,000
MSM
IDU
HRH
LRH
Sub
-pop
ulat
ion
0 50,000,000 100,000,000 150,000,000 200,000,000Size of sub-population
2121
Percent UntestedPercent Untested
30%
31%
65%
65%
MSM
IDU
HRH
LRH
Sub
-pop
ulat
ion
0 25% 50% 75% 100%Percent untested
2222
Percent Untested HIV+Percent Untested HIV+
8%
19%
2.5%
0.03%
MSM
IDU
HRH
LRH
Sub
-pop
ulat
ion
0 5% 10% 15% 20%Percent of untested who are HIV+
2323
Percent Untested Who Percent Untested Who Would Use a Home-Use Test Would Use a Home-Use Test KitKit
50%
40%
10%
5%
MSM
IDU
HRH
LRH
Sub
-pop
ulat
ion
0 10% 20% 30% 40% 50% 60%Percent of untested who would use a home use test
2424
Structure of the Model Structure of the Model 11
Size of the Sub-population
Number Untested
HIV + HIV -
% Untested
% HIV +
2525
Structure of the Model Structure of the Model 22
HIV +
Number Using Test
False Negative True Positive
Failed Test
% Failed
Sensitivity
% Using Test
2626
Model ResultsModel Results
2727
Absolute Numbers:Absolute Numbers:Model ResultsModel Results
Totals for all sub-populationsTotals for all sub-populations
Persons who would not otherwise be Persons who would not otherwise be testedtested
2828
Test SensitivityTest Sensitivity
105,279
100,520
5,565
527
5,286
True+, 99.5%
True+, 95.0%
Failed Tests
False-, 99.5%
False-, 95.0%
0 50,000 100,000 150,000Number of test results
Mean 5th to 95th %tile
Predicted Number of Test Results99.5% vs 95.0% Sensitivity
2929
Test SpecificityTest Specificity
6,774,414
6,706,370
358,120
33,997
102,040
True-, 99.5%
True-, 98.0%
Failed Tests
False+, 99.5%
False+, 98.0%
0 2,000,000 4,000,000 6,000,000 8,000,000 10,000,000Number of test results
Mean 5th to 95th %tile
Predicted Number of Test Results99.5% vs 98.0% Specificity
3030
Using Ratios to Explore Using Ratios to Explore TradeoffsTradeoffs
As a surrogate for quantitative As a surrogate for quantitative estimates of benefit and risk, ratios of estimates of benefit and risk, ratios of benefits to risks can provide some benefits to risks can provide some insight into public health tradeoffsinsight into public health tradeoffs
We will explore the number of We will explore the number of beneficial test outcomes to the number beneficial test outcomes to the number of adverse test outcomesof adverse test outcomes
There is no accepted threshold value There is no accepted threshold value for any of these ratiosfor any of these ratios
Professional judgment is required to Professional judgment is required to determine what ratio is acceptabledetermine what ratio is acceptable
3131
True Positive to False True Positive to False NegativeNegative
The ratio indicates how many newly The ratio indicates how many newly identified True Positive results are identified True Positive results are obtained for each False Negative resultobtained for each False Negative result
True Positive and False Negative results True Positive and False Negative results are related solely to the sensitivity of the are related solely to the sensitivity of the test and the number of HIV+ persons test and the number of HIV+ persons taking the testtaking the test
Failed tests for HIV+ persons are counted Failed tests for HIV+ persons are counted as False Negative results in this as False Negative results in this calculationcalculation
3232
Tradeoff between True Positive Test Tradeoff between True Positive Test Results and False Negative Test Results and False Negative Test ResultsResults
Sensitivity Ratio
99.5 20
97 13
95 10
92 7
90 6BPAC suggested that the lower bound
of the 95% CI should be no less than 95%010
2030
40R
atio
90 92 94 96 98 100
Sensitivity
5th Percentile/95th Percentile Mean Ratio
Ratio of True Positive to False Negativeand Failed HIV+ Results
3333
True Negative to False True Negative to False PositivePositive
The ratio indicates how many True The ratio indicates how many True Negative results are obtained for each Negative results are obtained for each False Positive False Positive
True Negative and False Positive results True Negative and False Positive results are related solely to the specificity of the are related solely to the specificity of the test and the number of HIV- persons test and the number of HIV- persons taking the testtaking the test
Failed tests for HIV- persons are counted Failed tests for HIV- persons are counted as False Positive results in this calculationas False Positive results in this calculation
3434
Tradeoff between True Negative Tradeoff between True Negative Test Results and False Positive Test Test Results and False Positive Test ResultsResults
1015
2025
3035
Rat
io
97.5 98 98.5 99 99.5
Specificity
5th Percentile/95th Percentile Mean Ratio
Ratio of True Negative to False Positiveand Failed HIV- Results
Specificity Ratio
99.5 20
99.0 18
98.5 16
98.0 15
97.5 13
3535
True Positive to False True Positive to False PositivePositive
True Positive and False Positive True Positive and False Positive results are related to both the results are related to both the sensitivity and the specificity of the sensitivity and the specificity of the test as well as the number of HIV+ test as well as the number of HIV+ and HIV- persons taking the testand HIV- persons taking the test
The ratio indicates how many True The ratio indicates how many True Positive results are obtained for Positive results are obtained for each False Positiveeach False Positive
3636
True Positive to False True Positive to False Positive RatioPositive Ratio
99.5 97.0 95.0 92.0 90.0
99.5 3.10 3.02 2.96 2.86 2.80
99.0 1.55 1.51 1.48 1.43 1.40
98.5 1.03 1.01 0.99 0.95 0.93
98.0 0.77 0.75 0.74 0.72 0.70
97.5 0.62 0.60 0.59 0.57 0.56
Specificity
Sensitivity
Under current assumptions, the ratio of true positive to false positive test results is more strongly related to specificity than to sensitivity.
3737
Current BPAC Recommended Current BPAC Recommended PerformancePerformance
The lower bound of the 95% The lower bound of the 95% confidence interval for sensitivity and confidence interval for sensitivity and specificity must be greater than or specificity must be greater than or equal to 95%equal to 95%
What is the mean sensitivity and What is the mean sensitivity and specificity needed to meet this?specificity needed to meet this?
Assume a 2% prevalence and a study Assume a 2% prevalence and a study population consisting of 100 positive population consisting of 100 positive subjects and 4,900 negative subjectssubjects and 4,900 negative subjects
3838
Test Test SensitivitySensitivity to Meet BPAC to Meet BPAC Recommended Lower Bound of the Recommended Lower Bound of the 95% CI95% CI
SuccessSuccess FailureFailure MeanMean Lower Lower BoundBound
9999 11 99%99% 97.0%97.0%9898 22 98%98% 95.2%95.2%9797 33 97%97% 93.6
%
3939
Test Test SensitivitySensitivity to Meet BPAC to Meet BPAC Recommended Lower Bound of the Recommended Lower Bound of the 95% CI95% CI
SuccessSuccess FailureFailure MeanMean Lower Lower BoundBound
46554655 245245 95.095.0 94.494.4
46804680 220220 95.595.5 94.994.946844684 216216 95.695.6 95.095.047044704 196196 96.096.0 95.495.4
40
Estimated ratio of true + to false + test results as a function of sensitivity and specificity of tests. Colors represent a temperature map. Assumes 111,000 HIV+ test users and 7,200,000 HIV- test users (mean values from the simulation). Specificity has a strong relationship with the ratio.
Ratio of True+ to False+ by Characteristics
Typical Currently Approved Rapid Tests
BPAC Minimum(see assumptions)
7.6
0.34
Low Sensitivity,Low Specificity
High Sensitivity,High Specificity
41
Ratio of True+ and False+ by Test Characteristics.
Estimated ratio of true + to false + test results as a function of sensitivity and specificity of tests. Colors represent a temperature map. Assumes 126,000 HIV+ test users and 6,000,000 HIV- test users (75 and 25 percentile values from the simulation). Specificity has a strong relationship with the ratio.
Typical Currently Approved Rapid Tests
BPAC Minimum(see assumptions)
More HIV+ and Fewer HIV- Users
10.4
0.47
4242
SummarySummary
We have presented a tool to explore the We have presented a tool to explore the public health benefits and risks of a home-public health benefits and risks of a home-use HIV test kit with different test use HIV test kit with different test characteristicscharacteristics
Tool provides estimates of the absolute Tool provides estimates of the absolute numbers of each type of test result and numbers of each type of test result and three ratios to assist in making informed three ratios to assist in making informed judgments about public health tradeoffsjudgments about public health tradeoffs
Each model result is important for Each model result is important for determining the overall public health determining the overall public health impactimpact
4343
Discussion 1Discussion 1
Several of the input parameters have Several of the input parameters have significant uncertainty because of a lack significant uncertainty because of a lack of dataof data
Our estimates that a large number of Our estimates that a large number of HIV- persons will use the test implies HIV- persons will use the test implies that high specificity will be required to that high specificity will be required to minimize False+ resultsminimize False+ results
Ratio of True + to False + is strongly Ratio of True + to False + is strongly related to specificity of the test under related to specificity of the test under current assumptionscurrent assumptions
Ratio of True+ to False- is related to Ratio of True+ to False- is related to sensitivity of the testsensitivity of the test
4444
Discussion 2Discussion 2
Assessing the public health impact of Assessing the public health impact of each test outcome requires a value each test outcome requires a value judgmentjudgment
We seek the committee’s input We seek the committee’s input regarding the appropriate balance regarding the appropriate balance between different test outcomes, e.g. between different test outcomes, e.g. how many False- test results would be how many False- test results would be acceptable for each newly identified acceptable for each newly identified True+ result?True+ result?
4545
Thank you!Thank you!
Richard Forshee, Ph.D.Richard Forshee, Ph.D.FDA/CBERFDA/CBER
[email protected]@fda.hhs.gov
4646
Reserve SlidesReserve Slides
4747
Sub-population SizeSub-population Size
3,840,000
1,580,000
16,200,000
167,200,000
MSM
IDU
HRH
LRH
Sub
-pop
ulat
ion
0 50,000,000 100,000,000 150,000,000 200,000,000Size of sub-population
Anderson et al. 2009
Brady 2008
National Survey of Family Growth, various criteria
National Survey of Family Growth
4848
Percent UntestedPercent Untested
30%
31%
65%
65%
MSM
IDU
HRH
LRH
Sub
-pop
ulat
ion
0 25% 50% 75% 100%Percent untested
Assumption
Heimer 2007
National Health Interview Survey
Jenness 2009
4949
Percent Untested HIV+Percent Untested HIV+
8%
19%
2.5%
0.03%
MSM
IDU
HRH
LRH
Sub
-pop
ulat
ion
0 5% 10% 15% 20%Percent of untested who are HIV+
Entering drug treatment, CDC website
Anderson et al. 2009
STD clinics, CDC website
Testing of military applicants, CDC website
5050
Percent Untested Who Percent Untested Who Would Use a Home-Use Test Would Use a Home-Use Test KitKit
50%
40%
10%
5%
MSM
IDU
HRH
LRH
Sub
-pop
ulat
ion
0 10% 20% 30% 40% 50% 60%Percent of untested who would use a home use test
Assumption
Assumption
Assumption
Assumption