results from the la rapid testing study: what can they tell us about proposed point of care...
TRANSCRIPT
![Page 1: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/1.jpg)
Results from the LA rapid testing study: What can they tell us about proposed point of care
strategies?
Kevin Delaney, MPHDivision of HIV/AIDS Prevention
Behavioral and Clinical Surveillance Branch
Diagnostics Applications Team
![Page 2: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/2.jpg)
Thank You (So Far…)LA County STD
Peter KerndtApurva UniyalStaff at LAGLC/Altamed
DHAP/ODBernie Branson
BCSBSteve EtheridgeDawn GnesdaDuncan MackellarPatrick Sullivan
QSDM Debra Hansen
Serum BankDollene Hemmerlein
Lab BranchDebra CandalTrudy DobbsTom FolksTim GranadeSusan KennedySteve McDougalMichelle Owen
![Page 3: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/3.jpg)
Outline
• Original study methods
• Problematic specimens
• Comparing strategies
• Why prevalence matters
![Page 4: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/4.jpg)
Original Study Objectives• Describe the performance of individual rapid tests
– Sensitivity, specificity and predictive value
• Evaluate the performance of combinations of rapid tests– Particular focus on tests designed for use at point of
care
• Foster discussion of a point of care diagnostic algorithm
![Page 5: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/5.jpg)
• Testing – Specimens from participants were tested with all
6 FDA approved rapid tests– Serum also tested with EIA and Western blot:
• Vironostika HIV-1 Microelisa system • Genetic Systems HIV-1 Western Blot
– Samples of both serum and plasma retained for future testing
• Tested at CDC with both Genprobe Aptima and Biorad HIV1/2 + O EIA
Original Study Methods
![Page 6: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/6.jpg)
• Analysis– Sensitivity, specificity and predictive value for
individual tests and combinations used in POC strategies:
• Performed in Sequence• First two tests performed in Parallel• Second and third must agree to be considered
positive• Proposed strategies 1, 2, 3 and 4
Original Study Methods
![Page 7: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/7.jpg)
Line List of “Multiply Discordant” Specimens (n=8) – Including CDC results
ID LA res
Aptima Biorad OQ-o OQ-b Sp-b Sp-pl UG-b Ug-pl Co-b MS-pl Re-pl
1 N P P X 3 N N N N P N P N
4 N N N P P N N N N P N N
6 N N N N N N P N N N P N
9 N N N N N N N N N N P P
10 P N P X 3 N N N N N P P N
11 P P P X 3 N P P P N N P P
13 P P P X 3 P P P N P N P N
15 P P P X 3 P P P N P N P P
![Page 8: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/8.jpg)
Line List of Western blot IND Specimens (n=6) – Including CDC results
ID Aptima Biorad OQ-o OQ-b Sp-b Sp-pl UG-b Ug-pl Co-b MS-pl Re-pl
16 N N N N N N N N P N
17 N N P P N N P P P P P
18 P P X 3 N N N N N N N N
19 P P X 3 N N N N N N N N
20 N P X 3 P P P P P P P P
21 P P X 3 P P P P P P P P
![Page 9: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/9.jpg)
Comparing Strategies• Bootstrap resampling
– Random selection of individuals with replacement– Random selection of a test combination– 1100 theoretical combinations of test and specimen type– Excluded:
• Combinations of the same test used on the same (n=290) • Or different (n=200) specimen types • Combinations that used the two Clearview tests together (n=152) • Combinations with a CLIA moderate complexity test followed by a
CLIA-waived test (n=232) • Combinations in which Oraquick performed on oral fluid was used
as the second or third test (n=21)
– 51 two-test and 154 three-test potential combinations
![Page 10: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/10.jpg)
Comparing Strategies• Bootstrap resampling
– Includes variability introduced by individual test combinations
• e.g. starting with OMT vs. starting with blood • “bad batch” of tests with low specificity
– and problematic specimens • e.g. likely specimen mix• “specific non-specific” reactions with antigens
– Medians and bootstrap confidence intervals reported
![Page 11: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/11.jpg)
Comparing Strategies
• Diagnostic likelihood– The “relative risk” of being infected given that
you test:• Positive (PLHR) = sensitivity/(1-specificity)• Negative (NLHR) = (1-sensitivity)/specificity • Good test performance PLHR > 10 and NLHR < .1• Combining with prevalence will give Positive
(PLHR) or Negative (NLHR) predictive value
![Page 12: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/12.jpg)
Compare algorithms – Diagnostic Likelihoods = Remember these tests are GOOD!scenario
Nlhr2.5%
Nlhr50%
Nlhr97.5%
Plhr2.5%
Plhr50%
Plhr97.5%
AUC2.5%
AUC50%
AUC97.5%
one test (Strategy 1)
0 0.013288 0.030633 390 777 3870 0.98360 0.99320 0.99949
Two different Rapids (Strategy 2)
0 0.014778 0.036810 3774 Inf. Inf. 0.98193 0.99268 1.00000
Two Oraquicks (Strategy 3)
.004149864 0.017471 0.037450 1385 4226 Inf. 0.98121 0.99119 0.99784
Strategy 4+ (allow +,-,- = not infected)
0 0.014634 0.034216 3341 Inf. Inf. 0.98312 0.99272 1.00000
Parallel screening
0 0.006536 0.024004 3074 Inf. Inf. 0.99018 0.99733 1.00000
3 Tests must agree
0 0.016393 0.038674 Inf. Inf. Inf. 0.98054 0.99171 1.00000
![Page 13: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/13.jpg)
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
Tw oRapid
one test
paraalg
seqalg
tw o oq
w hoseq
0.97
0.985
1
0 0.015 0.03
Tw oRapid
one test
paraalg
seqalg
tw o oq
w hoseq
Sensitivity
1-Specificity
ROC Curves for various POC Algorithms
![Page 14: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/14.jpg)
Compare algorithms – Predictive Value (prevalence ≈ 5%)
scenarioPpv2.5%
Ppv50%
Ppv97.5%
Npv2.5%
Npv50%
Npv97.5%
one test (Strategy 1)
0.95181 0.97566 0.99490 0.99851 0.99925 1.00000
Two different Rapids (Strategy 2)
0.99338 1.00000 1.00000 0.99850 0.99926 1.00000
Two Oraquicks (Strategy 3)
0.98502 0.99548 1.00000 0.99819 0.99909 0.99977
Strategy 4+ (allow +,-,- = not infected)
0.99402 1.00000 1.00000 0.99808 0.99915 1.00000
Parallel screening 0.99260 1.00000 1.00000 0.99876 0.99968 1.00000
3 Tests must agree
1.00000 1.00000 1.00000 0.99809 0.99916 1.00000
![Page 15: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/15.jpg)
Compare algorithms – Further testing; Incorrect results
scenarioInd
2.5%Ind50%
Ind97.5%
Fn2.5%
Fn50%
Fn97.5%
Fp2.5%
Fp50%
Fp97.5%
one test (Strategy 1)
0 0 0 0 3 6 1 5 10
Two different Rapids (Strategy 2)
0 2 10 0 3 6 0 0 1
Two Oraquicks (Strategy 3)
0 1 3 1 4 8 0 1 3
Strategy 4+ (allow +,-,- = not infected)
0 0 0 0 3 7 0 0 1
Parallel screening
0 0 0 0 1 4 0 0 1
3 Tests must agree
1 4 11 0 3 7 0 0 0
Remember, these numbers won’t change for a given number of tests
![Page 16: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/16.jpg)
0.4
0.5
0.6
0.7
0.8
0.9
1.0
p
5% 4% 3% 2% 1% 0%
Prevalence affects predictive value, if you have false positives…
Predictive value positive – Single test
Prevalence
![Page 17: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/17.jpg)
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
p
5% 4% 3% 2% 1% 0%
The percentage of specimens going to the lab (red line) will increase as prevalence goes down
Predictive value positive – Single test
Percentage of tests that will require lab follow-up in strategies 2 and 4
Prevalence
![Page 18: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/18.jpg)
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
p
5% 4% 3% 2% 1% 0%
In the worst case (Positive diagnostic likelihood ~ 400) @ .1% prevalence 70% of specimens would still require laboratory follow-up
Predictive value positive – Single test
Percentage of tests that will require lab follow-up in strategies 2 and 4
Prevalence
![Page 19: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/19.jpg)
Compare algorithms – Further testing; Incorrect results
scenarioInd
2.5%Ind50%
Ind97.5%
Fn2.5%
Fn50%
Fn97.5%
Fp2.5%
Fp50%
Fp97.5%
one test (Strategy 1)
0 0 0 0 3 6 1 5 10
Two different Rapids (Strategy 2)
0 2 10 0 3 6 0 0 1
Two Oraquicks (Strategy 3)
0 1 3 1 4 8 0 1 3
Strategy 4+ (allow +,-,- = not infected)
0 0 0 0 3 7 0 0 1
Parallel screening
0 0 0 0 1 4 0 0 1
3 Tests must agree
1 4 11 0 3 7 0 0 0
But in that case that’s still less than 20 specimens per 4000 tested…
![Page 20: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/20.jpg)
• Specificity of a single rapid test >99%– At low prevalence, the proportion of positive tests that
are false positive increases.– Adding a second test resolved nearly all false positives
correctly– There will likely always be some problematic specimens
no testing strategy can fix
Summary
![Page 21: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/21.jpg)
• All testing strategies perform exceptionally well– In low prevalence settings a POC strategy that resolves
all false positives without lab intervention may be necessary
– In high prevalence settings the sensitivity of the first test and the option to test for acute infection should be considered
Summary
![Page 22: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/22.jpg)
Results – Sensitivity of a single testTest Specimen type True
PosFalse neg
Sensitivity (95% CI)
Oraquick Oral fluid 280 5 98.2 (95.9-99.4)
Whole blood 277 1 99.6 (98.0-100)
Unigold Whole blood 173 5 97.2 (93.6-99.1)Plasma 247 8 96.8 (93.7-98.6)
Complete Whole blood 213 0 100 (98.6-100)
Stat-pak Whole blood 246 2 99.2 (97.1-99.9)
Plasma 246 4 98.4 (95.9-99.6)
Multispot Plasma 252 1 99.6 (97.8-100)
Reveal Plasma 278 3 98.9 (96.8-99.8)
![Page 23: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/23.jpg)
Results – Specificity of a single testTest Specimen type True
NegFalse Pos
Specificity (95% CI)
Oraquick Oral fluid 5473 2 100 (99.9-100 )
Whole blood 5466 1 100 (99.9-100 )
Unigold Whole blood 3591 2 99.9 (99.8-100 )Plasma 4747 2 100 (99.9-100 )
Complete Whole blood 4202 2 99.9 (99.8-100 )
Stat-pak Whole blood 4771 1 100 (99.9-100 )
Plasma 4763 2 100 (99.9-100 )
Multispot Plasma 4835 33 99.3 (99.0-99.5)
Reveal Plasma 5478 6 99.9 (99.8-100 )
![Page 24: Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649dbc5503460f94aae283/html5/thumbnails/24.jpg)
• Location – Two clinics in Los Angeles, CA
• Participants– Persons of unknown serostatus seeking HIV
testing– Persons known to be HIV positive
Original Study Methods