07-qualitative semi quant
TRANSCRIPT
-
8/3/2019 07-Qualitative Semi Quant
1/20
Copyright 2002-2010, Data Innovations, LLC. All rights reserved world-wide.This document may not be reproduced without express written permission.
Qualitative and Semi-quantitativeModule (QMC)
By Carol Lee
-
8/3/2019 07-Qualitative Semi Quant
2/20
2
Method Comparison Qual/SemiQuant
Purpose To evaluate the degree of concordance between two
methods for qualitative or semi-quantitative methods.
Principle
Qualitative methods and semi-quantitative methods lacksufficient analytical sensitivity to demonstrate accuracyacross a wide range of observed results. Therefore:
The results are categorized in states of diagnostic
description. i.e.: postive / negative, equivocal, gray zone,
Qualitative 2 states (i.e. positive and negative)
Semi-quantitative a limited number of states (up to 6are allowed in EP Evaluator).
-
8/3/2019 07-Qualitative Semi Quant
3/20
3
QMC
Evaluate the degree of concordance between two qualitative orsemi-quantitative methods.
colorful bubble chart size of bubbles proportional to number of points
concordance or truth tables
Qualitative 2 states (i.e. positive and negative)
% pos agreement, % neg agreement False pos and false neg in red
If one method is gold standard
Sensitivity and specificity is calculated
Semi-quantitative up to 6 states allowed in EP Evaluator Click custom codes to set up states
Alphanumeric i.e., Equivocal, gray zone
Numeric cutoff values
< cutoff, cutoff
2002-2008 David G. Rhoads Associates, Inc
-
8/3/2019 07-Qualitative Semi Quant
4/20
4
Experimental Design
Qualitative: At least 20 specimens.
Equal numbers of positives and negatives
Semi quantitative Same number of states for both methods
User defined results or cutoffs and labels
0 to 10, few,
Try for similar N in each state.
-
8/3/2019 07-Qualitative Semi Quant
5/20
5
Method Comparison Qualitative
Enter values as 2 states User defined labels
Calculations
If One method is specified as a gold standard (i.e.
absolutely correct). calculates specificity and sensitivity
If Neither method is a gold standard.
Only calculates relationship of the two methods, namely
the degree of agreement and whether one method ismore or less sensitive relative to the other.
-
8/3/2019 07-Qualitative Semi Quant
6/20
Qualitative
Default is 2 states
P = positive
N = Negative
Allowed results showin the drop down boxin the data entry area
-
8/3/2019 07-Qualitative Semi Quant
7/20
Gold Standard Reference
-
8/3/2019 07-Qualitative Semi Quant
8/20
Enter 2state
resultsGold
standard
-
8/3/2019 07-Qualitative Semi Quant
9/20
Not Gold Standard
-
8/3/2019 07-Qualitative Semi Quant
10/20
Report Excerpt
Qualitative Method ComparisonRef. Method: Immunochromatic Test Method: NCCLSEx2
neg pos1 2
1
2
Kappa is the proportion of agreement above what's expected bychance. Rule of thumb is Kappa>75% indicates "high" agreement.We would like to see VERY high (close to 100%) agreement.
Statistical Analysis
Agreement 94.6% (92.3 to 96.2%)
Positive Agreement 95.3%Negative Agreement 93.7%95% confidence intervals calculated by the "Score" method.
McNemar Test for Symmetry:Test < Reference 14 (2.6%)Test > Reference 15 (2.8%)Symmetry test PASSES p = 0.853 (ChiSq=0.034, 1 df)A value of p
-
8/3/2019 07-Qualitative Semi Quant
11/20
Concordance table
-
8/3/2019 07-Qualitative Semi Quant
12/20
Statistical tests
Cohen's Kappa: Similar to Agreement, but adjusted for theprobability that the two methods agree by chance. Kapparanges from -100% to 100%. A value of zero indicatesrandom agreement. A value of 100% indicates perfectagreement. It is desirable for Kappa to be well above 75%.
McNemar Test for Symmetry: A test for bias -- whether onemethod is consistently larger than the other. If the number ofcases where X>Y is equal (within random error) to thenumber of cases XY (or when X
-
8/3/2019 07-Qualitative Semi Quant
13/20
Semi quantitative
Ref. Method: Chem Assay Test Method: Analyzer
neg pos1 2 3 4 5 6
1
2
3
4
5
6
Kappa is the proportion of agreement above what's expected bychance. Rule of thumb is Kappa>75% indicates "high" agreement.We would like to see VERY high (close to 100%) agreement.
Statistical Analysis
Agreement 71.9% (61.8 to 80.2%)95% confidence intervals calculated by the "Score" method.
McNemar Test for Symmetry:Test < Reference 23 (25.8%)Test > Reference 2 (2.2%)Symmetry test FAILS p < 0.001 (ChiSq=17.640, 1 df)A value of p
-
8/3/2019 07-Qualitative Semi Quant
14/20
Allow 1 step difference
-
8/3/2019 07-Qualitative Semi Quant
15/20
Allow 1 step
Ref. Method: Chem Assay Test Method: Analyzer
neg pos1 2 3 4 5 6
1
2
3
4
5
6
Kappa is the proportion of agreement above what's expected bychance. Rule of thumb is Kappa>75% indicates "high" agreement.We would like to see VERY high (close to 100%) agreement.
Statistical Analysis
Agreement 71.9% (61.8 to 80.2%)
Agreement within two 98.9% (93.9 to 99.8%)95% confidence intervals calculated by the "Score" method.
McNemar Test for Symmetry:Test < Reference 23 (25.8%)Test > Reference 2 (2.2%)
Symmetry test FAILS p < 0.001 (ChiSq=17.640, 1 df)A value of p
-
8/3/2019 07-Qualitative Semi Quant
16/20
Custom results codesCutoff values Cutoff value > than
the number entered
-
8/3/2019 07-Qualitative Semi Quant
17/20
Policy Definitions: Editing Analytesettings Your choices in Modules
and options will define whatheaders are visible in theanalyte settings
If you dont select an sdgoal in SP, the column forrandom err budget will notbe present.
If you check QMC, a
separate tab appears forQMC
17Jan 6, 2010
-
8/3/2019 07-Qualitative Semi Quant
18/20
18
QMC entering analyte parameters
edit paste headers into excel
Make changes
Copy / paste from excel back to EE9
18Jan 6, 2010
Analyt
e
Custom
?
Result
sType
LvlN
ame
1
LvlVal
ue1
LvlNa
me2
LvlVal
ue2
LvlNa
me3
LvlVal
ue3
LvlNa
me4
LvlVal
ue4
LvlNa
me5
LvlVal
ue5
LvlNa
me6
LvlVal
ue6
BACT y 1
Nor
m Abn 1200
fuzzy
cells y 1
Nor
m Abn 1
-
8/3/2019 07-Qualitative Semi Quant
19/20
QMC analyte settingspolicy definitions
19Jan 6, 2010
0 = alpha
1 = numeric large is pos2 = numeric large is neg
-
8/3/2019 07-Qualitative Semi Quant
20/20
20
Questions and Discussion
2002-2008 David G. Rhoads Associates, Inc 20