11

Calibration,

Calibration Verification,

Analytical Measurement Range,

and Clinically Reportable Range

For the CAP Instrumentation and Chemistry Resource Committees

22

The Truth, Please

• What’s your response when• You receive the results of a CAP Linearity Survey?

• You come to Checklist Question CHM.13400 (one of several relating to Calibration Verification)?

33

The Truth, Please

• What’s your response when• You receive the results of a CAP Linearity Survey?

• You come to Checklist Question CHM.13400 (one of several relating to Calibration Verification)?

• If you’re anything like me, it’s not “Wow, let me at it. I was really looking forward to taking care of this!”

• Who I Am:Gary L. Horowitz, MD, FCAPDirector, Clinical Chemistry, Beth Israel Deaconess Medical CenterAssociate Professor of Pathology, Harvard Medical SchoolChair, CAP Chemistry Resource Committee

44

Terms

Calibration

55

Terms that Instill Fear

Calibration

Calibration Verification

Analytical Measurement Range (AMR)

Clinically Reportable Range (CRR)

66

Terms that Instill Fear

Calibration

Calibration Verification

Analytical Measurement Range (AMR)

Clinically Reportable Range (CRR)

CAP Linearity Surveys

77

Why We’re Here Today• confusing, but important, concepts

• audioconference format: allows anyone to visit the web as often as needed

• objectives:• explain the concepts• reveal how they relate to the accreditation process• show you in detail how the CAP Linearity Surveys enable you to

address these issues for your laboratory

• ambitious goals, but we have two excellent speakers, experts• in Laboratory Medicine• in CAP Surveys

88

Introductions

• Greg Miller, PhD Professor of Pathology, Virginia Commonwealth University Consultant Member, CAP Chemistry Resource Committee

99

Introductions

• Greg Miller, PhD Professor of Pathology, Virginia Commonwealth University Consultant Member, CAP Chemistry Resource Committee

• Anthony Killeen, MD, PhD, FCAP Associate Professor of Pathology, University of Minnesota Member, CAP Instrumentation Resource Committee Former Member and Chair, CAP Chemistry Resource Committee

• Program developed as joint effort of these two CAP Resource Committees

1010

Objectives

• Differentiate between the terms calibration, calibration verification, analytical measurement range (AMR) and clinically reportable range (CRR).

• Clarify the accreditation requirements regarding these items.

• Explain techniques to perform the appropriate verifications.

• Clarify CAP Linearity (LN) Survey reports.

1111

Reportable range: span of test result values over which the lab can establish or verify the accuracy of the measurement response.

CLIA 493.2

1212

Analytical measurement range: range of analyte values that a method can directly measure on the specimen without any dilution, or other pretreatment not part of the usual assay process.

Reportable range: span of test result values over which the lab can establish or verify the accuracy of the measurement response.

CAP (two concepts) CLIA 493.2

1313

Clinically reportable range: range of analyte values that are reported as a quantitative result, allowing for specimen dilution or other pretreatment used to extend the actual AMR.

Reportable range: span of test result values over which the lab can establish or verify the accuracy of the measurement response.

CAP (two concepts) CLIA 493.2

1414

CAP CLIA

Calibration: set of operations that establish the quantitative relationship between the reagent system / instrument response and the concentration or activity values of an analyte.

Calibration: process of testing and adjusting to establish a correlation between the measurement response and the concentration.

1515

Calibration verification: assaying materials of known concentration to substantiate the system’s calibration throughout the reportable range.

CAP (two concepts) CLIA 493.2

1616

Calibration verification: assaying materials of known concentration to substantiate the system’s calibration throughout the reportable range.

Calibration verification: process of confirming that the current calibration settings remain valid for a method.

CAP (two concepts) CLIA 493.2

1717

Calibration verification: assaying materials of known concentration to substantiate the system’s calibration throughout the reportable range.

Analytical measurement range verification: process of confirming that the assay system will correctly recover the concentration or activity of the analyte over the AMR.

CAP (two concepts) CLIA 493.2

1818

CLIA 493.1255: Calibration and calibration verification procedures

(a) Perform and document calibration procedures.

1. Follow manufacturer’s instructions.

2. Use criteria established by lab during validation of method when put in service.

3. Whenever calibration verification fails to meet the lab’s specifications.

1919

CLIA 493.1255: Calibration and calibration verification procedures

(b) Perform and document calibration verification procedures.1. Follow manufacturer’s instructions2. Use criteria established by lab during validation of the

method when put in service Include minimum, mid-point and maximum value to verify the

laboratory’s reportable range (this is AMR) 3. At least every 6 months or when the following occur:

• Change of reagents unless it does not affect the range to report patient results (this is AMR), or QC values (this is CAL)

• Major preventive maintenance (this is CAL)• QC trend, shift or out of limits (this is CAL)• Schedule for verifying reportable range (this is AMR) requires

more frequent calibration verification

2020

Calibration (two point)

SIG

NA

LAnalytical Measurement Range extends beyond the high calibrator

CONCENTRATION

CALIBRATOR TARGET VALUES • May be measurement system specific

= Reportable Results

2121

Calibration (two point)

Example: typical serum glucose method• Low calibrator = reagent blank (0 mg/dL)

• High calibrator = 356 mg/dL

• AMR = 10-600 mg/dL

• The AMR is based on the expectation that the signal is linearly related to glucose concentration over the 10-600 mg/dL range.

• Verification of the AMR expectation is required because the calibrators do not define the complete measurement range.

2222

Calibration (multi-point)

CALIBRATOR TARGET VALUES

SIG

NA

L = Reportable Results

Note that when calibrators span the AMR, the calibration is fully defined over the AMR

• May be measurement system specific

Analytical Measurement Range

CONCENTRATION

2323

Accuracy (Trueness) Traceability

Manufacturer provides calibration traceability to the highest order reference method or reference material available

• Manufacturer’s product calibrator has method (and sometimes reagent lot) specific target values

• Cannot mix reagents and calibrators from different manufacturers

2424

Terminology

Trueness: the mean bias between a routine method and a reference method determined from replicate measurements of multiple patients’ samples

• Imprecision is removed (or minimized) by replication

Accuracy: the difference between a routine method and a reference method for a single measurement on a single patient’s sample

• Difference includes influence of both bias and imprecision for a single measurement

Refer to the CLSI harmonized terminology database for more detailed definitions at www.clsi.org/resources

2525

Calibration Verification• Verify that measurement system calibration has not

changed since the last calibration event

And, where applicable (many routine methods),

• Verify that method calibration is in conformance to the manufacturer’s specifications

• (i.e. is consistent with the manufacturer’s calibration traceability)

• Special case (uncommon) when a lab can verify calibration traceability to an accuracy based reference system

2626

Commutable Reference/ PT/ QC Material

• Has equivalent mathematical relationships among the results of different measurement procedures for a reference material and for native clinical samples (International Standards

Organization, Clinical and Laboratory Standards Institute derived).

• Produces a numeric result from a routine method that is the same as would be obtained for a native clinical sample that has the same quantity of an analyte (measurand) present.

2727

Non-commutable materials may have a matrix bias

• Bias in a result that is attributable to modification of the usual clinical sample matrix caused by preparation of a reference material, PT material or QC material

• CAP has investigated matrix bias in PT materials using fresh frozen off-the-clot samples (that are expected to be commutable with clinical samples)

CAP Home (www.cap.org) > CAP Reference Resources and Publications > Chemistry

• CAP has developed accuracy based PT Surveys from this research

2828

CLSI EP14 to validate commutability

0

2

4

6

8

10

0 2 4 6 8 10

Comparison Method (1 RMP if available)

Tes

t M

eth

od

Clinical Specimen

95% prediction interval

2929

CLSI EP14 to validate commutability

0

2

4

6

8

10

0 2 4 6 8 10

Comparison Method (1 RMP if available)

Tes

t M

eth

od

Clinical SpecimenRM Commutable

95% prediction interval

3030

CLSI EP14 to validate commutability

0

2

4

6

8

10

0 2 4 6 8 10

Comparison Method (1 RMP if available)

Tes

t M

eth

od

Clinical SpecimenRM CommutableRM Not-Commutable

95% prediction interval

31

Effect of non-Commutable PT materials

150

200

250

300

350

du

Po

nt

Dim

ensi

on

C

ho

les

tero

l (m

g/d

L)

150 200 250 300 350Abell-Kendall Cholesterol (mg/dL)

CAP SurveyPatient serum

A dapt ed from: Naito, et. al., Arch Pathol Lab Med. 1993;117:345-51

PT bias = -9.5%

Patient bias = 0.2%

Reference Method

3232

CAP Accuracy Based Surveys

• Glycohemoglobin (GH2) – uses fresh whole blood with National Glycohemoglobin Standardization Program target values

• Creatinine Accuracy Calibration Verification and Linearity (LN24) – uses off-the-clot serum with NIST target values

• Accuracy Based Lipids (ABL) – uses off-the-clot serum with CDC reference method target values

• Additional accuracy based Surveys are in development for analytes with clinical practice guidelines

3333

Calibration Verification

Run assay materials with measurement system and compare results to measurement system appropriate target values

Result = Target ?

Calibration is OKRe-calibrate

YESNO

3434

Calibration Verification Materials

• Method product calibrator(s)

• Method vendor provided materials for calibration verification

• Previously tested clinical specimens

• Reference materials with matrix and target values appropriate for the method

• PT (or PT validated) materials with matrix and target values appropriate for the method, including materials designed for calibration verification

3535

Verification of Calibration Trueness (or accuracy for individual samples)

• Requires a reference material that is commutable with native clinical samples (not commonly available)

• Most PT materials are not commutable due to the cost to prepare commutable materials and the practical need to have many analytes in the same material

3636

LN Surveys for Calibration Verification

Lab verifies conformance to the manufacturer’s calibration, using:

Commutable reference materials that have been validated to give results equivalent to those for native clinical samples (e.g. CAP LN 24 Creatinine accuracy based Survey with NIST target values)

Non-commutable materials with target values appropriate for a specific method (e.g. most CAP LN Survey materials with peer group evaluation)

3737

Calibration Verification Concentration Levels

• Approximately the same as the calibrators used for method calibration

• i.e. to confirm correct calibration

• Can also verify the AMR if concentrations, or activities, of the calibrators cover the full measurement range.

• LN Survey provides a series of defined concentrations to verify calibration over the AMR

3838

Frequency of Calibration Verification

• Manufacturer’s instructions

• Troubleshoot a QC problem

• Calibration OR calibration verification at least every 6 months (CLIA), or when:

• Change in critical reagents, component, or maintenance

• If cal <6 months do not need calibration verification unless part of manufacturer’s calibration process

3939

Analytical Measurement Range VerificationC

ON

CE

NT

RA

TIO

NR

EC

OV

ER

ED

CONCENTRATION EXPECTEDBased on: (a) admixture ratio (b) assigned value

Specimen concentration relationship is linear over the AMR

Non-linear region

4040

Method Calibration and AMR Verification can be a single process

An

alyt

ical

sig

nal

=

rep

ort

able

val

ues

Method Calibrator

ANALYTICAL MEASUREMENT RANGE

Calibrator must:

1. Have at least 3 values that span the AMR, and

2. Have target values specific for the method (and reagent lot, if necessary).

No separate verification of AMR is necessary

4141

AMR Verification

1. Specimens (can use patients) with low and high values2. Mix 1:1 to get midpoint value3. Mix low:mid and mid:high to get additional values4. Expected values are:

Sample Proportion Concentration expectedLow 0% 20 mg/dL0.75 L + 0.25 H 25% 160 mg/dL0.50 L + 0.50 H 50% 299 mg/dL0.25 L + 0.75 H 75% 438 mg/dLHigh 100% 578 mg/dL

Criteria for acceptability = recovery of concentration in proportion to the amount present (linear relationship)

42

AMR Verification

• AMR verification can be based on demonstrating linearity either by measuring recoveries or by demonstrating expected proportions

42

4343

AMR Verification

0

200

400

600

0 200 400 600

Expected mg/dL

Mea

sure

d m

g/d

L

0

200

400

600

0 50 100

Expected ProportionM

ea

su

red

mg

/dL

Linear relationship verifies recovery of correct concentration relationship over AMR.

4444

Materials for AMR Verification

• Linearity material of appropriate matrix

• PT (or PT validated) materials

• Previously tested clinical specimens

• Previously tested clinical specimens, altered by admixture, dilution, spiking

• Reference materials with matrix and target values appropriate for the method

• Method product calibrator(s)

4545

AMR: how close to low and high limits

• For high limit: within 10-15% of upper limit of AMR

• Challenging to find residual samples for some analytes; need to use clinical judgment, e.g.

• blood gas (use commercial materials)

• cardiac markers, e.g. Troponin (use LN Survey)

• Goal is to verify acceptable clinical performance

• For low limit: use clinical judgment, e.g.

• creatinine, 0.2 or 0.3 mg/dL

4646

CAP LN Surveys• Most evaluate performance vs. peer group due to

commutability limitations (matrix effect bias)

• LN24 offers matrix-free creatinine challenges with reference method values assigned

• Verify that calibration conforms to manufacturer’s specifications

• Verify that calibration is correct over the analytical measurement range

• When accuracy based (e.g., LN24), verify that calibration is traceable to a reference system

47

Example of LN2 Report (Calcium)

47

48

Example of LN2 Report (Calcium)

48

49

Example of LN24 Report (Creatinine)

49

50

Example of LN24 Report (Creatinine)

50

51

Example of a Non-Linear Result

51

Creatinine

52

Calibration Verification and Linearity Users Guide

52

5353

Clinically Reportable Range

• Range of analyte values that will be reported as a quantitative result, allowing for specimen dilution or other pretreatment to extend the AMR

• Policy based on the laboratory director’s medical judgment regarding the clinical use requirements for a laboratory test

5454

Clinically Reportable Range

• Truncation or extension of the AMR based on clinical requirements.

Established once when method is put in service

Does not need to be verified on a recurring schedule

• Values outside the CRR are reported as < or > a numeric value.

• Checklist question is: Are dilution protocols and diluents (or concentration protocols) specified for all methods for which the CRR exceeds the AMR?

5555

Clinically Reportable Range

Example: hCG may have a CRR with both low and high limits.

• AMR is 3-1,000 mIU/mL (method specification).

• CRR is 5-1,000,000 mIU/mL (clinical usefulness).

• Results of <3, 3, or 4 are reported as “<5 mIU/mL”.

• Results >1,000 are diluted and rerun to obtain quantitative values up to 1,000,000 mIU/mL.

• Results >1,000,000 are reported as “>1,000,000 mIU/mL”.

• NOTE: CRR limits are the judgment of the lab director and may be different for different labs

5656

Clinically Reportable Range

Example: AST may have a CRR with only a low limit.

• AMR is 4-900 IU/L (method specification).

• CRR is 4 IU/L to any value (clinical usefulness).

• Results of <4 are reported as “<4 IU/L”.

• Results >900 are diluted and re-assayed.

• The upper CRR is not specified because specimens are diluted until a quantitative value is obtained.

5757

Summary

Calibration or Calibration Verification at least every 6 months.

Calibration verification is not necessary if calibration is performed at least every 6 months.

5858

Summary

Analytical Measurement Range verification at least every 6 months.

May be required when change a critical reagent lot or other analytical parameter that can influence the AMR.

Not required when a recalibration is performed.

Not required when method calibrators include at least 3 concentrations that span the full AMR.

• REQUIRED for initial method validation.

5959

Summary

Clinically Reportable Range is established once when a method is introduced based on clinical requirements as defined by the lab director.

SOP for dilution or other specimen pre-treatment must be defined.

Not required to verify CRR on a recurring schedule

60

Summary

For additional information on the content of the Audioconference or questions about the CAP Calibration Verification/Linearity Surveys, please contact the CAP:

By telephone: Customer Contact Center at 800-323-4040, option 1#

By email: contactcenter@cap.org

By mail: Surveys Program325 Waukegan RoadNorthfield IL 60093-2750