APPLICATION OF SIX SIGMA METRICS TO INTERNAL QC DATA

D. Suhasini,Senior Consultant and HOD, Department of Biochemistry & Therapeutic Drug Monitoring,

Apollo Health City, Jubilee Hills, Hyderabad 500 033,India.e-mail: suhasini_d@apollohospitals.com; suhasini.dr@gmail.com

INTRODUCTION

“SIGMA METRICS” provide a universal benchmark forprocess performance. The performance of all processescan be characterized on the “Sigma scale.” Valuestypically range from 2 to 6, where the goal for “worldclass quality” is 6. If the Sigma metric is less than 3, theprocess is very unreliable.

Clinical laboratories are required to participate inExternal Quality Assessment/Proficiency TestingPrograms to meet with the requirements of a goodlaboratory practice. An external QC sample is similar to apatient’s sample in that both are given to be analyzed andreported only once. Any undetected error in thelaboratory systems is bound to get reflected in both theresults. If the performance of an analyte is notsatisfactory in the PT program, it is required by thelaboratory to analyze the reasons and to initiatecorrective actions to improve on the performance. Theperformance of the analyte, in terms of inaccuracy andimprecision if monitored continuously and effectively,would definitely instill confidence in the reporting of anyresult – be it the PT or patient samples by the laboratoryand also reduces the necessity of the drone ofdocumenting corrective actions.

METHODOLOGY

The performance of 26 analytes was analyzed usingSigma Metrics on Dade Dimension systems.

The monthly internal QC data using Bio-Rad Assayedcontrols (Lots 14130 and 14140), Levels 1 and 2 for 7months – from December-2006 to June 2007 werecompared with the respective monthly peer group means(Dade Dimension Series) from the monthly ‘LaboratoryComparison Report, Assayed Chemistry’ issued by Bio-Rad through their ‘UNITY’ Programme, to calculate ourlab’s % Bias. (We send the data to Bio-Rad by using theirQC on Call software). The respective monthly % CVs of

our lab were noted. The total allowable error (TEa%) foreach parameter were taken from the CLIA guidelines(when given) and from the Biological goals updated list.

Sigma Metrics were calculated for each month atboth level 1 and level 2 control concentrations by usingthe formula: (% TEa-% Bias) / % CV.

The following averages for our lab were calculatedfor the data of the 7 months as mentioned above.

• Average Concentration of the analyte (for bothControl Levels 1 and 2)

• Average % Bias of the analyte (for both ControlLevels 1 and 2) as compared with the peer groupmeans

• Average % CV of the analyte of our lab (for bothControl Levels 1 and 2) and

• Average Sigma Metric of the analyte (for bothControl Levels 1 and 2) for our lab.

Sample data analyzed for Glucose for both thecontrol levels is given in Table 1 & 2

Based on the performance, the analytes wereclassified into 2 groups:

(a) GROUP I: 7 Analytes of “Average” performance: atconcentrations of either or both Control levels withSigma Metrics < 5.0 , viz. Alkaline Phosphatase,Creatinine, LDL Cholesterol (direct measure),Lactate, Prealbumin, Phosphorus and Urea; and

(b) GROUP II: 19 Analytes of excellent performance atconcentrations of both Control levels, with SigmaMetrics 5.0-6.0 or more, viz. Albumin, ALT, AST,Amylase, Calcium, Conjugated Bilirubin, CK,Glucose, GGT, HDL Cholesterol, Iron, LDH,Lipase, Magnesium, Total Bilirubin, TotalCholesterol, Total Protein, Triglycerides and Uricacid.

Review Article

37 Apollo Medicine, Vol. 4, No. 1, March 2007

Apollo Medicine, Vol. 4, No. 1, March 2007 38

Review Article

Table 1: Glucose on Dimension RXL Max; TEa = 10%

BIORAD LEVEL-1

Lot No. Mth/Yr Peer Group Mean Observed Lab Observed Monthly Observed Monthly Monthly(IQC) Monthly Mean % BIAS (IQC) % CV (ICQ) SIGMA

(IQC) Metric

14130 Dec-06 85.9 84.8 1.32 1.4 6.214130 Jan-07 85.3 84.3 1.21 1.4 6.314140 Jan-07 89.6 88.0 1.83 1.7 4.814140 Feb-07 89.4 87.5 2.14 1.4 5.614140 Mar-07 89.0 86.6 2.65 1.3 5.714140 Apr-07 89.5 88.3 1.37 1.9 4.514140 May-07 89.1 88.7 0.44 1.7 5.614140 Jun-07 89.1 90.4 1.48 1.3 6.6

Average Overall Overall Observed Overall Overall Mean SIGMASIGMA Metric Peer Group Lab Monthly Mean Mean Metric-

L1 & L2 for Monthly Mean Mean % BIAS %CV Level 1Glucose on (IQC) (IQC)

RxL Max6.84 88.36 87.32 1.56 1.51 5.66

Table. 2: Glucose on Dimension RXL Max; TEa = 10%

BIORAD LEVEL-2

Lot No. Mth/Yr Peer Group Mean Observed Lab Observed Monthly Observed Monthly Monthly(IQC) Monthly Mean % BIAS (IQC) % CV (ICQ) SIGMA

(IQC) Metric

14130 Dec-06 278.8 275.1 1.33 1.0 8.714130 Jan-07 276.8 273.5 1.19 1.7 5.214140 Jan-07 279.7 279.5 0.07 1.1 9.014140 Feb-07 279.2 277.7 0.54 1.0 9.514140 Mar-07 278.8 276.8 0.72 1.1 8.414140 Apr-07 278.7 279.1 0.14 1.2 8.214140 May-07 278.3 275.8 0.90 1.0 9.114140 Jun-07 278.5 279.5 0.36 1.6 6.0

Overall Overall Observed Overall Overall Mean SIGMAPeer Group Lab Monthly Mean Mean Metric-

Monthly Mean Mean % BIAS %CV Level 2(IQC) (IQC)

278.60 277.13 0.66 1.21 8.01

Method decision charts were plotted to point the laboperating points at levels 1 and 2, for all the analytes (notshown in this article).

The QC screens of all auto analyzers and the specificQC software (like QC on Call from Bio-Rad that we usein our laboratory) only show monthly average, StandardDeviation and % CV for all analytes. “Tolerable CV”

from the Sigma Metrics data, may be used to give us firsthand information alerting us to take corrective action,using the following formulae:

Tolerable CV at 6.0 Sigma = (% Total Allowableerror – % Bias) /6.0

Tolerable CV at 5.0 Sigma = (% Total Allowableerror – % Bias) /5.0

Review Article

39 Apollo Medicine, Vol. 4, No. 1, March 2007

Keeping the bias % constant, the desirable %CVrange to achieve a 6-5 sigma process has been calculated.This of course, may be applied effectively to thoseparameters where % Bias is within tolerable limits.

ALP on Dade Lab LabDimension RxL Max Performance Performance

L1 L2

TEa% 30.0 30.0Lab average (U/L) 76.5 311.9Average CV % 6.49 6.35Average Bias% 11.25 5.15Average Sigma 2.97 3.98Calculated % Impre- 3.12 - 3.75 4.14 - 4.97cision Range for a 6to 5 Sigma Process

Both inaccuracy and imprecision need to be addressed withALP

Creatinine on Dade Lab LabDimension RxL Max Performance Performance

L1 L2

TEa% 15.0 15.0Average Lab 2.18 6.60Concentration (mg/dL)Average CV % 3.33 1.65Average Bias% 3.49 1.50Average Sigma 3.53 8.25Calculated % Impre- 1.92 - 2.30 2.25 - 2.70cision Range for a 6to 5 Sigma Process

Both inaccuracy and imprecision need to be reducedwith creatinine at the Level 1 control concentration of 2.18mg/dL

Lactate on Dade Lab LabDimension RxL Max Performance Performance

L1 L2

TEa% 30.4 30.4Average Lab 4.05 1.24Concentration (mmol/L)Average CV % 3.58 9.80Average Bias% 1.41 8.61Average Sigma 8.18 2.32Calculated % Impre- 4.83 - 5.80 3.63 - 4.36cision Range for a 6to 5 Sigma Process

Again, it’s the bias and the CV that need to be reduced withLactate at an average low concentration of 1.24 mmol/L ofLevel 2.

LDL Cholesterol on Lab LabDade Dimension RxL Performance Performance

L1 L2

TEa% 13.6 13.6Average Lab 140.8 61.2Concentration (mg/dL)Average CV % 2.65 2.97Average Bias% 4.92 7.82Average Sigma 3.73 2.13Calculated % Impre- 1.45 - 1.74 0.97 - 1.16cision Range for a 6to 5 Sigma Process

The bias needs to be reduced with Direct LDL at anaverage low concentration of 61 mg/dL of Level 2.

Prealbumin on Dade Lab LabDimension RxL Max Performance Performance

L1 L2

TEa% 14.50 14.50Average Lab 26.0 16.9Concentration (mg/dL)Average CV % 2.11 2.73Average Bias% 2.66 1.81Average Sigma 6.21 4.66Calculated % Impre- 1.97 - 2.36 2.11 - 2.54cision Range for a 6to 5 Sigma Process

Peer group comparison and comparisons with group valuesby method for Prealbumin were not available. Comparisonswere made between monthly and cumulative means of ourlab.

Urea on Dade Lab LabDimension RxL Max Performance Performance

L1 L2

TEa% 15.7 15.7Average Lab 34.5 100.5Concentration (mg/dL)Average CV % 4.41 2.73Average Bias% 1.54 1.47Average Sigma 3.44 5.48Calculated % Impre- 2.36 - 2.83 2.37 - 2.84cision Range for a 6to 5 Sigma Process

The imprecision needs to be reduced further with Urea at anaverage concentration of 34.5 mg/dL of Level 1

Apollo Medicine, Vol. 4, No. 1, March 2007 40

Review Article

Analyte % Tea used Average Sigma Metrics of MonthlyPerformance at Levels 1 & 2 of

Control Concentrations

1. Albumin 10.0 16.72

2. ALT 20.0 10.90

3. AST 20.0 7.09

4. Amylase 14.6 (BV Goals) 13.12

5. Calcium 14.3: L 1 8.059.1: L 2

6. Conjugated Bilirubin 44.5 (BV Goals; Given for only 9.51Total Bilirubin in CLIA )

7. CK 30.0 8.88

8. Glucose 10.0 6.84

9. GGT 22.2 (BV Goals) 10.29

10. HDL Chol 30.0 7.90

11. Iron 20.0 8.24

12. LDH 20.0 9.37

13. Lipase 29.1 (BV Goals) 5.36

14. Magnesium 25.0 8.10

15. Total Bilirubin 20.0 10.43

16. Total Cholesterol 10.0 6.08

17. Total Protein 10.0 5.88

18. Triglycerides 25.0 12.48

19. Uric Acid 17.0 8.07

RESULTS

GROUP I: Analytes of “Average” performance: atconcentrations of either or both Control levels withSigma Metrics < 5.0:

1. Alkaline Phosphatase

2. Creatinine

3. LDL Cholesterol (direct measure)

4. Lactate

5. Prealbumin

6. Phosphorus and

7. Urea

Phosphorus on Dade Lab LabDimension RxL Max Performance Performance

L1 L2

TEa% 10.2 10.2Average Lab 3.37 7.25Concentration (mg/dL)Average CV % 2.01 1.30Average Bias% 1.12 0.70Average Sigma 4.76 7.52Calculated % Impre- 1.51 - 1.81 1.58 - 1.90cision Range for a 6to 5 Sigma Process

The imprecision of Phosphorus at an average concentrationof 3.37 mg/dL of Level 1 is high which needs to be reduced.

GROUP II: Analytes of excellent performance: summarized in the following table:

Review Article

41 Apollo Medicine, Vol. 4, No. 1, March 2007

CONCLUSIONS

The translation of method performance data intoSigma metrics will verify the manufacturer’s claims ofmethod performance. Sigma Metrics help to quantify theperformance of analytes in a whole some manner in that,the imprecision; inaccuracy and the total allowable errorfor the particular analyte are considered. They also helpin deciding upon the QC rules to be applied for differentanalytes. We have already brought a reduction in therules for the very good analytes, making our life easier,not getting disturbed with false alarms. These metricsmay be applied to hematological parameters too.

The application of Sigma metrics to monthly internalQC gives a more meaningful interpretation. The wholeexercise has opened up our minds much more too toconfidently deal with the problems faced in our lab. Theoverall understanding about QC practices has improvedgreatly and though a lot of ground work was required tobe done, it has definitely laid a foundation to get a

baseline of the performance in lucid terms; this has alsomade all our technical staff to get involved (apart fromthe select ‘senior ones’) in more than ‘just runningcontrols and filing the printouts’.

We have undertaken a prospective study in applyingthe calculated tolerable CVs based on Sigma Metrics toserve as ready reckoners for assessing methodperformance on a day-to-day basis.

BIBLIOGRAPHY

1. http://www.westgard.com/lesson25.htm (MV-The Deci-sion on Method Performance)

2. http://www.westgard.com/essay 111.htm (A word fromJOW: The Meaning and Application of Total Error).

3. CLIA Requirements for Analytical Quality. http://westgard.com

4. Desirable Specifications for Total Error, Imprecisionand Bias derived from Biologic Variation. http://westgard.com