measuring quality in anatomic pathology

Measuring Quality in AnatomicPathology

Stephen S. Raab, MD*,Dana Marie Grzybicki, MD, PhDDepartment of Pathology, University of Colorado Denver,

12605 East 16th Avenue, Aurora, CO 80045, USA

Quality

In anatomic pathology, quality is the product (ie, the diagnostic informa-tion) or service that meets the requirements of a wide number of individualsor groups, including patients, clinicians, pathologists, pathologist extenders,organizations, and regulatory agencies [1,2]. Quality assurance (QA) is a sys-tem of control activities that promotes the higher level functioning ofspecific processes within the anatomic pathology laboratory. These pro-cesses usually are composed of multiple steps or subprocesses, each of whichis subject to quality control (QC) activities to ensure that these steps or sub-processes meet acceptable parameters [1,2]. QC activities often entail thedevelopment of standards of acceptable performance. In an ideal system,QA/QC activities are used to guide quality improvement (QI) activitiesthat attempt to improve the quality of a specific product or service beyondits current status [1,2]. Well-designed QA programs by themselves may leadto improved product or service quality [1].

In industry, standardization is a key component of quality, and consider-able effort is expended on standardizing work processes and materials [1,3–5]. One rationale for standardization is that there is a best method of per-forming work, which should be adopted to produce work of optimal quality.In medicine, standardization is highly variable within and across areas ofcare delivery, including anatomic pathology laboratories [6–9]. For example,one anatomic pathology laboratory may have well-documented, standard-ized protocols for the gross examination of specific specimen types, but

Clin Lab Med 28 (2008) 245–259

* Corresponding author.

E-mail address: [email protected] (S.S. Raab).

0272-2712/08/$ - see front matter � 2008 Elsevier Inc. All rights reserved.

doi:10.1016/j.cll.2007.12.004 labmed.theclinics.com

mailto:[email protected]

http://www.labmed.theclinics.com

246 RAAB & GRZYBICKI

the actual work processes of gross examination may not be standardized, soindividuals within that laboratory grossly examine specimens differently. Asecond laboratory may have standardized policies and work processes. Dif-ferent approaches do not necessarily imply that one anatomic pathologylaboratory provides higher quality services compared to another, becausestrengths may be seen in alternative approaches [10]. The lack of standard-ization implies that an optimal level of product or service is not being deliv-ered across the population of all anatomic pathology laboratories, however.

In anatomic pathology, standards for acceptable performance have beenadopted for several quality metrics, and other quality metrics are closelyexamined despite the lack of performance standards [1,11–15]. Examplesof traditional measures of anatomic pathology quality are diagnostic accu-racy, customer (clinician) satisfaction, and specimen turn-around time. Thisarticle focuses mainly on diagnostic accuracy, recognizing that measuringany quality metric is complex and demanding [11]. Laboratories use severalmethods to standardize quality metrics, and the College of AmericanPathologists, the American Society for Clinical Pathology, and the Papani-colaou Society of Cytopathology have taken leadership roles in developingthese quality metrics [12]. Laboratories traditionally have created qualitystandards by benchmarking current practice or using expert opinion [12]rather than developed standards based on evidence-based outcome assess-ments. The separation between laboratories and clinical services and thelack of health information technology tools are two reasons why linkingclinical outcome to testing services is difficult.

No diagnostic or screening test has 100% sensitivity and 100% specificity[1,2]. Much to the chagrin of health care personnel involved in diagnostictesting and screening, specific processes in testing services often are heldto a high level of performance, which leads to the impression that laboratoryservices must achieve near perfection. Pap test screening and interpretationis one example in which laboratories are held to exceptionally high levels ofaccountability. In reality, measures of diagnostic accuracy depend on labo-ratory and clinical performance, and for some diagnostic or screening tests,the literature reports wide accuracy variations, which partly reflects a lack ofclinical or laboratory process standardization. Consequently, althoughlaboratory QA/QC activities evaluate metrics of diagnostic accuracy, labo-ratories can only address and control domain-specific subprocesses that con-tribute to the overall test accuracy [10]. Measuring the diagnostic accuracyof individual subprocesses is difficult because multiple factors from preced-ing testing steps bias assessments [10,16] (see later discussion). Although weintuitively know what constitutes good and poor diagnostic or screening testperformance, a lack of standards limits our ability to evaluate true test per-formance for individual clinical and laboratory services.

Because testing within the anatomic laboratory is composed of numeroussteps [10,16], laboratories may use QA/QC activities to target the quality ofany individual step to improve the quality of the overall laboratory process.

247MEASURING QUALITY IN ANATOMIC PATHOLOGY

QA/QC activities may be applied to processes that occur in any one of theindividual anatomic pathology laboratory components or testing areas, suchas an accessioning area, gross room, frozen section area, histology or cytol-ogy laboratory, transcription area, laboratory information service area,autopsy suite, ancillary testing laboratory, fine-needle aspiration clinic, orpathologist’s domain [10]. For example, histologic slide staining qualityinfluences the overall laboratory quality metrics of turn-around time anddiagnostic accuracy, because poorly stained sections may necessitate restain-ing (which affects specimen turn-around time) and interfere with diagnosticinterpretation (affecting diagnostic accuracy). Histology section quality maybe monitored through QC activities that track the quality of hematoxylinand eosin–stained slides.

Medical error

In 1999, the Institute of Medicine published its famous report ‘‘To Err isHuman: Building a Safer Health System,’’ which estimated that between44,000 and 98,000 patients die each year as a result of medical error.Many more patients suffer from morbidities associated with medical error[17]. The Institute of Medicine estimated that medical errors result in totalcosts (including the expense of additional care, lost income and householdproductivity, and disability) of between $17 billion and $29 billion in UnitedStates hospitals annually [17]. Medical errors that occur in ambulatory caresettings also result in considerable costs.

The Institute of Medicine defined a medical error as the failure ofa planned action to be completed as intended or the use of a wrong planto achieve an aim, although others have advocated alternative definitionsof error [17]. The determination of whether a medical error occurred is per-formed separately from patient outcome assessment or root cause analysis,although medical errors may be subclassified after outcome assessment orroot cause analysis. When examining outcomes, errors may be classified as‘‘no harm,’’ ‘‘near miss,’’ or ‘‘harm’’ events, and some have labeled errorsassociated with harm as ‘‘adverse events’’ [17]. Most medical errors are notassociated with poor patient outcomes. Root cause analysis shows thatdiagnostic testing or screening errors may be caused by process failures out-side of the laboratory domain and do not necessarily imply that a pathologistmisinterpreted a specimen [10]. Unfortunately, many health care personneltend to protect their domain, which limits the ability to determine how mul-tiple factors compound error in diagnostic testing and screening.

Based on the Institute of Medicine definition of error, anatomic pathologydiagnostic testing or screening error is the failure to diagnose correctly thedisease process in a patient [18]. Pathologists have long known about failuresin diagnostic testing and screening processes, and QA/QC activities docu-ment these failures using diagnostic accuracy metrics [1,2]. False-negativeand false-positive diagnoses are examples of errors of interpretation


[11,18]. Some patient safety researchers also consider that indeterminate (eg,atypical or suspicious) diagnoses are forms of error, because this category ofdiagnosis does not accurately convey if a patient does or does not have dis-ease [11]. Unfortunately, the term ‘‘error’’ often connotes a value judgmentthat is highly negative and pejorative for many health care professionals.The fact that pathologists use indeterminate diagnoses reflects the currentstate of practice, especially for less-than-optimal specimens, because indeter-minate diagnoses provide probability estimates that limit errors of greaterseverity (eg, false-negative or false-positive diagnoses) [19]. In the ideal diag-nostic testing or screening scenario, the frequency of less-than-optimal spec-imens would be considerably lower, thereby limiting the use of indeterminatediagnoses.

In some pathology circles, there has been an avoidance of the term ‘‘er-ror’’ partly because of the negative connotations and because of the fearof the consequences of reporting. Alternatively used terms include discrep-ancy, defect, flaw, deficiency, and variance.

Diagnostic testing and screening error taxonomies

Practitioners and researchers use different taxonomies to classify diagnos-tic testing and screening error, and these taxonomies have different strengthsand weaknesses. A few of these taxonomies are discussed in these sections.

Error classified by testing phase

Diagnostic testing and screening are composed of multiple steps thatcomprise the total testing cycle, as defined by Lundberg [20]. The five phasesof the total testing cycle in the Lundberg model are as follows:

1. Pre–pre-analytic: Determining to test and choosing the specific test2. Pre-analytic: Procuring and transporting the specimen3. Analytic: Processing and interpreting the specimen4. Post-analytic: Reporting the test results5. Post–post-analytic: Acting upon the test results

The anatomic pathology (or analytic) processes, including the diagnosticinterpretation, play a critical but only partial role in the overall care of thepatient. QA/QC activities may be difficult to undertake in diagnostic testingand screening because the processes in diagnostic testing and screening crossmultiple domains of ownership [1]. Because tests may be viewed as parcels ofinformation that are transformed at each step, problems (or errors) in anystep may result in compromised data. In some instances (partly becauseof a lack of standardization), it is impossible to determine when (or if) infor-mation has been compromised, and a probabilistic approach may be used todetermine the level of error after a diagnostic or screening test. Some clini-cians use Bayesian methods to determine the post-analytic probabilities of


disease based on the pre-analytic probabilities of disease and the test results(expressed probabilistically).

Stroobants and colleagues [21] estimated that errors occurred in approx-imately 20% of clinical pathology tests and that the proportion of errorwithin the pre–pre-analytic, pre-analytic, analytic, post-analytic, and post–post-analytic testing phases was 12.0%, 2.0%, 0.2%, 2.2%, and 5%, respec-tively. Estimates of the error proportions in anatomic pathology testing andscreening cycle have not been published, although most likely they are atleast the same as those reported by Stroobants and colleagues [21]. Mostanatomic pathology testing errors likely occurs in the pre–pre-analyticand the post–post-analytic phases. The lack of standardization of test order-ing, choice, and follow-up clearly are major sources of error. The pre–pre-analytic evaluation of patients with a lung nodule suspicious for cancer isan example, because the choice of test depends on the entry point (ratherthan evidence based analysis of patient outcomes of different combinationsof diagnostic pulmonary tests) of patients [22]. Primary care health care pro-viders are more likely to order sputum cytology, pulmonologists are morelikely to order bronchoscopy, and surgeons are more likely to opt for wedgeexcision for patients with the same clinical findings [22]. These test choicesaffect the types of specimens received by anatomic pathology laboratories(and the types of errors that occur within them). Anatomic pathology-basedstudies of diagnostic testing error generally have not evaluated the correct-ness of test choice for many patient scenarios.

Most anatomic pathology QA/QC activities evaluate the errors thatoccur in the analytic phase of testing, although some QA/QC activitiesreport specimen collection and other pre-analytic or post–post-analytic fail-ures. Within the anatomic pathology laboratory, errors may be classified as

Accessioning (eg, specimen identification switch with a second patient,wrong physician entered, wrong patient information entered)

Grossing (eg, specimen not properly sampled, tissue blocks mislabeled,tissue blocks too thick, tissue not properly fixed)

Histology/processing (eg, sections cut too thickly, tissue not properlyprocessed, floater picked up in waterbath, tissue not adequatelystained)

Transcription (eg, misspelling, incorrect format of report, case assignedto the incorrect pathologist, information omitted from report)

Ancillary testing (eg, wrong ancillary test performed, failure of ancillarytest, ancillary test reported to the wrong patient)

Sign-out (eg, pathologist misinterpretation, disease process incorrectlydescribed, report dictated in a confusing manner)

Reporting (eg, wrong report issued, report sent to the wrong physician,report lost)

Some anatomic pathology laboratories separate errors into the categoriesof laboratory-related and pathologist-related errors [1]. In this scheme,


laboratory-related errors are secondary to failures in all the analytic testingphases, excluding the interpretation phase (or the grossing phase if pathol-ogists perform the gross examination). This separation is helpful in labora-tories with separate domains of hospital and pathologist control. At theUniversity of Pittsburgh Medical Center, laboratory information systempersonnel created a dictionary of anatomic pathology laboratory-relatederrors, which contains more than 200 distinct deficiencies attributable todifferent laboratory sections.

Active and passive methods and timing of error detection

Anatomic pathology error detection may be separated into active andpassive methods. Active methods of error detection collect more errorsthan passive methods. In a preliminary study of active error detection in an-atomic pathology specimen accessioning areas in multiple hospitals, Grzy-bicki [23] used an observational method of error detection in which errorswere recorded by a third party. Grzybicki [23] reported that this activemethod detected errors in the accessioning phase in more than 70% of spec-imens, whereas passively detected errors, measured through anatomic pa-thology QA logs, occurred in less than 3% of specimens. The active errordetection method markedly increased the frequency of specific accessioningerror types, such as accessioning cases (1) without two patient identifiers,(2) without sufficient patient information on the requisition form, and (3)with improperly matched requisition form and specimen container [23].Most of these errors presumably resulted in no patient harm, althoughone can imagine that extremely rare instances of these errors could leadto catastrophic consequences. The accessioning personnel generally per-formed ‘‘work-arounds’’ to correct these problems because they occurredso frequently and were never fixed [1]. In a second active error detectionstudy of the accessioning phase, Zarbo and D’Angelo [16] reported that28% of anatomic pathology specimens were defective. These data indicatedthat the frequency of anatomic pathology error may be far higher than thatassessed by Stroobants and colleagues [21] in the clinical laboratory and thatdefects are present in most anatomic pathology specimens. This findingshould not be surprising given the large number of steps in anatomic pathol-ogy testing.

Error detection also may be classified as prospective or retrospective [11].Prospective methods generally are active in nature and are aimed at limitingthe number of adverse events. An example is a secondary slide review of allcases diagnosed as malignant by a first pathologist. The number of errorsdetected in this fashion is relatively unknown, because some laboratoriesdo not track these errors and some pathologists do not even consider theseevents as true errors. In conventionally accepted patient safety terms, theseerrors are classified as near-miss events, because they are corrected beforehaving an affect on patient outcome.


Retrospective error detection methods may be active or passive. Activeretrospective error detection is helpful when these error data are used toguide QI activities.

Traditional anatomic pathology error detection methods

Error detection methods in anatomic pathology are presented in Box 1.These error detection methods are examples of secondary case review andoften serve as the basis for much of the assessment of error frequencyreported in the pathology literature [11]. Errors detected by these methodsmainly occur in the interpretation process [24], except in the correlationreviews in which errors are more likely to occur as a result of pre-analyticfailures. The proportion of anatomic pathology cases with an interpretationerror depends on the specific review method. Much of the pathology errorliterature is based on single institutional studies, which contain biases causedby the lack of pre-analytic and post-analytic process standardization [25,26].

Comparing institutional error proportions using secondary case reviewmethods is challenging because of the lack of QA/QC process standardiza-tion. Vrbin and colleagues [27] studied the level of standardization incytologic-histologic correlation. Vrbin and colleagues sent a survey to 162American laboratories requesting copies of the materials they used in theircytologic-histologic correlation process. They developed a checklist (derivedfrom the College of American Pathologists Laboratory Accreditation Cyto-pathology Checklist) to classify the minimum expected (15) and additionaldata points that laboratories collected when they performed cytologic-histologic correlation. No laboratories collected the exact same data, and17.3% of laboratories did not record any data on forms, logs, or tally sheets.The mean number of minimum expected and additional variables recorded

Box 1. Error detection methods in anatomic pathology

Frozen section–permanent section correlationCytologic-histologic correlationFine-needle aspiration immediate and final diagnosis correlationConference case review (eg, tumor board, unknown conference,

daily difficult case conference, subspecialty conference)Random or pseudorandom case reviewFocused review of specific case typesSecond opinion or consultation (eg, retrospective or prospective,

internal or external), performed outside of conference casereview

Amended report reviewAutopsy case reviewClinician driven review


on forms was 6.5 and 8.7, respectively. Vrbin and colleagues concluded thatlaboratories recorded data from the cytologic-histologic correlation processin several ways, which indicated the lack of standardization of the data col-lection process. Raab and colleagues [18] confirmed the absence of standard-ized cytologic-histologic error detection processes in four laboratories thatshared correlation data for QI activities.

Error detection by amended report review

Zarbo and colleagues [11] classified errors into four categories based onthe review of amended reports:

1. Interpretation errors, further subclassified as false-negative diagnoses(undercalls), false-positive diagnoses (overcalls), and misclassifications(not altering primary or secondary diagnostic classifications; eg, small-cell carcinoma versus non–small-cell carcinoma of the lung)

2. Specimen errors, including lost specimens, inadequate or nonrepresenta-tive specimens, and improperly handled specimens (eg, specimens notprocessed appropriately in the analytic testing phase or specimens thatdid not receive the proper ancillary test in the analytic phase)

3. Identification errors, including incorrectly identified patient material(eg, wrong patient), tissues (eg, stomach versus colon), or anatomiclocation (eg, right versus left lung)

4. Reporting errors, including erroneous or missing nondiagnostic infor-mation, dictation or typing errors, and report formatting errors

Zarbo and colleagues [11] used the term ‘‘defect’’ instead of ‘‘error’’ andbased this classification scheme on the type of change made in an amendedreport and included changes in the (1) primary diagnostic characteristics, (2)secondary diagnostic characteristics, (3) diagnostic reclassification, (4) pa-tient or specimen identification, (5) specimen characteristics (eg, resamplingthe specimen leading to a changed diagnosis), and (6) other edits not includ-ing the changes made in categories 1 to 5. Essentially, this classificationscheme is a form of root cause analysis (other methods of root cause anal-ysis are presented later) because it links errors to specific process failures.

An example shows the use and overlap of these different error classifica-tion schemes. As mentioned in the preceding paragraph, cytologic-histologiccorrelation is a method of error detection that may be performed retrospec-tively or prospectively with conjoint cytologic and histologic specimens (eg,Pap test and cervical biopsy procured during colposcopy or bronchial brushand bronchial biopsy procured during bronchoscopy). In the frame of errorsclassified by phase of total testing cycle (including the phases within theanatomic pathology laboratory), cytologic-histologic correlation detectserrors mainly in the pre-analytic (ie, specimen sampling) and analytic (ie,specimen processing and interpretation) testing phases. Using the amendedreport error classification scheme advocated by Zarbo and colleagues [11],


cytologic-histologic correlation errors may be subclassified as secondary todefective specimens or defective interpretations. Cytologic-histologic corre-lation may be performed actively but generally is performed retrospectively.

Diagnostic disagreement and error

Physician interpretation is a complex, cognitive task, and physicians un-dergo long periods of training and evaluation before becoming credentialedto make these interpretations. Many areas of medicine exhibit high levels ofvariability in physician judgments [28–30], with pathology being no excep-tion. In internal medicine, for example, physicians may disagree regardingthe interpretation of chest radiographs, EKGs, and the signs and symptomsof patients. These disagreements reflect not only the complexity of humanphysiology and disease processes but also the complexity of medical decisionmaking. Some of these disagreements reflect erroneous decisions caused byhuman cognitive error and occur commonly in all fields of medicine.

In most scenarios, diagnostic disagreements are not associated withharmful outcomes. For example, two pathologists may differently subclas-sify the same high-grade sarcoma. These differences may not affect patientprognosis or clinical management; however, the patient either has one ma-lignancy or the other, and one diagnosis fails to describe the disease processin the patient. In some situations, we currently may lack the knowledge baseto make the distinction among tumor types, and research may be needed toaddress this issue further. Some pathologists have argued that most interpre-tive disagreements are not true errors and only interpretive disagreementsassociated with harm are errors. The weakness of this argument lies in itsnecessary association of error with an adverse event, when only a small sub-set of the total population of errors results in a clinical adverse event. Manypatient safety scientists argue that focusing on adverse events is important,because detecting and preventing these errors reduce patient harm. A chal-lenge in this stage of patient safety investigation is determining whichanatomic pathology errors are associated with harm so that they may be tar-geted for further study.

Root cause analysis

There are several well-accepted methods of performing root cause analy-sis. The amended report method presented by Zarbo and colleagues [11] isuseful because it specifically examines error causes from a diagnostic testingand screening perspective. This method ignores specific causes of error (eg,system error), however, and does not delve deeply into other causes (eg,causes of cognitive error). Patient, specimen, provider, and system factorscause diagnostic testing and screening errors [1]. A root cause analysismethod that has been applied effectively to anatomic pathology is the Eind-hoven Classification Model for the Medical Event Reporting System for

Table 1

Classification of root causes

Code Category Definition

Latent errors Errors that result from underlying system

failures

Technical: Physical items, such as equipment, physical installations, software, materials,

labels, and forms

TEX External Failures beyond the control of the investigating

organization

TD Design Inadequate design of equipment, software,

or materials; can apply to the design

of workspace software packages, forms,

and label design

TC Construction Designs that were not constructed properly;

examples include incorrect set-up and

installation of equipment in an inaccessible

area

TM Materials Material defects found; examples could be the

weld seams on blood bags, defects in label

adhesive or ink smears on preprinted labels

or forms

Organizational

OEX External Failures beyond the control and responsibility

of the investigation organization

OP Protocols/procedures Quality and availability of protocols that are

too complicated, inaccurate, unrealistic,

absent, or poorly presented

OK Transfer of knowledge Failures resulting from inadequate measures

taken to ensure that situational or

site-specific knowledge or information is

transferred to all new or inexperienced staff

OM Management priorities Internal management decisions in which safety

is relegated to an inferior position when

there are conflicting demands or objectives,

which is a conflict between production

needs and safety

OC Culture A collective approach, and its attendant

modes, to safety and risk rather than the

behavior of just one individual; groups

might establish their own modes of

function as opposed to following

prescribed methods

Active errors Errors or failures that result from human

behavior

HEX External Failures that originate beyond the control

and responsibility of the investigation

organization

Knowledge-based behaviors

HKK The inability of an individual to apply his or

her existing knowledge to a novel situation

(continued on next page)


Table 1 (continued )

Code Category Definition

Rule-based behaviors

HRQ Qualifications The incorrect fit between an individual’s

qualification, training, or education

and a particular task

HRC Coordination A lack of task coordination within a health

care team in an organization

HRV Verification The incorrect or incomplete assessment of

a situation, including related conditions

of the patient/donor and materials to be

used before beginning the task

HRI Intervention Failures that result from faulty task planning

and execution, which would be selecting the

wrong rule or protocol (planning) or

executing the protocol incorrectly (execution)

HRM Monitoring Failures that result from monitoring of process

or patient status

Skill-based behaviors

HSS Slip Failures in the performance of highly developed

skills

HST Tripping Failures in whole-body movement; errors are

often referred to as ‘‘slipping, tripping,

or falling’’

Other factors

PRF Patient-related factors Failures related to patient/donor characteristics

or actions that are beyond the control of the

health care professional team and influence

treatment

Unclassifiable Failures that cannot be classified in any of the

current categories


Transfusion Medicine [31–33]. This method focuses on three domains: tech-nical (equipment, forms, and software), organizational (procedures, policiesand protocols), and human (knowledge based, rule based, and skill based).These three domains are useful in classifying contributing factors and orga-nizing causes of error. They allow for error investigation to focus on systemfactors rather than entirely on human factors. Limitations in our current QCmeasures of diagnostic testing and screening error are the excessive focus oninterpretation error and the inability to determine contributing factors todiagnostic misinterpretation. Table 1 shows a more detailed list of errorcauses in this classification model [31–33].

Raab and colleagues [34] performed root cause analysis by examining theoverall and individual QA/QC diagnostic test performance data and deter-mining causes of error based on less-than-optimal test performance. Thelimitation in this method was that root cause analysis was performeda long time after the error occurred, and aspects of the testing process couldnot be evaluated in retrospect [31–33]. A benefit of studying overall test


performance data was that system issues could be studied better [34]. Aproblem in studying diagnostic testing and screening error is that test fail-ures may not be known for considerable time periods until a repeat testor definitive procedure shows a different disease process than was originallydiagnosed or clinical suspicion evokes case re-evaluation.

Raab and colleagues [34] coded errors using the Eindhoven ClassificationModel and created a table displaying major factors that contributed toerror. A few examples of organizational error causes in thyroid glandfine-needle aspiration are as follows [34]:

1. OM (organizational management priorities): Radiology division pro-cesses patients too quickly to allow for proper fine-needle aspirationperformance. The cytology schedule is too busy to implement cytolo-gists in the performance of immediate interpretations. The hospitaldoes not mandate the sending of patients with palpable lesions tomore experienced aspirators.

2. OP (organizational protocols and procedures): There is a lack of stan-dardization for pathology sign-out procedures, diagnostic criteria forcategory use, and radiology procedures.

3. OC (organizational culture): The system focuses on punishment and notimprovement. There is no system for formal root cause analysis.

4. OK (organizational transfer of knowledge): New cytologists or lessexperienced cytologists are not taught in a rigorous fashion.

Raab and colleagues [34] also evaluated the causes of error in individualcases and constructed causal trees that represented the factors, activities,and decisions that possibly lead to errors. Although specific for individualcases, false-negative and false-positive diagnoses generally were related tomultiple causes, including lack of immediate interpretation services, patient-related factors, and overall high workloads. These error causes compoundedother knowledge-based error causes that led to the procurement of less-than-optimal samples or misinterpretation of these samples.

Most thyroid gland fine-needle aspiration errors are detected by the cyto-logic-histologic correlation process, which traditionally has used a rootcause analytic method that classifies error into the two categories of sam-pling and interpretation. The studies by Raab and colleagues [34] and Noditand colleagues [35] indicated that this binary classification is highly useful instudying general process failures but is less informative in determining latenterror causes in the interplay between sampling and interpretation error.Simply classifying cytologic-histologic correlation errors as either interpre-tation or sampling generally does not provide sufficient information thatmay be used for system QI. The understanding of the root cause of errorrequires more detailed analysis of factors extending beyond anatomicpathology laboratories.

Raab and colleagues [36] reported that the range of pairwise interobserverkappa values for pathologists who assessed error cause of pulmonary


cytologic-histologic correlation cases was�0.154 to 1.0. These data indicatedthat the traditional method of root cause analysis for cytologic-histologiccorrelation was handled differently in different hospitals and that some pa-thologist pairs exhibited marked disagreement in assessing if the error causewas sampling or interpretation. Raab and colleagues [37] found that this dis-agreement generally was based on variable assessments of specimen interpret-ability, defined as the combination of sample quality (eg, representativeness,obscuring factors) and the amount of tumor present. Pathologists disagreedon what constituted a good sample and the amount of tumor necessary torender a malignant diagnosis.

Raab and colleagues proposed using a new QC method for cytologic-histologic correlation, termed the ‘‘No Blame Box’’ (Fig. 1). The amount oftumor is depicted vertically, increasing from ‘‘no tumor’’ at the top of thebox to ‘‘abundant tumor’’ at the bottom. A specimen that contains many can-cer cells would be graded on the lower portion of the vertical axis and a spec-imen that contains only rare, questionable cancer cells would be gradedhigher on the vertical axis. The specimen quality is depicted horizontally,increasing from a poor quality specimen at the left to an excellent qualityspecimen at the right. Specimen quality relates to pre-analytic and analyticprocesses. The four squares of the No Blame Box divide specimens into com-binations of cancer/no cancer and good quality/poor quality. Completing theNo Blame Box generally illustrates that the error cause is multifactorial.

Quality of specimen

A

Poor quality specimen

No tumor identified

B

Excellent quality specimen

No tumor identified

Amount of tumor

C

Poor quality specimen

Tumor identified

D

Excellent quality specimen

Tumor identified

Fig. 1. ‘‘No Blame Box’’ for root cause analysis.


Summary

The study of anatomic pathology quality and patient safety is ongoing,and currently, much effort involves defining and measuring error. Datathat link error to patient outcome are critical for developing QI initiativestargeting errors that cause patient harm. Using methods of root cause anal-ysis beyond those traditionally used in cytologic-histologic correlation alsoassists in developing error reduction and QI plans.

References

[1] Valenstein P. Qualitymanagement in clinical laboratories: promoting patient safety through

risk reduction and continuous improvement. Northfield (IL): CAP Press; 2005.

[2] Bozzo P. Implementing quality assurance. Chicago: ASCP Press; 1991.

[3] Ohno T. Toyota production system: beyond large-scale production. Portland (OR): Produc-

tivity Press; 1988.

[4] Womack JP, Jones DT, Roos D. The machine that changed the world: the story of lean

production. How Japan’s secret weapon in the global auto wars will revolutionize western

industry. New York: Rawson Associates; 1990.

[5] ChaliceRW. Stop rising healthcare costs using Toyota lean productionmethods: 38 steps for

improvement. Milwaukee (WI): Quality Press; 2005.

[6] Wennberg JE, Gittelsohn AM. Small area variations in health care delivery. Science 1973;

183:1102–8.

[7] McPherson K, Wennberg JE, Hoving OB, et al. Small-area variations in the use of common

surgical procedures: an international comparison of New England, England, and Norway.

N Engl J Med 1982;307:1310–40.

[8] Carlisle DM, Valdez RB, Shapiro MF, et al. Geographic variation in rates of selected

surgical procedures within Los Angeles County. Health Serv Res 1995;30:27–42.

[9] ChassinMR,BrookRH,ParkRE, et al. Variations in the use ofmedical and surgical services

by the Medicare population. N Engl J Med 1986;314:285–90.

[10] Condel JL, Sharbaugh DT, Raab SS. Error-free pathology: applying lean production

methods to anatomic pathology. Clin Lab Med 2004;24:865–99.

[11] Zarbo RJ, Meier FA, Raab SS. Error detection in anatomic pathology. Arch Pathol Lab

Med 2005;129:1237–45.

[12] Novis DA. Detecting and preventing the occurrence of errors in the practices of laboratory

medicine and anatomic pathology: 15 years’ experience with the College of American

Pathologists’ Q-Probes and Q-Tracks programs. Clin Lab Med 2004;24:965–78.

[13] Novis DA, Walsh MK, Dale JC, et al. Continuous monitoring of stat and routine outlier

turnaround times: twoCollege ofAmericanPathologistsQ-Tracksmonitors in 291 hospitals.

Arch Pathol Lab Med 2004;128:621–6.

[14] Dale JC, Novic DA,Meier FA. Reference laboratory telephone service quality. Arch Pathol

Lab Med 2001;125:608–12.

[15] Novis DA, Zarbo RJ, Saladino AJ. Interinstitutional comparison of surgical biopsy diagno-

sis turnaround time: a College of American Pathologists Q-Probes study of 5384 surgical

biopsies in 157 small hospitals. Arch Pathol Lab Med 1998;122:951–6.

[16] Zarbo RJ, D’Angelo R. Transforming to a quality culture: the Henry Ford production

system. Am J Clin Pathol 2006;129(Suppl 1):S21–9.

[17] Kohn LT, Corrigan JM, Donaldson MS. To err is human: building a safer health system.

Washington, DC: National Academy Press; 1999.

[18] Raab SS, Grzybicki DM, Janosky JE, et al. Clinical impact and frequency of anatomic

pathology errors in cancer diagnosis. Cancer 2005;104:2205–13.


[19] Raab SS. Subcategorization of Papanicolaou tests diagnosed as atypical squamous cells of

undetermined significance. Am J Clin Pathol 2001;116:631–4.

[20] Lundberg GD. Acting on significant laboratory results. JAMA 1981;245:1762–3.

[21] StroobantsAK,GoldschmidtHM, PlebaniM. Error budget calculations in laboratorymed-

icine: linking the concepts of biological variation and allowable medical errors. Clin Chim

Acta 2003;333:169–76.

[22] Grzybicki DM, Gross T, Geisinger KR, et al. Estimation of performance and sequential

selection of diagnostic tests in patients with lung lesions suspicious for cancer. Arch Pathol

Lab Med 2002;126:19–27.

[23] Grzybicki DM. Laboratory specimen identification detection. 2007 Academy Health Na-

tional Meeting. Available at: http://www.academyhealth.org. Accessed July 30, 2007.

[24] Raab SS, Nakhleh RE, Ruby SG. Patient safety in anatomic pathology: measuring discrep-

ancy frequencies and causes. Arch Pathol Lab Med 2005;129:459–66.

[25] Raab SS. Improving patient safety by examining pathology errors. Clin Lab Med 2004;24:

849–63.

[26] Raab SS. Improving patient safety through quality assurance. Arch Pathol Lab Med 2006;

130:633–7.

[27] Vrbin CM, Grzybicki DM, Zaleski MS, et al. Variability in cytologic-histologic correlation

practices and implications on patient safety. Arch Pathol Lab Med 2005;129:893–8.

[28] Landis JR,KochGG. Themeasurement of observer agreement for categorical data. Biomet-

rics 1977;33:159–74.

[29] Llewellyn H. Observer variation, dysplasia grading, and HPV typing: a review. Am J Clin

Pathol 2000;114:S21–35.

[30] Reason J. Human error: models and management. BMJ 2000;320:768–70.

[31] Aspden P, Corrigan J, Wolcott J, et al. Patient safety: achieving a new standard of care.

Washington, DC: National Academies Press; 2003.

[32] Kaplan HS, Battles JB, Van der Schaaf TW, et al. Identification and classification of the

causes of events in transfusion medicine. Transfusion 1998;38(11–12):1071–81.

[33] Simmons D. Sedation and patient safety. Crit Care Nurs Clin North Am 2005;17:279–85.

[34] Raab SS, Vrbin CM, Grzybicki DM, et al. Errors in thyroid gland fine needle aspiration.

Am J Clin Pathol 2006;125:873–82.

[35] Nodit L, Balassanian R, Sudilovsky D, et al. Improving the quality of cytology diagnosis:

root cause analysis for errors in bronchial washing and brushing specimens. Am J Clin

Pathol 2005;124:883–93.

[36] Raab SS,Meier FA, ZarboRJ, et al. The ‘‘Big Dog’’ effect: variability in assessing the causes

of error in patients with lung cancer. J Clin Oncol 2006;24:2808–14.

[37] Raab SS, Stone CH, Wojcik EM, et al. Use of a new method in reaching consensus on the

cause of cytologic-histologic correlation discrepancy. Am J Clin Pathol 2006;126:836–42.

http://www.academyhealth.org

measuring quality in anatomic pathology

Documents