business case for quality: best practices surrounding information technology stephen s. raab, m.d....

Business Case for Quality: Best Practices Surrounding

Information Technology

Stephen S. Raab, M.D.

Department of Pathology, University of Pittsburgh, Pittsburgh, PA

April 29, 2005

Background

Medical error is the failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim

Medical errors permeate all levels of patient care and the IOM report has resulted in the greater recognition of error and an increased focus on reporting and reducing errors

Background Scant data on anatomic pathology error

frequencies Institutional error frequencies may be low,

limiting hypothesis testing Limited data on how pathology errors affect

patient care Root cause analysis of anatomic pathology

errors rarely performed and results of these analyses are not disseminated

Error prevention Minimize psychological precursors;

design buffers; build in redundancy Design tasks: simplification;

constraints; standardization Systems designed to absorb error Root cause analysis Shift from training-focus to

performance-focus Data collection to define problem

Initiatives to decrease errors

Creation of audit systems Use of benchmarking and error tracking

systems Implementation of immediate error

reduction systems (e.g., Toyota Production Systems, Six Sigma)

Information technology and patient safety Most systems are focused on diagnostic

reporting Difficulty in performing quality

assurance Few checks to prevent errors Systems themselves are often a source

of error Few databases to study quality

Quality assurance

Cytologic-histologic (CH) correlation performed in all American labs (CLIA’ 88)

No national standards to perform CH correlation Information systems not structured to perform

cytologic correlation– Case pairs may be visualized prior or after sign-out– Most CH correlation performed with manual review

Cytologic-histologic correlation

What is the best way to perform correlation and how does one use the data?

Letter sent to 162 American labs requesting information on how they performed correlation (response frequency: 32.1%)

Separated material into forms, logs and tally sheets

CAP Checklist

Cytology case number Sign-out cytology diagnosis Sign-out cytologist Original cytotechnologist diagnosis Sign-out cytotechnologist Review cytology diagnosis Review cytologist

CAP Checklist

Surgical pathology case number Sign-out surgical pathology diagnosis Sign-out surgical pathologist Review surgical pathology diagnosis Review surgical pathologist Significance of discrepancy Action taken Reason for correlation

Minimum expected and additional variables listed on forms only. Bold line represents minimum expected variables that should be present (n = 15).

0

5

10

15

20

25

30

35

Received Form Material (n=31)

Nu

mb

er o

f It

ems

Additional VariablesMinimum Expected

UPMC technical quality assurance

Record laboratory error:– Accessioning– Gross groom– Histology– Cytology– Transcription

PUH Shy Total PUH Shy Total PUH Shy Total

ACC- Total 39 19 58 52 6 58 38 6 44

APLIS- Total 1 1 2 0 0 0 0 0 0

BILL- Total 2 0 2 1 0 1 1 0 1

CQI - Total 0 0 0 0 0 0 0 0 0

DICT- Total 41 37 78 46 13 59 61 39 100

FREE TEXT- Total 1 1 2 2 1 3 1 2 3

HISTO- Total 8 2 10 9 1 10 7 3 10

IHC- Total 1 0 1 1 0 1 1 0 1

PATH- Total 20 0 20 15 1 16 2 3 5

PROC- Total 14 0 14 3 0 3 2 1 3

QUAL- Total 12 5 17 12 11 23 14 7 21

REGIST- Total 35 1 36 28 0 28 55 1 56

REQ- Total 33 30 63 29 50 79 165 155 320

TRAN- Total 3 3 6 3 1 4 3 1 4

TUBE- Total 0 0 0 0 0 0 0 0 0

Total Specimens 6946 6275 6924

Total Part 9935 9038 10074

Total Blocks 13890 14146 15078

Total Stains 22231 20913 22217

J an Feb March

National benchmarking

CAP Q-Probes studies – error rates, turn around time, amended report rates

CAP Q-Tracks studies– Cytologic-histologic correlation– Frozen-permanent section review– Small surgical specimen turn around time

Second viewing of cases

CAP – Q-Probes study 2004 74 labs, 6186 specimens, 415 discrepancies

(6.7% rate) Breakdown of errors

– 48% with change in same category of diagnosis– 21% change in category of diagnosis– 18% typographical errors– 9% patient or specimen information– 4% change in margin status

Cytologic-histologic correlation

Year end report for participating labs (56) In 2003, a total of 19,478 Paps were correlated

with 11,336 true positive correlations and 2,433 false positive correlations

Predictive value for positive cytology: 82.3% Percent positive ASC diagnoses: 64.3% (range

34.1%-89.9%); percent positive AGC diagnoses: 31.4% (range 0%-66.7%)

Cytologic-histologic correlation

Best performers and clustering performed Best performers:

– Quarterly reports, track and trend– Summary reports to clinicians– Document correlation in report 5/9 labs– Keep written log of findings 2/9 labs– Review biopsy if discrepancy 3/9 labs

AHRQ national errors database

5 year project to monitor pathology errors by creation of a multi-institutional database

University of Pittsburgh, University of Iowa, Henry Ford Health System, Western Pennsylvania Hospital

Goal is to devise plans to reduce cytology and surgical pathology errors

Specific aims

1. Create voluntary, Web-based database and collect errors detected by correlation, secondary review, amended reports, frozen section review

2. Quantitatively analyze error data and generate performance improvement reports

3. Perform root cause analysis; plan and implement interventions to reduce errors

4. Assess success of interventions by quantitative measure; disseminate successful error reduction plans

Database challenges

Current absence of standardized and detailed laboratory workload and quality assurance data sets in widely used laboratory information systems

Current lack of efficient and comprehensive electronic de-identification of un-linked institutional laboratory information system and clinical data

Database construction

The database is Oracle 9.2.0.4 Enterprise Edition implemented on a Sun Ultra E450 Server running Solaris 2.9. The mid-tier is implemented with Oracle’s Application Server (v9.0.3) on a Compaq DL360 Server running Windows 2000. The application uses the Oracle http server and mod_pl/sql extensions to generate dynamic web pages from the database to the users via a Microsoft Internet Explorer web browser version 6.0 or higher.

Logical system design Schema layer contains the actual data and data

relations that are stored entirely as numbers and keys

Meta-data layer, the data is defined in terms of data elements and “data objects”

Procedure layer contains a set of dynamic procedures/functions (in PL/SQL) that externalize the data elements

Presentation layer is a series of forms (for data entry, display, query, etc.) that are populated by data elements from meta-data

Future database plans

Database interface to laboratory information systems

Data de-identification Revision of database architecture Incorporation of histopathologic image data

Variable frequency of errors across sites

Institution Number of errors

Total number of

cases

Error percent

A 196 28,396 0.69

B 279 66,115 0.42

C 796 143,835 0.55

D 174 82,787 0.21

Correlating case error frequency

Error frequency

Site Gyn correlating % Non-gyn correlating %

A 9.49 11.03

B 1.65 5.86

C 4.72 11.72

D 3.33 6.14

Variability in assessing sampling versus interpretive error

Institution

Reason A B C D Total

Interpretation 21% 11% 65% 4% 33%

Screening 2% 8% 4% 2% 5%

Sampling 61% 73% 38% 92% 60%

Unknown 18% 9% 0 3% 6%

Agreement between original and review reason for discrepancy (interpretation or sampling)

Review reason

Site A B C D

Original reason

A 0.615 0.412 0.024 0.286

B 0.737 0.211 0.118 0,737

C 0.400 0.545 0.615 0.40

D Und Und Und Und

Percentage of total error by specimen type

Institution

Organ A B C D Total

Lung 52% 20% 35% 17% 34%

Thyroid 0% 5% 8% 2% 6%

Bladder 22% 14% 16% 33% 18%

Breast 2% 2% 5% 17% 5%

Error outcome taxonomy

Significant event: error that affects patient outcome; may be classified by severity (mild, moderate, and severe)

No harm event: error does not affect patient outcome

Near miss event: intervention occurred before harm could take place

Differences in grading of discrepancy assignments

Institution

Assignment A B C D Total

Significant event

0 0 28% 14% 22%

No Harm 86% 98% 32% 50% 44%

Near miss 14% 0 40% 14% 32%

Unknown 0 2% 0 21% 2%