prelimcheck: software to facilitate follow-up of resident preliminary reports phillip cheng, md ms...

PrelimCheck: Software to Facilitate Follow-Up of

Resident Preliminary Reports

Phillip Cheng, MD MSAssistant Clinical Professor of Radiology

Objective

The purpose of this exhibit is to illustrate an open source desktop tool for follow-up of resident preliminary notes, allowing residents to rapidly evaluate their notes for discrepancies compared to final dictated reports.

Patient and study identifiers shown in this presentation have been obscured, altered or synthesized.

Introduction

• At our teaching institution, radiology residents begin taking independent call at night in their second year of training.

• During the night, their preliminary interpretations are recorded as notes in the PACS (Fuji Synapse). Typically 50-100 notes are placed by a resident per call night.

• The attending dictates the final reports in the morning and reviews preliminary notes by the resident.

Resident Notes in PACS

Resident Notes in PACS

Resident preliminary note at night

Attending note in the morning, resident now post-call at home

A Feedback Problem

• Independent call experience is considered by our residents and faculty as one of the most important educational experiences of their residency.

• However, feedback is an essential part of this education, and is not immediately available since the resident leaves in the morning before the attendings dictate final reads for the overnight studies.

• In particular, there is no mechanism in the PACS for discrepancies between preliminary and final reads to be flagged for later review.

Reviewing Discrepancies

We have reviewed several unsatisfactory options for reviewing overnight discrepancies with residents:

• Morning readout with attending: post-call resident is typically tired and not receptive to feedback

• Attending e-mail to residents: requires extra effort by attending especially for minor discrepancies, so such feedback is sporadic

• Residents reviewing own reads from a previous call night– Attractive since the residents are usually motivated to review

their call performance; however, identification and sorting of the preliminary and final reads from a call night is tedious and prone to incompleteness.

?PrelimCheck

PrelimCheck: Facilitating Feedback

Resident

Attending

Notes in PACS Notes in PACS,Dictated reports

PrelimCheck is a tool that completes the communication loop between attendings and residents.

PrelimCheck

• Open source, portable Windows desktop application

• Written in C#, requires .NET Framework 2.0

• Communicates directly with Fuji Synapse PACS

PrelimCheck - Usage

• Easy-to-use tool for residents to compare their preliminary notes with attending follow-up notes and final reports

• A resident specifies the location and time period of the call shift and clicks the Download button:

LocationTime Period

PrelimCheck - Usage

Download– Performs NTLM authentication to Synapse PACS server– SQL query over network for notes performed in

requested time period– Retrieve text of each note, and any dictated report for

the corresponding study– Typical download time for 12 hr shift: approx 1 minute

PrelimCheck - Usage

All preliminary notes for highlighted patient study

Dictated report, automatically scrolled to impression section

Sortable list of studies with notes

placed in the specified

time frame

Search list by keywords or

save to comma separated value

(CSV) file

PrelimCheck - Usage

• Requires approximately 5-10 minutes for review of overnight call cases

• No built-in intelligence to detect and flag discrepancies between preliminary and final report– Ease of rapid review partially obviates this

problem

Survey

• Use of the program is encouraged but optional in our residency

• One year after release, residents of all training levels were anonymously surveyed on usage, survey response rate 83%

Survey - Usage

Most respondents used the program. The main stated reason for non-usage was lack of familiarity with program, or lack of call experience.

Survey – Usability

Most respondents rated the program high in usability / ease of use.

Survey - Effectiveness

Despite lack of automated flagging of discrepancies, most respondents indicated that the program improved discrepancy detection.

Sample of Anonymous Comments

Limitations and Future Work

• Lack of automated discrepancy detection– Attendings can use standardized code words in notes to

accurately flag and rate discrepancies

• Slow network client-based implementation– Requires approximately 1 minute to download notes for a

call shift– Faster response possible with a server-side implementation

• Inability to filter results during network retrieval– Since notes are not indexed on PACS server, all notes

within time period are retrieved and then filtered on the client side

– Solvable with server-side implementation

Source Code Availability

• PrelimCheck source code is available under the Apache 2.0 open source license.

• All source available at Github:http://github.com/pmcheng/prelimcheck

Summary

PrelimCheck, an open source desktop application, facilitates rapid resident self-evaluation of on-call preliminary radiologic interpretations.

http://hsc.usc.edu/~phillimc/prelimcheck

phillip.cheng@usc.edu