Effects of structured template reporting on the quality of

radiology reports

Juan Carlos Batlle, MDGiles W Boland, MD

Daniel I Rosenthal, MD

Partners Research Retreat March 24, 2009

Originally presented at RSNA December 2, 2008

Background

Structured reporting increasingly used as a means of systematizing reports and making them clinician and “computer” friendly Stillman 2008 JACR 51; Taira 2001 Radiographics 21

Transition from free dictation to structured reporting will involve structured templates for free dictation

Effects of templates on reporting are uncertain

Purpose

To evaluate surrogate objective metrics for assessment of text quality of radiology reports

To assess the change in text quality of reports after a department-wide transition to automatic structured templates issued by the department

Brief Methods 1

In early 2007, MGH Radiology transitioned to Commissure platform (Nuance Communications) voice-recognition dictation software

Previous dictation software allowed personal templates (“macros”) used in addition to free-dictation

Commissure allows automatic “population” of a structured template into the dictation upon opening the patient exam, according to exam code (e.g. CTABP+, CTCH+, etc.).

Structured templates for each exam code created by division committee

Brief Methods 2

OLAP (Online Analytical Processing) Cube used to extract dictations of all abdominopelvic CTs interpreted by three randomly-selected radiologists from 2006 and 2007 (n=9594) into Excel.

Comparison was made of the text of the reports before (n=5100) and after (n=4494) a transition to automatically populated structured templates.

Text was analyzed for word count, sentence structure, and Flesch Reading Ease by semi-automated computer software.

Flesch Reading Ease rates text on a 100-point scale based on the average number of syllables per word and words per sentence.

Results

After the transition to structured reports: Significant decrease in word count (292 to 226

words, p<0.001) Significant increase in number of words per

sentence (4.8 to 5.0, p<0.001) Significant worsening in Flesch reading ease

score (30.6 to 24.6, p<0.001)

Word Count Significantly Decreased After Transition

Readability Significantly Decreased After Transition

Follow-up Data

Preliminary assessment of two randomly selected radiologists from the thoracic imaging division: Attending 1

Word count decrease from 266 to 231 Words/sentence increase from 4.9 to 5.4

Attending 2 Word count decrease from 246 to 232 Words/sentence increase from 5.2 to 5.5

Limitations

Small group of radiologists studied, though large sample size

Metrics of text “quality” are imperfect surrogates for true quality measurement

Conclusions

Implementation of structured templates led to significant changes as measured in the text of dictations.

This may significantly influence overall readability of reports and may be unanticipated

Future attention should be paid to optimizing structured templates according to surrogate metrics of text quality

Ongoing assessment of these metrics may be performed easily using OLAP cube software