the clinical impact of subspecialized radiologist reinterpretation of abdominal imaging studies,...

The clinical impact of subspecialized radiologistreinterpretation of abdominal imaging studies,with analysis of the types and relative frequencyof interpretation discrepancies

Elizabeth A. Lindgren,1 Maitray D. Patel,2 Qing Wu,2 Jeff Melikian,2 Amy K. Hara2

1Mayo Clinic Rochester, 200 First Street SW, Rochester, MN 55905, USA2Department of Radiology, Mayo Clinic Arizona, 5777 E. Mayo Blvd, Phoenix, AZ 85054, USA

Abstract

Purpose: The primary objective of this study was todetermine the clinical impact and value of abdominalimaging reinterpretations by subspecialized abdominalimagers.Methods: Secondary interpretations for computedtomography (CT), magnetic resonance (MR), and ultra-sound (US) abdominal studies performed outside ourinstitution over a 7-month period were retrospectivelycompared to the primary (outside) interpretation, withinterpretive differences recorded. Clinical notes, pathol-ogy and subsequent imaging determined ground truthdiagnosis and the clinical impact of any interpretivediscrepancies were graded as having high, medium, orlittle/no clinical impact. Interpretive comparisons werescored into categories: (1) no difference; (2) incidentalfindings of no clinical impact; (3) finding not reported;(4) significance of finding undercalled; (5) significance offinding overcalled; (6) finding misinterpreted; and (7)multiple discrepancy types in one report.Results: 398 report comparisons were reviewed on 380patients. There were 300 CT, 60 MR, and 38 USexaminations. The primary report had 5.0% (20/398)high clinical impact interpretive discrepancies and 7.5%

(30/398) medium clinical impact discrepancies. Thesubspecialized secondary report had no high clinicalimpact discrepancies and 8/398 (2.0%) medium clinicalimpact discrepancies. In order of frequency, high andmedium impact discrepancies in the primary reportconsisted of 50% overcalls, 26% unreported findings,18% undercalls, 4% misinterpretations, and 2% multiplediscrepancies.

Conclusions: Subspecialty review of abdominal imagingexams can provide clinical benefit. Half of the discrep-ancies in this series of abdominal reinterpretations weredue to overcalls.

Key words: Secondary interpretation—Peerreview—Subspecialization—Over read—Discrepancies

Diagnostic disagreement between two radiologists read-ing the same study is not new, as evidenced by Garland’sPresidential Address at the 1949 Radiological Society ofNorth America (RSNA) annual meeting [1]. While theseinterpretive discrepancies are often considerations whendeveloping performance benchmarks and quality assur-ance programs [2], they also are highly relevant in theevolution of radiologist subspecialization, reflecting theincreasing role of medical tertiary care centers. Patientsseeking specialist clinical care often already have hadclinically relevant imaging studies performed [3–5]. Thepractice of official radiology reinterpretation originallybegan because third-party payers were reluctant orunwilling to reimburse for a radiologic examination if thesame study had been recently done at an outside institu-tion [6]. The rationale for reinterpretation has evolved toreflect other considerations, including: (1) increased pro-vider confidence in a radiology report when that providerknows the interpreting radiologist, (2) potentially im-proved diagnostic performance by subspecialist exami-nation interpretations, (3) radiation dose and cost savingsby reinterpreting a previously performed exam rather thanrepeating the study, and (4) practice-specific (and some-timesmedicolegal) considerations when importing outsideimaging studies into an institution’s imaging repository.Correspondence to: Maitray D. Patel; email: [email protected]

ª Springer Science+Business Media New York 2014

AbdominalImaging

Abdom Imaging (2014)

DOI: 10.1007/s00261-014-0140-y

The practice of subspecialized radiologist reinterpre-tation of imaging studies requires radiology departmen-tal resources and can increase costs to patients, insurers,and health care systems [7]. In fact, the whole enterpriseof radiology subspecialization can increase systemichealth care costs as radiologists train longer, depart-ments become bigger, and after hours coverage becomesmore complicated. It becomes important, therefore, tobroadly understand and document the benefits of asubspecialized radiology practice as well as analyzeoptimal subspecialization workflows.

The primary aim of this study was to determine theclinical impact and value of abdominal imaging reinter-pretations by subspecialized abdominal imagers in oneinstitution. Our secondary aim was to better understandthe types and relative frequency of interpretive errorsmade by non-subspecialized as well as subspecializedradiologists.

Materials and methods

The institutional review board of our institution deemed itunnecessary to require patient informed consent for thisretrospective study, which was compliant with the HealthInsurance Portability and Accountability Act (HIPAA).

At our institution, providers can request a formalreinterpretation of radiologic examinations performed atother institutions. Providers do not request reinterpre-tations of all outside examinations because this can leadto additional cost if no comparison examination exists atour institution. When prior outside examinations arereviewed in the context of a more recent or new imagingstudy performed at our institution, the comparison isconsidered part of standard care, is discussed in the re-port of the study performed at our institution either atthe time of initial dictation or in an addendum, does notgenerate a separate comparison report, and does notgenerate any additional charge. Review of these outsideimages not generating a unique reinterpretation reportfell outside the parameters of this study. When reinter-pretation of outside imaging with a unique reinterpre-tation report is requested, the examination is loaded intoour institutional Picture Archiving and CommunicationSystem if possible; if not, they are loaded into an insti-tutionally developed image viewer that is accessible ondesktop computers. The outside radiology reports arerequested and are scanned into the electronic medicalrecord when available. The fraction of cases with avail-able reports and the degree to which those reports werereviewed by the subspecialized radiologist could not beascertained in this retrospective study. During the studyperiod, trainees were not involved in these reinterpreta-tion requests; all cases were reviewed, interpreted, anddictated solely by the subspecialized radiologist. Thesecondary reinterpretation performed at our institutionbecomes part of the patient’s permanent medical record.

The Radiology Information Management System(RIMS) at our institution was queried to identify alloutside abdominal imaging reinterpretations performedby our department between 1/1/2010 and 7/15/2010. Allabdomen and pelvic computed tomography (CT), mag-netic resonance (MR), and digital radiograph secondaryinterpretations were performed by abdominal imagingfellowship-trained, board-certified staff radiologists ofthe Abdominal Division (13 individual radiologists, 10–27 years of post-residency experience). Outside abdomi-nal and/or pelvic ultrasound (US) studies were reinter-preted by members of the Ultrasound Division (5individual radiologists, 12–20 years of post-residencyexperience), consisting of radiologists with Ultrasoundfellowship training or Abdominal Imaging fellowshiptraining with subsequent 10-year or greater experience asa member of the Ultrasound Division. When a patienthad both a CT and MR examination reviewed by thesame Abdominal Division radiologist (as occurred in fivecases), these were considered as one single episode ofabdominal imaging reinterpretation of an MR exami-nation. The primary radiologist was defined as thereferring/outside institution radiologist. The extent andtype of subspecialty training of the primary radiologistwere unknown. The secondary radiologist was defined asour department’s subspecialized radiologist.

For all episodes of abdominal imaging reinterpreta-tion, the clinical indication for the reinterpretation wascategorized (by author EL) into one of three categories:(1) ‘‘acute’’: patient acute process or symptoms (such asacute abdominal pain or new fever); (2) ‘‘cancer’’: sus-pected or known malignancy (such as a patient with aliver mass identified on outside imaging); or (3) ‘‘non-acute’’: all other presentations (such as subacute orchronic pain, or microscopic hematuria).

The primary and secondary interpretation radiologyreports were compared and scored into one of sevendiscrepancy categories. In this study, when the interpre-tation between the primary and secondary radiologistdiffered, a discrepancy was ‘‘assigned’’ to the interpre-tation which was incorrect when compared to the out-come. If the two interpretations were similar, thecomparison was scored as showing no difference (dis-crepancy category 1) and no discrepancy (mistake) wasassigned to either the primary radiologist or secondaryradiologist. When the two interpretations were different,the clinical record was reviewed to determine groundtruth using subsequent imaging, pathology, and clinicalcourse. If the two interpretations differed only withrespect to incidental findings unrelated to the outcome, itwas scored as discrepancy category 2 and no discrepancywas assigned to either radiologist. Otherwise, the com-parison was scored with one of five additional discrep-ancy categories with a discrepancy assigned to either theprimary or secondary radiologist based on the nature ofthe suboptimal interpretation: (3) finding not reported;

E. A. Lindgren et al.: The clinical impact of abdominal imaging studies

(4) undercall or hedge (significance of a finding is givenless importance than ground truth); (5) overcall or hedge(significance of a finding is given more importance thanground truth); (6) miscall (finding is recognized and de-scribed but misdiagnosed); and (7) multiple discrepancytypes by the same radiologist.

Discrepancies were further classified according toclinical impact into one of three categories: high clinicalimpact, medium clinical impact, or low/no clinical im-pact (Table 1). High clinical impact discrepancies werethose that led or could have led to an incorrect treatmentplan, had the recommendations arising from the inter-pretation been pursued. Medium clinical impact dis-crepancies were those that either led or could have led tominor delays in treatment or unnecessary additionaltests. Discrepancies classified as low/no clinical impactdid not affect patient care. The number and type ofadditional imaging tests resulting from both primary andsecondary interpretations were recorded.

The methodology for identifying discrepancies wasdesigned to minimize bias. All cases were initially re-viewed by author EL to identify cases with no differencesin reports (discrepancy category 1); these were not fur-ther reviewed. Subsequently, all cases marked as poten-tially having a difference in report by author EL werethen independently reviewed by author AH to furthersubdivide the cases by discrepancy category and assignclinical impact scores. Cases placed into category 2(incidental) were not further reviewed; all cases in dis-crepancy category 3–7 were then additionally reviewedby author MP. Any cases with classification disagree-ments between author AH and author MP were resolvedby consensus.

Statistics

McNemar’s test was used to assess the statistical signif-icance of the difference in rates of discrepancies betweenprimary and secondary interpreters. McNemar’s test wasalso used to assess the statistical significance of differ-ences in the rates of discrepancies based on type ofimaging. A p value of <0.05 was considered statisticallysignificant.

Results

There were 402 episodes of abdominal imaging reinter-pretation performed on 380 patients (224 women and 156

men), with an average age of 57.6 years (range 18–95 years). Since there were so few conventional radio-graphs reviewed without other examinations, these threeepisodes of reinterpretation were excluded from furtheranalysis. In one case with a potentially high clinical im-pact difference, ground truth could not be determinedbecause the patient did not have follow-up at our insti-tution; this case was excluded from the data set. Theremaining 398 reinterpretations were for 300 CT (75.4%),60 MR (15.0%), and 38 US examinations (9.5%). Theclinical indications for the review were categorized asacute for 17 cases, cancer for 136 cases, and non-acutefor 245 cases. Due to the nature of our referral practice,there were no abdominal imaging studies reviewed re-lated to acute trauma.

The vast majority of comparisons showed either nointerpretive differences (244/398, 61.3%) or interpretivedifferences of little/no clinical impact (96/398, 24.1%)(Table 1). In the remaining 58 cases (58/398, 14.6%) withmedium or high clinical impact interpretive discrepan-cies, 29 were classified as overcalls (50.0%), 14 wereclassified as unreported findings (24.1%), 11 wereclassified as undercalls (19.0%), 3 were classified asmisdiagnoses (5.1%), and 1 had a combination of dis-crepancies (1.7%).

Primary radiologist interpretive discrepancies

Primary interpretation high clinical impact discrepanciesoccurred in 20 cases (20/398, 5.0%) and medium clinicalimpact discrepancies occurred in 30 cases (30/398, 7.5%),as illustrated by representative cases in Figures 1, 2, 3, 4,and 5. The 50 cases with primary interpretation mediumor high clinical impact discrepancies consisted of 5medium impact and 3 high impact discrepancies in the 38US cases (21.1%), 22 medium impact and 13 high impactdiscrepancies in the 300 CT cases (11.6%), and 3 mediumimpact and 3 high impact discrepancies in the 60 MRcases (10.0%). The difference in discrepancy rates be-tween imaging modalities was not statistically significant.Of the 50 primary radiologist medium or high clinicalimpact discrepancies, findings were overcalled in 25(50%), not reported in 13 cases (26%), undercalled in 9(18%), miscalled in 2 (4%), or had multiple discrepanciesin 1 (2%) (Table 2). Primary radiologists had higher ratesof clinically impactful unreported findings and undercallsin patients with suspected or known malignancy (13/136,

Table 1. Summary of discrepancies and type

Nodiscrepancy

High impact discrepancyby primary reader

Medium impactdiscrepancyby primary reader

Low impactdiscrepancyor unimportant incidental

Medium impactdiscrepancyby secondary reader

High impactdiscrepancyby secondary reader

244/398 (61.3%)

20/398 (5.0%)� 30/398 (7.5%)� 96/398 (24.1%) 8/398 (2.0%) 0 (0%)

� Statistically significant using McNemar’s test (p < 0.0001)


9.6%) compared to patients with other clinical presen-tations (9/262, 3.4%).

Secondary radiologist interpretive discrepancies

Discrepancies assigned to the secondary (subspecialty)radiologist occurred in eight cases (2.0%) and all were

medium clinical impact; there were no high clinicalimpact discrepancies (Fig. 1). The medium impact sec-ondary radiologist discrepancies were made on 6 of the300 CT examinations (2.0%), 1 of the 60 MR examina-tions (1.7%), and 1 of the 38 US cases (2.6%). Of theeight secondary radiologist medium clinical impactdiscrepancies, findings were not reported in 1 (12.5%),

Fig. 1. A 82-year-old man with high clinical impact primaryinterpretation error due to overcall. Coronal (A) and axial (B) IVcontrast-enhanced CT images show changes in the left midkidney related to prior thermal ablation (arrows). Primaryinterpretation: ‘‘Since it contrast enhances, it is not thought to

be due to postoperative change. Perhaps some type of residualor recurrent tumor is present here.’’ Subspecialty reinterpre-tation: ‘‘Postoperative changes from ablation of a left interpolarrenal mass. No evidence for recurrent tumor.’’ There was noindication of recurrent tumor on clinical follow-up.

Fig. 2. A 50-year-old man with high clinical impact primaryinterpretation error due to an undercall. Axial (A) and coronal(B) non-contrast CT images demonstrate a mass along theright iliopsoas muscle (arrows). Primary interpretation: ‘‘Fatty-containing lesion within the right iliopsoas tendon…suggestive

that this represents a lipoma rather than a liposarcoma.’’Subspecialty reinterpretation: ‘‘Fat containing lesion withsome possible soft tissue elements…worrisome for a lipo-sarcoma.’’ Surgical pathology revealed an atypical lipomatoustumor/well-differentiated lipoma-like liposarcoma.


undercalled in 2 (25.0%), overcalled in 4 (50.0%), andmiscalled in 1 (12.5%). The difference in error rates be-tween primary radiologists and secondary radiologistswas statistically significant (p < 0.0001).

Discussion

The existing literature regarding discrepancies betweenprimary radiologist interpretations and secondary radi-ologist interpretations can be broadly grouped into threemain categories: (1) primary interpretations by generalradiologists, non-subspecialist radiologists, or radiolo-gists of unknown specialization and secondary interpre-tations by subspecialists [3–6, 8–15]; (2) primaryinterpretations by residents and secondary interpreta-tions by faculty [16–36]; and (3) peer review and doublereading studies between radiologists without subspecial-ization differences [1, 2, 37, 38]. Within the first group,investigations have focused on subspecialty reinterpre-tations of neuroradiology studies [5, 12–14], abdominalimaging studies (both adult and pediatric) [4, 6, 8, 9, 39],

Fig. 3. A 53-year-old woman with high clinical impact pri-mary interpretation error due to overcall. Oblique sagittaltransabdominal sonogram shows a cystic collection in the cul-de-sac delineated with electronic calipers. Primary interpre-tation: ‘‘Septated right pelvic cyst with maximum diameter of14.9 cm likely arising from the right ovary. The possibility of acystic neoplasm cannot be excluded.’’ Subspecialized rein-terpretation: ‘‘Large cystic collection in the cul-de-sac whichappears to be outside the ovary. This is probably a peritonealinclusion cyst.’’ A peritoneal inclusion cyst was encounteredat surgery.

Fig. 4. A 60-year-old woman with high clinical impact pri-mary interpretation error due to undercall. Oblique sagittaltransvaginal sonogram shows a cystic and solid adnexalmass demarcated by electronic calipers. Primary interpreta-tion: ‘‘Two right ovarian cysts without internal echoes…thecysts appear to be simple cysts.’’ Subspecialty reinterpreta-tion: ‘‘There is a mixed cystic and solid area marked as theright ovary measuring 5.6 9 2.9 9 4.2 cm…cystic areas[have] intervening thick septations.’’ Pathology at surgicalremoval showed high grade papillary serous carcinoma.

Fig. 5. A 70-year-old man with medium clinical impact pri-mary interpretation error due to overcall. Axial CT image withoral contrast shows density in the presacral region in a patientwith prior colonic resection (arrow). Primary interpretation:‘‘1.5 9 2.2 cm presacral soft tissue…neoplasm is not ex-cluded.’’ Subspecialty reinterpretation: ‘‘There are postoper-ative changes at the coloanal region. No bowel obstruction,mass or extravasation is identified.’’ Subsequent endoscopicsonographic evaluation showed no evidence of recurrenttumor.


thoracic imaging studies [11], oncologic imaging studies[4, 6, 15], and emergency imaging studies (often traumarelated) [3, 10, 14]. Major interpretative differences inthese studies ranged from 10% to 33%.

The goal of our study was to determine the clinicalimpact and value of abdominal imaging reinterpretationsby subspecialized abdominal imagers, with the special-ization status of the primary interpreter unknown; assuch, it is comparable to the first group of existing lit-erature described above. Our results show that 12.5% ofthe reinterpreted cases had medium or high clinical im-pact discrepancies (7.5% and 5%, respectively) whencompared to the original report. This rate of clinicallymeaningful discrepancies puts it at the bottom of therange described in previous studies, but there areimportant differences in patient population and meth-odology that account for this. In a study of pancreaticcancer resectability, Kalbhen et al. [6] found major dis-crepancies in 17 of 53 patients (32%) thought to beresectable in the primary report but unresectable by thesubspecialist reinterpretation. In another cohort of 78patients referred for suspected pancreatic or hepatobil-iary malignancy, Tilleman et al. [4] found that subspe-cialist reinterpretation identified major discordance in 8patients (10.3%) and minor discordance in 11 patients(14.1%). Reviewing body CT studies in 143 patients withbiopsy-proven cancer, Gollub et al. [15] found majordisagreement with the primary report in 24 patients(17%) and minor disagreement in 29 patients (20%). In531 consecutive trauma patients with abdomen and pel-vic CT examinations, Yoon et al. [8] found discrepancieswith the primary report in 153 cases (29.9%), with 12 ofthese (7.8%) resulting in changes in patient care. Finally,in 305 pediatric body CT cases reviewed by pediatricabdominal imaging subspecialists, with more than half ofthese related to clinical concern for appendicitis, Eakinset al. [9] found major discrepancies in 99 (32.6%) andminor discrepancies in 57 (18.7%). In these five previousinvestigations, which constitute most of the previousdata regarding abdominal subspecialist review of non-trainee primary interpretations, the high prevalence ofdisease magnifies the effect of subspecialty expertise byreducing or eliminating less difficult cases in which errors(and thus discrepancies) would be unlikely. It should benoted that the specialization status of the primaryinterpreter in those studies was unknown, as in ours,although it is reasonable to assume that many of the

primary interpretations in our study were performed byradiologists not exclusively interpreting abdominalimaging studies in practice (i.e., not subspecialized to theextent found in the secondary radiologists) since thepatients were seeking tertiary care at our center, notpursuing tertiary care at their original medical facility.

These differences in patient population likely accountfor the lower rate of discrepancies when reviewing pa-tients at lower risk for disease. This is corroborated bythe study conducted by Bell et al., who reported thatsubspecialty review of 1303 consecutive after hoursabdominal imaging studies initially interpreted by board-certified or board-eligible radiologists who were notsubspecialized in abdominal imaging resulted in identi-fication of 44 (3.4%) clinically relevant discrepancies [39];this is substantially lower than the 12.5% rate of com-bined medium and high clinical impact discrepanciesidentified in our study. The important difference is thatreferring providers had some reason to request that weperform the reinterpretation in our current study,whereas in the study by Bell all cases were reviewedregardless of indication. Presumably, the outsideabdominal imaging examinations of patients with lowpretest probability of disease were ‘‘filtered out’’ by ourreferring providers who did not request review of thosestudies.

Clearly, the pretest probability of disease will influ-ence not only the rate of discrepancies found with sub-specialized reinterpretation, but also the type ofdiscrepancy; for example, at one extreme, if all studieswere straightforward normal exams performed on pa-tients without disease, one would expect a very low rateof discrepancy, and any discrepancy found would be anovercall. At the other extreme, if all studies reviewedwere performed on patients with high likelihood of dis-ease with complicated imaging findings (such as the priorstudies on patients with biopsy-proven cancer or trau-ma), one would expect higher rates of discrepancy (as hasbeen shown in those prior studies), with a variety of errortypes other than overcalls. It should come as no surprise,therefore, that discrepancy rates identified on variousstudies have differed because this reflects the differentpatient populations in those studies.

We believe understanding the most frequent types oferrors in the primary report can be useful to guideinstitutional policies and referring providers as to whichcases benefit most from subspecialized reinterpretation.

Table 2. Classification of primary interpretation combined high and medium clinical impact discrepancies based on clinical indication

Clinical indication Finding not reported Undercall Overcall Misdiagnosis Combination Total

Acute (n = 17) 0/17 (0%) 0/17 (0%) 2/17 (11.8%) 0/17 (0%) 0/17 (0%) 2/17 (11.8%)Cancer (n = 136) 6/136 (4.4%) 7/136 (5.1%) 6/136 (4.4%) 0/136 (0%) 0/136 (0%) 19/136 (14.0%)Non-acute (n = 245) 7/245 (2.9%) 2/245 (0.8%) 17/245 (6.9%) 2/245 (0.8%) 1/245 (0.4%) 29/245 (11.8%)Total (n = 398) 13/398 (3.3%) 9/398 (2.3%) 25/398 (6.3%) 2/398 (0.5%) 1/398 (0.3%) 50/398 (12.6%)


The majority of interpretive discrepancies in our studyconsisted of overcalling the significance of benign find-ings. These overcalls potentially lead to unnecessaryadditional tests or treatment. For example, in our study,a primary radiologist at an outside institution interpreteda cystic adnexal mass on pelvic ultrasound as having areasonable chance of being malignant (Fig. 3). Thesubspecialty radiologist at our institution interpreted thelesion on ultrasound as a peritoneal inclusion cyst,without need for surgical intervention. The patient, whowas seeking care at our institution specifically for thesurgical procedure that had been recommended by theprimary radiologist, elected to proceed with surgery,which confirmed a benign peritoneal inclusion cyst. Inthis instance, a surgical procedure was performedunnecessarily as a result of the primary interpretation.

On the other hand, our data shows it is more commonfor patients with known malignancy or findings poten-tially due to malignancy to have important findingsunrecognized or undercalled in the primary interpreta-tion by non-subspecialized radiologists. For example, inone patient with a history of prostate cancer with risingPSA, the primary interpretation of an abdominal CTstudy did not identify an abnormal pelvic lymph nodethat was recognized as the site of recurrent disease by thesubspecialized radiologist at secondary interpretation,allowing for the proper choice of treatment. In anothercase, the primary interpretation of an abdominal CTstudy misinterpreted changes in the pancreas as beinglikely due to pancreatitis, whereas subspecialized sec-ondary interpretation noted the presence of coexistingperipancreatic adenopathy not recognized or described inthe primary report, which prompted a biopsy provingpancreatic adenocarcinoma.

For abdominal imaging studies, recognizing thatovercalls are a common error in patients at low risk fordisease, understanding that other types of discrepanciesoften occur in the setting of cases that have positivefindings described, and knowing that the existing litera-ture indicates most clinically meaningful discrepancies inabdominal imaging interpretation occur in patients athigh risk for disease with cancer, trauma, and infection,helps to shape future approaches to subspecialist inter-pretations for abdominal studies. One reasonable ap-proach is to encourage abdominal subspecialtyreinterpretation if any potentially important finding isreported or in patients with known or suspected cancer,trauma, and infection. Routine subspecialty reinterpre-tation of abdominal imaging studies in patients with lowclinical suspicion for imaging-relevant disease and with-out any described findings in the primary interpretationwould be expected to be less valuable.

One reason for adopting this selective approach is tomaximize the cost–benefit ratio of performing subspe-cialty reinterpretation. Clearly, there is financial benefitto not performing erroneously recommended testing or

therapeutic interventions when an overcall is made.Furthermore, there are harder to quantify quality bene-fits that accrue when patients receive prompt appropriatecare, even if doing so may not actually save patient lives(as in the example of the pancreatic cancer previouslydescribed). These must be balanced by the increasedeffort and health care system costs that come with sub-specialty reinterpretation [7]. It is not firmly establishedwhere the balancing fulcrum lies in the spectrum ofpatient presentations that would benefit from abdominalimaging reinterpretation, but our study adds to theavailable data to help create guidelines.

Our study has several limitations. The study design isretrospective, which can introduce bias. The primaryradiologist may not have had the benefit of the samemedical history or physical examination findings as didthe secondary radiologist. By virtue of elapsed timebetween the date of the examination and the subspecialtyreinterpretation, secondary radiologists may have hadclues as to the significance or lack of significance ofreported findings. There is no method by which our studycould determine if the subspecialist reinterpreting theexamination reviewed the primary report. Our study onlyincluded those ‘‘official’’ secondary interpretations thatwere provided by the subspecialty radiologist and did notinclude unofficial secondary consults (‘‘curbside con-sults’’). As noted previously, only some of the abdominalimaging studies performed on patients seen by ourreferring providers were submitted for reinterpretation,so cases were not randomized. The classification ofclinical impact is subjective and is subject to interob-server variability. Finally, the subspecialization trainingand experience of the primary radiologists are unknown.

Additional questions remain to be answered. To whatdegree can abdominal subspecialists predict from clinicalhistory and/or review of the primary report whether ornot subspecialty reinterpretation is likely to have value?How well would articulated guidelines do prospectivelyin separating out patients who would benefit from rein-terpretation of their abdominal imaging studies fromthose who would not? Which clinical signs and symp-toms or preexisting patient conditions yield the highestvalue in abdominal imaging reinterpretation by special-ists? These questions and others will be important toinvestigate as institutions and health care systems tacklethe challenge of delivering high value care.

In summary, our study shows that subspecialistreinterpretation of abdominal imaging studies detects farmore false positive and false negative primary interpre-tation errors than created by the reinterpretation. Thispractice has the potential to reduce cost by eliminatingunnecessary additional testing and incorrect treatmentplans as well as improving patient care with more accu-rate diagnoses and treatment strategies. The challengefor radiologists is to define the parameters that optimizethe cost/benefit ratio of subspecialty reinterpretation of


abdominal imaging examinations. Our data combinedwith previous literature lends support to a selectivestrategy that would encourage abdominal subspecialtyreinterpretation if any potentially important finding isreported or in patients with known or suspected cancer,trauma, or infection.

References

1. Garland LH (1949) On the scientific evaluation of diagnostic pro-cedures. Radiology 52:309–328

2. Borgstede JP, Lewis RS, Bhargavan M, Sunshine JH (2004)RADPEER quality assurance program: a multifacility study ofinterpretive disagreement rates. J Am Coll Radiol 1:59–65

3. Sung JC, Sodickson A, Ledbetter S (2009) Outside CT imagingamong emergency department transfer patients. J Am Coll Radiol6:626–632

4. Tilleman EH, Phoa SS, Van Delden OM, et al. (2003) Reinter-pretation of radiological imaging in patients referred to a tertiaryreferral centre with a suspected pancreatic or hepatobiliary malig-nancy: impact on treatment strategy. Eur Radiol 13:1095–1099

5. Zan E, Yousem DM, Carone M, Lewin JS (2010) Second-opinionconsultations in neuroradiology. Radiology 255:135–141

6. Kalbhen CL, Yetter EM, Olson MC, Posniak HV, Aranha GV(1998) Assessing the resectability of pancreatic carcinoma: the valueof reinterpreting abdominal CT performed at other institutions.AJR Am J Roentgenol 171:1571–1576

7. Di Piro PJ, Van Sonnenberg E, Tumeh SS, Ros PR (2002) Volumeand impact of second-opinion consultations by radiologists at atertiary care cancer center: data. Acad Radiol 9:1430–1433

8. Yoon LS, Haims AH, Brink JA, Rabinovici R, Forman HP (2002)Evaluation of an emergency radiology quality assurance programat a level I trauma center: abdominal and pelvic CT studies.Radiology 224:42–46

9. Eakins C, Ellis WD, Pruthi S, et al. (2012) Second opinion inter-pretations by specialty radiologists at a pediatric hospital: rate ofdisagreement and clinical implications. AJR Am J Roentgenol199:916–920

10. Jeffers AB, Saghir A, Camacho M (2012) Formal reporting ofsecond-opinion CT interpretation: experience and reimbursementin the emergency department setting. Emerg Radiol 19:187–193

11. Kazerooni NL, Kazerooni EA, Quint LE, Orringer MB (1998)Added value of thoracic radiology specialist interpretation of CTscans for lung cancer consultations in a thoracic surgery clinic(abstr). Radiology 209:171

12. Babiarz LS, Yousem DM (2012) Quality control in neuroradiology:discrepancies in image interpretation among academic neuroradi-ologists. AJNR Am J Neuroradiol 33:37–42

13. Briggs GM, Flynn PA, Worthington M, Rennie I, McKinstry CS(2008) The role of specialist neuroradiology second opinionreporting: is there added value? Clin Radiol 63:791–795

14. Jordan YJ, Jordan JE, Lightfoote JB, Ragland KD (2012) Qualityoutcomes of reinterpretation of brain CT studies by subspecialtyexperts in stroke imaging. AJR Am J Roentgenol 199:1365–1370

15. Gollub MJ, Panicek DM, Bach AM, Penalver A, Castellino RA(1999) Clinical importance of reinterpretation of body CT scansobtained elsewhere in patients referred for care at a tertiary cancercenter. Radiology 210:109–112

16. Moser JW, Wilcox PA, Bjork SS, et al. (2006) Pay for performancein radiology: ACR white paper. J Am Coll Radiol 3:650–664

17. Carney E, Kempf J, DeCarvalho V, Yudd A, Nosher J (2003)Preliminary interpretations of after-hours CT and sonography byradiology residents versus final interpretations by body imagingradiologists at a level 1 trauma center. AJR Am J Roentgenol181:367–373

18. Wysoki MG, Nassar CJ, Koenigsberg RA, et al. (1998) Headtrauma: CT scan interpretation by radiology residents versus staffradiologists. Radiology 208:125–128

19. Cervini P, Bell CM, Roberts HC, et al. (2008) Radiology residentinterpretation of on-call CT pulmonary angiograms. Acad Radiol15:556–562

20. Chung JH, Strigel RM, Chew AR, Albrecht E, Gunn ML (2009)Overnight resident interpretation of torso CT at a level 1 traumacenter an analysis and review of the literature. Acad Radiol16:1155–1160

21. Cooper VF, Goodhartz LA, Nemcek AA Jr, Ryu RK (2008)Radiology resident interpretations of on-call imaging studies: theincidence of major discrepancies. Acad Radiol 15:1198–1204

22. Erly WK, Berger WG, Krupinski E, Seeger JF, Guisto JA (2002)Radiology resident evaluation of head CT scan orders in theemergency department. AJNR Am J Neuroradiol 23:103–107

23. Lal NR, Murray UM, Eldevik OP, Desmond JS (2000) Clinicalconsequences of misinterpretations of neuroradiologic CT scans byon-call radiology residents. AJNR Am J Neuroradiol 21:124–129

24. Le AH, Licurse A, Catanzano TM (2007) Interpretation of headCT scans in the emergency department by fellows versus generalstaff non-neuroradiologists: a closer look at the effectiveness of aquality control program. Emerg Radiol 14:311–316

25. Miyakoshi A, Nguyen QT, Cohen WA, Talner LB, Anzai Y (2009)Accuracy of preliminary interpretation of neurologic CT examin-ations by on-call radiology residents and assessment of patientoutcomes at a level I trauma center. J Am Coll Radiol 6:864–870

26. Platts-Mills TF, Hendey GW, Ferguson B (2010) Teleradiologyinterpretations of emergency department computed tomographyscans. J Emerg Med 38:188–195

27. Ruchman RB, Jaeger J, Wiggins EF 3rd, et al. (2007) Preliminaryradiology resident interpretations versus final attending radiologistinterpretations and the impact on patient care in a communityhospital. AJR Am J Roentgenol 189:523–526

28. Rufener SL, Patel S, Kazerooni EA, Schipper M, Kelly AM (2008)Comparison of on-call radiology resident and faculty interpretationof 4- and 16-row multidetector CT pulmonary angiography withindirect CT venography. Acad Radiol 15:71–76

29. Strub WM, Leach JL, Tomsick T, Vagal A (2007) Overnight pre-liminary head CT interpretations provided by residents: locationsof misidentified intracranial hemorrhage. AJNR Am J Neuroradiol28:1679–1682

30. Tieng N, Grinberg D, Li SF (2007) Discrepancies in interpretationof ED body computed tomographic scans by radiology residents.Am J Emerg Med 25:45–48

31. Walls J, Hunter N, Brasher PM, Ho SG (2009) The DePICTORSStudy: discrepancies in preliminary interpretation of CT scans be-tween on-call residents and staff. Emerg Radiol 16:303–308

32. Xiong L, Trout AT, Bailey JE, Brown RK, Kelly AM (2011)Comparison of discrepancy rates in resident and faculty interpre-tations of on-call PE CT and V/Q scans: is one study more reliableduring off hours? J Am Coll Radiol 8:415–421

33. Yavas US, Calisir C, Ozkan IR (2008) The interobserver agreementbetween residents and experienced radiologists for detecting pul-monary embolism and DVT with using CT pulmonary angiographyand indirect CT venography. Korean J Radiol 9:498–502

34. Wechsler RJ, Spettell CM, Kurtz AB, et al. (1996) Effects oftraining and experience in interpretation of emergency body CTscans. Radiology 199:717–720

35. Bechtold RE, Chen MY, Ott DJ, et al. (1997) Interpretation ofabdominal CT: analysis of errors and their causes. J Comput AssistTomogr 21:681–685

36. Filippi CG, Schneider B, Burbank HN, et al. (2008) Discrepancyrates of radiology resident interpretations of on-call neuroradiologyMR imaging studies. Radiology 249:972–979

37. Soffa DJ, Lewis RS, Sunshine JH, Bhargavan M (2004) Disagree-ment in interpretation: a method for the development of bench-marks for quality assurance in imaging. J Am Coll Radiol 1:212–217

38. Thurfjell EL, Lernevall KA, Taube AA (1994) Benefit of inde-pendent double reading in a population-based mammographyscreening program. Radiology 191(241–244):9

39. Bell ME, Patel MD (2014) The degree of abdominal imaging (AI)Subspecialization of the reviewing radiologist significantly impactsthe number of clinically-relevant and incidental discrepanciesidentified during peer review of emergency after-hours body CTstudies. Abdom Imaging. doi:10.1007/s00261-014-0139-4


http://dx.doi.org/10.1007/s00261-014-0139-4

the clinical impact of subspecialized radiologist reinterpretation of abdominal imaging studies,...

Documents