Can a Brief On-line Education Tool Improve Surgical Resident Operative Dictations?

A Prospective Evaluation

Alicia Kieninger, MD, Yi Wei Zhang, MD, Anna Dowdle, Amy Braddock, MD, Diego Hernandez, MD, Amy Kirby, MD, Robert

Robinson, MD

Background

• Surgical education encompasses much more than clinical and technical education– Patel, et. al, 2005

• Survey of ENT program directors and recent graduates• Topics lacking in current training programs

– Coding and documentation

– Office management

– Reimbursement

– “Business of medicine”

Background

• Proper documentation is the key to achieving adequate coding and reimbursement– Fakhry, et al., 2007

• Survey of surgical trainees and attendings regarding knowledge of coding and proper documentation

• Residents scored 54% correct vs. 77% for attendings

– Novitsky, et al., 2005• Deficiencies identified in resident operative notes would

have led to $18,200 in reduced revenues compared to attending notes

Background

• How do we teach residents the skills they need in less time?– Impact of the 80 hour work week

• Limited time spent in hospital must focus on clinical and technical skills

• Jones, et al., 2008– Implementation of a practice management program

improved resident coding accuracy from 36% to 88%

• Eichholz, et al., 2004– Brief teaching sessions improved ob-gyn resident dictation

scores

Background

• The operative note– Communication of details important for

patient care– Proper documentation for appropriate

billing– A skill important for all surgical trainees

Purpose

• To determine if a brief on-line education module can improve the quality and content of surgical resident operative dictations

Methods

• Participants– General surgery residents at a community based teaching

hospital

• IRB approval and informed consent obtained• Resident operative dictations collected for a 1 month

period before and after intervention– Blinded review by 2 of 4 participating attending surgeon

reviewers• Evaluation tool adapted from previously published, validated

tool (Vergis, et al., 2008)

• Dictations scored for content, quality, overall

Methods

• Statistical Analysis– Before and after scores compared using paired t-

test (SPSS v. 21.0)• Residents serve as their own controls

– p0.05 significant

Methods

• Module created for hospital-wide on-line compliance testing program– Brief power-point presentation

• Outlining keys to content and quality of operative dictations

– Several multiple choice questions

Results

Total Junior(PGY 1-3)

Senior(PGY 4-5)

Residents 11 7 4

Dictations (pre/post)

144 (86/58)

76 (46/30)

68 (40/28)

ResultsAll Residents

0

0.2

0.4

0.6

0.8

1

Overall Content Quality

Before

After

NS

ResultsJunior vs. Senior Residents

0

0.2

0.4

0.6

0.8

1

Overall Content Quality

Junior (PGY 1-3)

Senior (PGY 4-5)

p<0.0001*p<0.0001* p<0.0001*

ResultsPaired t-Test

0

0.2

0.4

0.6

0.8

1

Overall Content Quality

Before

After

p= 0.314 p= 0.163 p= 0.230

Conclusions

• Senior resident dictations scored significantly higher than junior resident dictations

• There was a trend toward improved dictations after participation in the educational module for the group as whole

• Using the paired t-test, individual residents showed improvement after our intervention when compared to their own pre-intervention performance

Limitations

• Small sample size

• Heterogeneity of operative procedures

Future Directions

• Expand information in education module based on deficiencies in operative notes

• Focus evaluations on single operative procedure to improve validity

• Expand to other residency programs• Add a component of direct one-on-one

feedback to residents to improve readability

References

• Patel, A.T., R.M. Bohmer, et al. (2005). “National assessment of business-of-medicine training and its implications for the development of a business-of-medicine curriculum.” Laryngoscope 115(1): 51-55.

• Fakhry, S.M., L. Robinson, et al. (2007). “Surgical residents’ knowledge of documentation and coding for professional services: an opportunity for a focused educational offering.” Am J Surg 194(2): 263-267.

• Novitsky, Y.W., R.F. Sing, et al. (2005). “Prospective, blinded evaluation of accuracy of operative reports dictated by surgical residents.” Am Surg 71(8): 627-631.

• Jones, K., R.A. Lebron, et al. (2008). “Practice management education during surgical residency.” Am J Surg 196(6): 878-881

• Eichholz, A.C., B.J. Van Voorhis, et al. (2004). “Operative note dictation: should it be taught routinely in residency program?” Obstet Gynecol 103(2): 342-346.

• Vergis, A., L. Gillman, et al. (2008). “Structured assessment format for evaluating operative reports in general surgery.” Am J Surg 195(1): 24-29.

Thank You