American Journal of Obstetrics and Gynecology (2006) 195, 1463–7

www.ajog.org

A surgical skills laboratory improves residents’ knowledgeand performance of episiotomy repair

Erika Banks, MD, Setul Pardanani, MD, Mary King, MD, Scott Chudnoff, MD,Karla Damus, PhD, Margaret Comerford Freda, EdD

Department of Obstetrics and Gynecology and Women’s Health Albert Einstein College of Medicine and MontefioreMedical Center, Bronx, NY

Received for publication February 23, 2006; revised May 8, 2006; accepted May 30, 2006

KEY WORDSSurgical skills

laboratory

Resident educationEpisiotomy repair

Objective: This study was undertaken to assess whether a surgical skills laboratory improves

residents’ knowledge and performance of episiotomy repair.Study design: Twenty-four first- and second-year residents were randomly assigned to either asurgical skills laboratory on episiotomy repair or traditional teaching alone. Pre- and posttests

assessed basic knowledge. Blinded attending physicians assessed performance, evaluating resi-dents on second-degree laceration/episiotomy repairs in the clinical setting with 3 validated tools:a task-specific checklist, global rating scale, and a pass-fail grade.

Results: Postgraduate year 1 (PGY-1) residents participating in the laboratory scored signifi-cantly better on all 3 surgical assessment tools: the checklist, the global score, and the pass/failanalysis. All the residents who had the teaching laboratory demonstrated significant improve-ments on knowledge and the skills checklist. PGY-2 residents did not benefit as much as PGY-1

residents.Conclusion: A surgical skills laboratory improved residents’ knowledge and performance in theclinical setting. Improvement was greatest for PGY-1 residents.

� 2006 Mosby, Inc. All rights reserved.

Effective teaching of surgical skills is vital to trainingskilled obstetricians and gynecologists. The reporteddecrease in the numbers of gynecologic surgery cases inObstetrics and Gynecology (OB/GYN) residency pro-grams highlights the importance of maximizing the edu-cational experience from each case.1 Traditionally a‘‘see one, do one, teach one’’ or apprenticeship methodhas been the model for surgical skills teaching. Morerecently, some residency programs have instituted a

Developed as part of the APGO/Solvay Educational Scholars

Development Program 2005-2006.

Reprints not available from the authors.

0002-9378/$ - see front matter � 2006 Mosby, Inc. All rights reserved.

doi:10.1016/j.ajog.2006.05.041

surgical skills laboratory that provides formal inanimatetraining outside of the operating room.

Training on simulators or inanimate models permitsassessment of surgical competency in OB/GYN and canimprove surgical abilities and performance in the lab-oratory setting.2,3 A randomized controlled study ofgeneral surgery residents showed that a surgical skillslaboratory improved interns’ performance and pre-paredness for placing their first central venous catheter.4

Cundiff surveyed OB/GYN resident and faculty atti-tudes before and after the institution of a laparoscopyteaching skills laboratory, showing an improved percep-tion of resident surgical ability after the surgical skills

1464 Banks et al

laboratory.5 Extensive work has been done in establish-ing the validity, reliability, and benefit of the objectivestructured assessment of technical skill (OSATS) for as-sessing and improving OB/GYN residents surgical skillsin the laborataory.2,3,6-8 However, what remains to bedemonstrated is the efficacy of a surgical skills labora-tory on actual performance in the operating room andobstetric suite.

In 2003 Nielsen et al9 developed an OSATS for episi-otomy repair and showed it was a reliable and validmethod of assessing resident skill in the laboratory set-ting. Episiotomy repair is among themost common surgi-cal procedures performed in OB/GYN, and it is usuallyone of the first surgical procedures residents learn to per-form. Thus it seems a relevant model for analyzing thetranslation of surgical laboratory teaching to improvedunderstanding and performance in the clinical setting.

The purpose of this study was to assess whether,compared with traditional apprenticeship training, a sur-gical skills laboratory would improve residents techni-cal performance and basic knowledge of an episiotomyor second-degree laceration repair. Initially, baselineknowledge and experience was assessed. Half the resi-dents were randomly chosen to participate in a surgicalskills laboratory on episiotomy repair. The residentsrandomly assigned to the skills laboratory had focuseddidactic instruction on episiotomy and second-degreelaceration repair with task-specific surgical skills stationsand an episiotomy repair model. Three validated instru-ments were used at the completion of the laboratory toevaluate these residents on their repair of the simulatedmodel. Finally, all the residents were observed in theobstetric suite performing episiotomy or second-degreelaceration repairs and rated with the same 3 instruments.

Materials and methods

This study is a prospective randomized controlled studyof first- and second-year residents and 2 approaches tosurgical skills training in episiotomy and second-degreelaceration repair. In July 2005, 24 first- and second-yearresidents (20 in OB/GYN and 4 in family practice) atAlbert Einstein College of Medicine were randomly as-signed to either: (1) apprenticeship surgical teaching aloneor (2) an inanimate surgical skills laboratory combinedwith apprenticeship surgical teaching. Both groups ofresidents were observed and evaluated performing eithera second-degree midline episiotomy or second-degreeperineal laceration repair during the 3-month observationperiod of the study from July to October of 2005. All theresidents completed pre- and posttests of knowledge andresponded to a questionnaire about prior experience withsuch repairs.

All of the postgraduate year 1 (PGY-1) and PGY-2OB/GYN residents, and the 4 family practice residents

rotating on labor and delivery during the study periodwere randomly assigned by drawing names from a hatof those who would participate in the laboratory. The12 residents who were randomly assigned to the surgicalskills laboratory group completed the laboratory 1 weekbefore the observation period began. They were in-structed not to reveal their participation in the labora-tory to their classmates, attending physicians, or seniorresidents. All residents, despite group assignment, ulti-mately participated in the surgical skills laboratory aftereach had been observed performing a repair, and datacollection was complete.

Two of the authors (E.B. and S.C.) developed thesurgical skills laboratory and conducted the sessions dur-ing the first week of July 2005. Each laboratory groupconsisted of 6 residents, 2 faculty members, and 1 sur-gical technician. The 3-hour laboratory session consistedof a brief background orientation on suture materialsand perineal anatomy, followed by technical skills teach-ing in an inanimate laboratory with 3 stations: knottying boards, suturing pigs’ feet, and repairing a lacer-ation on a perineal model (postpartum suturing trainerfor midline episiotomy shown in Figure 1). At the com-pletion of the skills laboratory, after feedback wascomplete, the primary investigator (E.B.) observed theresidents performing a repair on the perineal model andrated them with the same 3 measurement tools thatthey would later be used to evaluate them in the clinicalsetting (a checklist, pass/fail assessment, and a globalrating scale).

Basic knowledge was assessed via a brief question-naire developed by the primary investigator and vali-dated by 5 experienced OB/GYN attending physicians.To assess baseline knowledge before they were randomlyassigned, all the residents completed this pretest. Thepretest consisted of basic knowledge questions aboutperineal anatomy, indications for and complications ofepisiotomy repair, suture material, suturing technique,and anesthesia. The pretest also included questions aboutthe resident’s previous experience with episiotomy/second-degree laceration repair. After the observed re-pair for both groups of residents a posttest was givenwith the same knowledge and experience questions fromthe pretest as well as a question about what the residentfound most helpful in preparing them to do a repair.

Midline episiotomy and second-degree perineal lacer-ation repair (considered equivalent for the study) wereevaluated on labor and delivery by 3 faculty physicianseach using 3 validated tools: a task specific checklist, aglobal rating scale, and a pass-fail grade. The task-specificchecklist was based onNielsen’s validated objective scoresheet that assessed 6 task-specific basic components of anepisiotomy repair (evaluation of injury, anesthesia, choiceof suture material, reapproximation of anatomic land-marks, exposure, and self-evaluation of repair).9 Theglobal rating scale is a validated 5-point Likert scale

Banks et al 1465

evaluating 7 considerations: respect for tissue, time andmotion, instrument handling, knowledge of instruments,flowof operation, use of assistants, and knowledge of spe-cific procedure.10

Before evaluation began, all 3 evaluators, as wellas the principal investigator, evaluated a fourth-yearmedical student performing an episiotomy repair on apatient and rated the student’s performance with the 3measurement tools. The fourth-year student was sched-uled to begin OB/GYN residency training at anotherinstitution the following month. Thus, the observationapproximated the performance of an entering residentwithout compromising future study of our own resi-dents. Interrater reliability was calculated with a planfor remedial training if interrater reliability was below0.75 on any of the measurement tools. Each evaluatorwas blinded to randomization status until observationand evaluation of the residents was complete.

The raters did not interfere with the course of thesurgical procedure in any way. All surgical procedureswere performed with the usual resident and attendingcase assignment, and performed by the appropriate levelresident with the supervision of the patient’s assignedattending physician. The evaluator was not the attend-ing physician for the case but merely observed the repair.Only midline episiotomy or second-degree perineallaceration repairs were observed for evaluation.

Dichotomous variables on the checklists were com-pared for the 2 groups with c2analysis. Mean scores onthe global rating scale were compared with t tests. Thestudy received a waiver as an educational study by theInstitutional Review Board of the Albert Einstein Col-lege of Medicine and the Montefiore Medical Center.

Results

No significant differences in experience or baselineknowledge were noted between the group of residents

Figure 1 The perineal model: episiotomy suturing trainer.

who were randomly assigned to the surgical skills labo-ratory and those who were not (Table I). The groups werecomparable in terms of (1) the number of family practiceinterns compared with OB/GYN interns, (2) the resi-dents’ previous experience with episiotomy/second-degree laceration repairs, and (3) preexisting knowledge(pretest scores). In addition, no differences were foundin pretest scores between the family practice interns andthe OB/GYN interns, though the subgroups were toosmall (2 family practice interns in each group) to performmeaningful statistics.

Interrater reliability for the 3 faculty evaluators was89.5% for the checklist, 78.5% for the global ratingscale, and 100% for the pass/fail rating. Evaluationscores were consistent across ratings from the 3 evalu-ators, ie, no one evaluator graded consistently higher orlower than the others.

Figure 2 demonstrates the differences in performancebetween the residents who had the surgical skills labora-tory and the controls when the PGY-1 and PGY-2 res-idents were grouped together. The residents who hadthe surgical skills laboratory performed significantly bet-ter on the task-specific checklist than those without thelaboratory (P! .05). The differences observed on the glo-bal score and pass/fail assessment were not statistically

Table I Residents prior experience and baseline knowledgein episiotomy repair

Groups

Surgical skillslaboratory Control

Pvalue

PGY 1 (n = 14) 5 5OB/GYN PGY 1

Family practicePGY 1

2 2

No. repairs seenbefore study

3.1 G 3.4 2.9 G 3.5 .88

No. repairs donebefore study

1.2 G 0.4 1.6 G 0.8 .44

Pretest scores % 16.7 G 9.6 21.4 G 15.8 .74PGY 2 (n = 10 all

OB/GYNresidents)

5 5

No. repairs seenbefore study

9.2 G 7.9 18.0 G 19.6 .81

No. repairs donebefore study

15.2 G15.5 39.0 G 29.2 .92

Pretest score % 63.3 G 18.2 83.3 G 11.8 .12All residents

(N = 24)12 12

No. repairs seenbefore study

5.7 G 7.3 15.0 G 28.9 .32

No. repairs donebefore study

6.8 G 11.9 18.4 G 30.5 .17

Pretest score % 36.1 G 27.3 47.2 G 34.7 .39

Number or percent mean G SD.

1466 Banks et al

different (P = .05, and .06, respectively). On the knowl-edge assessment (posttest scores), those who had the sur-gical skills laboratory did significantly better than thosethan who did not (P ! .05).

Results for the 3 surgical rating parameters bypostgraduate year are in Table II. For all the param-eters, PGY-1 residents with the surgical skills labora-tory experience performed significantly better than theircounterparts in the control group: for the checklist(P = .01), for the global score (P =.009), and for thepass/fail evaluation (P =.008). PGY-2 residents whohad the skills laboratory showed no significant differ-ences on any of the technical skills parameters.

Residents who had the surgical skills laboratoryshowed no significant differences between their perfor-mance in the laboratory and in the clinical setting on anyof the surgical measurement parameters: the checklist,the global score, or the pass/fail analysis. The overallgrade in both settings was 91.7. Mean scores from thechecklist were 93 in the laboratory and 82 in the clinicalsetting; global scores for the laboratory and clinicalsetting were 74 and 78, respectively. Interestingly, theone intern who failed on the pass/fail analysis in thelaboratory also failed on the repair in the clinical setting.

Residents were asked to give feedback on their train-ing experience. Of the residents who had the surgicalskills laboratory, 9 of 12 (75%) said the surgical skillslaboratory prepared them best for performing the re-pair. Of 7 first-year residents who had the laboratory, 6(86%) said the laboratory was most helpful in preparingthem to do a repair, whereas only 3 of the 5 second-yearresidents (60%) responded the same.

Supporting the construct validity of our instruments,the second-year residents in the control group did signif-icantly better than their first-year counterparts on thechecklist, the global score, and the pretest (Table III). For

Figure 2 Mean scores of resident performance in episiotomy

repair (percent mean G SD, n = 24).

both groups overall, the second-year residents performedbetter than the interns on the checklist, the global ratingscale, and the pretest. This ability to distinguish betweenresident levels supports the construct validity of theseestablished testing measures in this environment.

Comment

To our knowledge this is the first prospective random-ized controlled trial in OB/GYN supporting the corre-lation between surgical skills laboratoary training andimproved resident performance in the operating room.The results suggest that, at least for novice surgeons,competence in the laboratory may be an appropriateproxy for competence in the operating room. For theparticular procedure we studied, the first-year residentsreceived more benefit from the surgical skills laboratory,both in knowledge and technical skills than did second-year residents. The fact that the residents’ performanceon all 3 technical skills measures (the checklist, the globalscore, and the pass/fail analysis) in the laboratory was not

Table II Surgical skills assessment by PGY level

Groups

Surgical skillslaboratory Control P value

ChecklistPGY 1 79.9 G 16.8 55.4 G 17.7 .01PGY 2 98.3 G 3.7 96.1 G 5.7 .24All 87.6 G 15.8 72.4 G 24.9 .04

Global scorePGY 1 69.4 G 14.8 46.9 G 15.8 .009PGY 2 89.1 G 6.2 86.8 G 9.4 .33All 77.6 G 15.4 63.5 G 24.3 .05

Passing gradePGY 1 85.7 28.6 .03PGY 2 100 100

PGY 1 (n = 14); PGY 2 (n = 10); All N = 24.

Percent mean G SD.

Table III Construct validity of tools used to assess knowl-edge and skill in episiotomy repair

PGY-1 Control(n = 7)

PGY-2 Control(n = 5) P value

Pretest 21.4 G 15.8 83.3 G 11.8 .00001Checklist 53.5 G 15.9 96.1 G 5.7 .0003Global score 46.9 G 15.8 86.8 G 9.4 .0003

All PGY 1(n = 14)

All PGY 2(n = 10) P value

Pretest 19.1 G 12.8 73.3 G 17.9 !.000005Checklist 64.8 G 23.6 90.7 G 9.5 .002Global score 58.2 G 18.8 88.0 G 07.6 .00005

Percent mean G SD.

Banks et al 1467

significantly different from their performance in theclinical setting suggests that surgical skills laboratorycompetence is an appropriate proxy for competence in theoperating room. Given the difficulty and inaccuracy ofassessing competence in the operating room, this is valu-able information for surgical educators concerned withAccreditation Council for Graduate Medical Educationcompetency requirements. If surgical ‘‘incompetence’’ inthe laboratory translates to surgical incompetence inthe operating room, then perhaps surgical remediationin the laboratory to the point of proficiency may translateto surgical competence by the time a resident is ready toperform that procedure for the first time.

In a commentary in last year’s Obstetrics and Gyne-cology, Fenner11 addressed the challenges of surgicaleducation in this era of decreased surgical volume, de-creasedworkhours, increased primary care requirements,increasing surgical technologies, and an expanding bodyof knowledge. She stated that the apprentice model(‘‘see one do one teach one’’) may be falling short, andthat ‘‘a new model for the teaching and assessment ofsurgical skills is needed that ensures the standardizationof skills with reliable performance measurements.’’11

On the basis of the general surgery precedent, Goffet al2 at University of Washington established the reli-ability and validity of OSATS to provide competencyassessments of residents’ surgical skills in gynecology.They have also supported the data that residents who re-ceive surgical skills teaching in an inanimate laboratoryare more competent on OSATS than those who do not.2

The findings in this study are consistent with this dataand extend the implication of improved surgical skills tothe clinical setting.

A surgical skills laboratory cannot replace the kind ofhands-on learning that goes on in the operating room.However, the advantages of using an inanimate labora-tory as part of our educational armamentarium areconsiderable. In the laboratory setting, one specific pro-cedure can be taught in a step-wise fashion with afocused detailed description of each technical step. Res-idents can learn standardized methods to perform aprocedure and begin practicing those organized stan-dardized steps in the laboratory without individual pa-tient differences and the stresses and complexities of theclinical situation. Leaders in the field describe the 2 mostimportant aspects of teaching surgical skills as teaching(1) in an environment where residents feel that mistakesare permissible and (2) in which their technique can beobserved and critiqued.12 An inanimate surgical skillslaboratory is inexpensive (our total supplies cost lessthan $100), easily replicable, straightforward to assem-ble, and thus costs mostly attending faculty time (forus, about 3 hours for 6 residents). Finally, practicingsurgical skills outside the operating room has potentialethical, medico-legal, as well as financial advantages andmay provide increased patient safety.

This study was limited by small numbers, singleobservations of resident performance, and the hetero-geneity of the second-year residents’ surgical experience.Future studies could evaluate larger groups of residentsand different skills laboratories for procedures at vari-ous resident levels. The fact that the first-year residentsimproved the most supports the concept that a surgicalskills laboratory may be most beneficial for the initialstage of learning to perform a procedure. As a means forcompetency assessment representative procedures foreach resident level could be used to support advance-ment to the next level. In this changing environment ofdecreased resident work hours and decreased surgicalvolume this feasible, evidenced-based, effective methodof teaching surgical skills may represent the direction ofresident surgical education in the future.

Acknowledgments

We gratefully acknowledge Diane Magrane, MD, forher substantive and invaluable guidance with the editingof this manuscript.

References

1. Sorosky JI, Anderson B. Surgical experiences and training of

residents: perspective of experienced gynecologic oncologists.

Gynecol Oncol 1999;75:222-3.

2. Goff BA, Mandel LS, Lentz GM, VanBlaricom A, Oelschlager

AA, Lee DM, et al. Assessment of resident surgical skills: is testing

feasible? Am J Obstet Gynecol 2005;192:1331-40.

3. Goff BA, LentzGM, LeeD, Houmard B,Mandel LS. Development

of an objective structured assessment of technical skills for obstetric

and gynecology residents. Obstet Gynecol 2000;96:146-50.

4. Velmahos GC, Toutouzas G, Sillin LF, Chan L, Clark RE, Theo-

dorou D, et al. Cognitive task analysis for teaching technical skills

in an inanimate surgical skills laboratory.AmJSurg 2004;187:114-9.

5. Cundiff G. Analysis of the effectiveness of an endoscopy education

program in improving residents’ laparoscopic skills. Obstet Gyne-

col 1997;90:854-9.

6. Goff BA, Lentz GM, Lee DM, Mandel LS. Formal teaching of

surgical skills in an obstetric-gynecologic residency. Am J Obstet

Gynecol 1999;93:785-90.

7. Lenz GM, Mandel LS, Lee D, Gardella C, Melville J, Goff BA.

Testing surgical skills of obstetric and gynecologic residents in a

bench laboratory setting: validity and reliability. Am J Obstet

Gynecol 2001;184:1462-6.

8. Goff BA, Lenz GM, Lee D, Fenner D, Morris J, Mandel LS.

Development of a Bench station objective structured assessment

of technical skills. Obstet Gynecol 2001;98:412-6.

9. Neilsen PE, Foglia LM, Mandel LS, Chow GE. Objective struc-

tured assessment of technical skills for episiotomy repair. Am J

Obstet Gynecol 2003;189:1257-60.

10. Reznick R, Regehr G, MacRae H, Martin J, McCulloch W. Test-

ing technical skill via an innovative ‘‘Bench Station’’ examination.

Am J Surg 1997;180:226-30.

11. Fenner D. Training of a gynecologic surgeon. Obstet Gynecol

2005;105:193-6.

12. Wanzel KR, Ward M, Reznick RK. Teaching the surgical craft:

from selection to certification. Curr Probl Surg 2002;39:574-659.