Assessing Shoulder Dystocia Simulations for Quality

Elizabeth Yznaga, CNM, MSN,sDNP University of San Francisco, Kaiser Permanente San

Jose

Quality Shoulder Dystocia Simulations

Quality simulations balance internal and external validity.

They are designed to demonstrate replicability, integrity and generalizability.

Quality can be observed and tracked by process checklists and assessment forms.

Participants recognize quality as learner support, excellent feedback, and a coherent reflection of real life events. Prion, 2008

Inherent structural validity

Internal Validity

External Validity

Generalizability

Unique restrictions of time, language, personnel, and actions promote internal and external validity

Quality characteristics

Clear objectives: Learning, practice or drill

Reliability of the content: Evidenced-based, and standardized

Reliability of the experience: Scripted scenario, consistency in treatments ( fetal manikin), clear directions, and endpoints Jefferies (2007)

Consistent trainers: Standardized cues and other participant support

Documentation: Checklists for process and assessment, audiovisual records decrease bias in assessment

Thoughtful Feedback: Respectful, standardized forms with empirical, aesthetic, personal, ethical, and reflection prompts Johns (2004), Jefferies (2007)

Evaluation: Promotes improvement and integrity

Tracking quality

Increase fidelity with checklists: the setting, participant roles, manikin, and equipment

Standardized forms for observation and audiovisual assessments

Pre and post testing of content and confidence

Electronic simulators are high fidelity by their limited nature

The plastic pelvis and doll are low fidelity unless consistent treatments are followed

Quality is not conferred by the manikin, but by the scenario

Design for the site

Best control of most factors

Education costs Easier to use audio video

technology

Flexible participants Informal team building Short programs Shift acuities may

degrade the simulation

On shift simulations

Scheduled simulations

Multidisciplinary teams Complexity = participants Multiple checklists and

evaluators

Birth center and home birth

Hospital sites

Simple checklists Small teams Multiple responsibilities

Defi

ne th

e b

asic

tasks

Performance check points

Identify the problem

Call for help: assistants, nurses, pediatricians, neonatal code team

Communicate with the staff and the patient and family

Call out and perform the maneuvers

Document the delivery

Feedback

Assessments: measuring quality

Study Pretest/training

Posttest/training

Written Test in Class

Student Self-Report/feedback

Clinical Instructor Feedback

Clinical Elements Checklist

Repeat Demo/ Other

Crofts 2006

Baseline test

1-3 wks 185 questions

Force data Video recording

Key events Actor rated com

Identifies the problem

Call for help

Rubins Woods Posterior arm

MacRobert’s Suprapubic pressure

Com/ Doc.

Time

Clear objectives

Fidelity Adherence to scenario

Students are supported

Reflective feedback

Evaluation

Scenario 1

Track the study or course methods of assessment

Track the shoulder dystocia maneuvers

Track the internal validity of the scenario

Summative Evaluation

Simulation Summative Evaluation Scorecard Category Element 0 1 2 Comments

Knowledge and understanding

Clear objectives

Appropriate level of complexity

Skill performance

High expectations of everyone

Collaboration and teamwork

Learner satisfaction Active learners

Diverse learning

Fidelity for engagement

Critical thinking abilities

Appropriate debriefing

Appropriate level of problem solving

Self confidence Appropriate student supportTotal

Total out of 20

Summary

Design the simulation to reflect the site, the objectives

Maintain internal validity by the use of scripts, checklists and consistent trainers

Develop external validity by sharing scripts, checklists and assessments with other entities

Design basic simulations for on-site, spontaneous practice