simulation: head to head with other educational methods

Simulation: Head to Simulation: Head to Simulation: Head to Simulation: Head to Head with other Head with other

Educational MethodsEducational MethodsValerie M. Howard EdD MSN RNAssociate Professor of Nursing

Si l i C diSimulation CoordinatorRobert Morris University

howardv@rmu eduhowardv@rmu.edu

DisclosureDisclosure• Co-author of Elsevier Simulation Learning

S tSystem

IntroductionIntroduction• “Clinical simulation is on the point of

h i i ifi t i t h lth having a significant impact on health care education across professional boundaries and in both the undergraduate and and in both the undergraduate and postgraduate arenas.”• Bradley, P. (2006). The history of simulation in Bradley, P. (2006). The history of simulation in

medical education and possible future directions. Medical Education, 40, p. 254.

DefinitionsDefinitions

• The technique of imitating the behavior of some situation or process by means of a s itabl analogo s sit ation o appa at s suitably analogous situation or apparatus, especially for the purpose of study or personnel trainingpersonnel training.• Bradley, P. (2006). The history of simulation in

medical education and possible future directions. Medical Education, 40, p. 254.

• Set of techniques for re- creating aspects of the real world, typically to replace or amplif act al e pe iencesamplify actual experiences.• Dutta, S, Gaba, D, Krummel, T. (2006). To

simulate or not to simulate: what is the simulate or not to simulate: what is the question? Annals of Surgery, 243(3), p. 301.

• Simulations include devices, trained persons lifelike virtual trained persons, lifelike virtual environments, and contrived social situations that mimic problems events or conditions problems, events, or conditions that arise in professional encounters.

– Issenberg, SB, et al. (2005). g, , ( )Features and uses of high-fidelity medical simulation that lead to effective learning: a BEME systematic review. Medical Teacher, 27(1), p. , ( ), p11.

• Activities that mimic the reality of a clinical environment and are designed to demonstrate procedures decision-making demonstrate procedures, decision making, and critical thinking through techniques such as role playing and the use of devices such as interactive videos or mannequins.– Jeffries, P. (2005). A framework for designing,

implementing and evaluating simulations used implementing, and evaluating simulations used as teaching strategies in nursing. Nursing Education Perspectives, 26(2), p. 97.

• May be very detailed and closely simulate reality, or it can be a grouping of components that a e combined to p o ide components that are combined to provide some semblance of reality. • Jeffries P (2005) A framework for designing • Jeffries, P. (2005). A framework for designing,

implementing, and evaluating simulations used as teaching strategies in nursing. Nursing Ed ti P ti 26(2) 97Education Perspectives, 26(2), p. 97.

Low Fidelity Simulation in Low Fidelity Simulation in Professional DisciplinesProfessional DisciplinesProfessional DisciplinesProfessional Disciplines

• Standardized patients (Ali et al • Standardized patients (Ali, et al, 1998 and 1999)

• Cardiac simulator for heart sounds (Issenberg, et al, 2000)

• Case studies at Kaiser Permanente (Leonard et al Permanente (Leonard, et al, 2004)

• CD’s to understand the interdisciplinary training in care of asthma patients (Rodehorst, 2005)2005)

HPS in Professional HPS in Professional EducationEducationEducationEducation

• AnesthesiaM di i• Medicine

• Trauma Team PerformanceT a ma and Wa skills Pe fo mance• Trauma and Warskills Performance

• Orientation to the professional workplace• Critical Care Nurse Training• Critical Care Nurse Training• Pre-Hospital and in-hospital care providers• Neonatal Skills Training• Neonatal Skills Training• Evaluative purposes

The Use of SimulationThe Use of Simulationin Nursing Educationin Nursing Educationin Nursing Educationin Nursing Education

• Lack of research studies involving nursing t d t students (Rhodes & Curran, 2005)

• Lack of research studies focusing on ed cational o tcomes of sim lation educational outcomes of simulation (Cioffi, et al, 2005)

• Lack of research studies or data focusing Lack of research studies or data focusing on the use, implementation or value of simulation in the curriculum (Ravert, 2002; Seropian, 2004)

The Use of SimulationThe Use of Simulationin Nursing Educationin Nursing Educationin Nursing Educationin Nursing Education

• Ravert’s Review (2002)513 f ith t b d • 513 references with computer based simulation in the educational process

• Only 9 references met her inclusion • Only 9 references met her inclusion criteria:• Contains some type of computer based yp p

simulation• Utilizes an outcome measure related to

d tieducation

HPS and Undergraduate HPS and Undergraduate Nursing EducationNursing EducationNursing EducationNursing Education

• Competency based instruction and critical incident i t nursing management (Nehring, et al, 2002)

• Objective structured clinical examination (Alinier, et al, 2004)

• Student and faculty perceptions with respect to the use of HPS (Feingold,et al, 2004)

• Process of Creating Scenarios (Jeffries, 2005, Medley & Process of Creating Scenarios (Jeffries, 2005, Medley & Horne, 2005; Rhodes & Curran, 2005)

• HPS as an alternate clinical experience (Bearnson & Wiker, 2005), )

• HPS to teach difficult concepts (Doyle, 2002)

• Enhance learning of resuscitation scenarios (Long, 2005)2005)

HPS in Graduate Nursing HPS in Graduate Nursing EducationEducationEducationEducation

• Nurse anesthesia training (Monti, et al 1998; O’Donnell et al 1998)O’Donnell, et al, 1998)

• ACNP and CNS training (Hravnak, et al, 2005)

• Midwifery students and clinical decision • Midwifery students and clinical decision making (Cioffi et al, 2005)

• Remediation for Nurse anesthesia • Remediation for Nurse anesthesia students (Haskvitz & Koop, 2004)

Updates since 2005Updates since 2005pp• Clinical Judgment Rubric designed to describe the

development of Clinical judgment (‘learning to think like a development of Clinical judgment ( learning to think like a nurse’) Lasater, 2006

• Sim involves successes and failures which are a prerequisite to expertise (Feltovick, et al, 2006)prerequisite to expertise (Feltovick, et al, 2006)

• Self reflection leads to self-correcting behaviors (Rudolph, et al, 2006)

• Importance of debriefing in sim education (Fanning & Gaba • Importance of debriefing in sim education (Fanning & Gaba, 2007)

• *Sim vs. Case Study approach for NP’s (Scherer, et al, 2007)

Updates since 2005Updates since 2005pp• Theoretical Frameworks and Sim (Waldner & Olson, 2007)

• *Sim for measuring clinical parameters (Radhakrishnan et al • Sim for measuring clinical parameters (Radhakrishnan, et al, 2007)

• Sim to teach new clinical instructors (Kraugscheid, et al, 2008)2008)

• Sim and Self efficacy – lit review (Leigh, 2008)

• Use of Sim in Ambulatory care (Maynes, 2008)

Updates since 2005Updates since 2005pp• Survey: Use of Sim in Nursing Anesthesia Programs

(Turcato, et al, 2008)( , , )

• Implementation of Sim throughout a curriculum (Tuoriniemi, et al, 2008; Starkweather & Kardong, 2008)

• Sim to enhance nurse-physician collaboration (Messmer, Sim to enhance nurse physician collaboration (Messmer, 2008)

• Simulation Typology-low to high fidelity – for developing competencies in new nurses (Decker, 2008)p ( , )

• Debriefing and Clinical Reasoning – used Outcome Present State-Test Model of clinical reasoning for debriefing –Clinical reasoning web and worksheets (Kuiper, et al, 2008)g ( p , , )

New Simulation JournalsNew Simulation Journals• Simulation in Healthcare

• Journal of the Society for Simulation in Healthcare

• Clinical Simulation in Nursing• Journal of the INACSL Organization

Basis of Simulation Basis of Simulation EducationEducationEducationEducation

• Criticized for adopting and implementing • Criticized for adopting and implementing education innovations without sound evidence of their efficacy.

• Literature on simulation is increasing• Literature on simulation is increasing.• Limited scope on simulation evaluation.• Research including an improved g p

methodological base.• Retains a peripheral place in education and

training.training.• Increased need for outcomes-based

education.

Gaps in the LiteratureGaps in the Literaturepp• Costs• Cost benefit ratio??• Cost-benefit ratio??• Minimal evidence that demonstrates

positive impact on learning outcomesp p g• Impact of simulation on critical thinking

skills?Mi i l di i h i d• Minimal studies with nursing students

• What are considered best practices?

Best Practice in Best Practice in Nursing EducationNursing EducationNursing EducationNursing Education

• P: Population• I: Intervention• C: Comparison Group• O: Outcome

A COMPARISON OF EDUCATIONAL A COMPARISON OF EDUCATIONAL A COMPARISON OF EDUCATIONAL A COMPARISON OF EDUCATIONAL STRATEGIES FOR THE ACQUISITION STRATEGIES FOR THE ACQUISITION

OF MEDICALOF MEDICAL--SURGICAL NURSING SURGICAL NURSING OF MEDICALOF MEDICAL SURGICAL NURSING SURGICAL NURSING KNOWLEDGE AND CRITICAL KNOWLEDGE AND CRITICAL

THINKING SKILLS: HUMAN PATIENT THINKING SKILLS: HUMAN PATIENT SIMULATOR VS. THE INTERACTIVE SIMULATOR VS. THE INTERACTIVE

CASE STUDY APPROACHCASE STUDY APPROACH

(HOWARD, 2007)(HOWARD, 2007)(HOWARD, 2007)(HOWARD, 2007)

The StudyThe Studyyy• Independent Variables

• Educational Intervention• Sim or Case Study

• Dependent Variables (Instrumentation?)• Medical Surgical Nursing Knowledge• Critical Thinking• Learner’s Perspectivep

Critical ThinkingCritical ThinkingCritical ThinkingCritical Thinking

Critical ThinkingCritical Thinkinggg• “Include the ability to reason, deduce, and

i d b d t h d induce based upon current research and practice findings”

• Conger & Mezza 1996• Conger & Mezza, 1996

Measuring Critical Measuring Critical Thinking and Nursing Thinking and Nursing

KnowledgeKnowledgeKnowledgeKnowledge• Health Education systems Incorporated

(HESI)*(HESI)*• Methods of exam construction / evaluation

t d i iti l thi ki th rooted in critical thinking theory (Paul, 1990)

and Bloom’s cognitive taxonomy (Bloom, et al, 1956))

*Now Evolve Elsevier*Now Evolve Elsevier

HESI Exam ConstructionHESI Exam Construction• Requirements:

• Inclusion of rationale• Writing test item at the application level

or above• Requiring multi-logical thinking to

h tianswer each question• Requiring a high level of discrimination

t h f l ibl lt tito choose from plausible alternatives

HESI Item AnalysisHESI Item Analysisyy• Measures of reliability

• KR 20• Point biserial correlation coefficients

• Measures of validity• Content validity• Construct validityy• Convergent validity

Kolb’s Theory of Kolb’s Theory of Kolb s Theory of Kolb s Theory of Experiential Learning Experiential Learning (1984)(1984)

• Application of theoretical and abstract concepts enhance cognitive development

• Education is a result of Experience (Dewey, 1938)p ( y, )

• Explains the importance of incorporating practice (clinical) in the curriculum

• Importance of reflection (debriefing)Importance of reflection (debriefing)

Learner Participation & Learner Participation & Learning Retention RatesLearning Retention RatesLearning Retention RatesLearning Retention Rates

Source:  National Training LaboratoriesBethel, Maine

Research QuestionsResearch QuestionsQQ• How does the effect of an educational

intervention using the HPS on nursing students’ intervention using the HPS on nursing students knowledge of medical surgical nursing compare to the effect of an educational intervention using a written case study?a written case study?

• How does the effect of an educational intervention using the HPS on nursing students’ critical thinking abilities compare to the effect of critical thinking abilities compare to the effect of an educational intervention using a written case study?

h h d ’ f h• What is the nursing student’s perspective of the educational intervention?

R h D iR h D iResearch DesignResearch DesignResearch Methodology

• Quasi Experimental 2 Group Pretest/Posttest

• Sample / Setting• Instrumentation• Instrumentation

• Customized HESI Exam• Pretest:

• PBCC Ave: .13• Diff Ave: .70• Rel: 0.93

• Posttest: • PBCC ave: .15• Diff Ave: .69• Rel: 0.94

• Researcher Developed Questionnaire• Face Validity• Face Validity• Content Validity• Internal Consistency (Cronbach’s alpha 0.87)

• Data Analysis• Descriptive statistics for survey• ANCOVA to determine significant differences

b d f between pretest and posttest for groups

Si l ti DSimulation Day(Repeated 6 times)

Group 1 Pre-Test using specialized

HESI exam on MI / CVA

Simulation Experience

Post-Test

Administer questionnaire

Offer Case Study

ExperienceMI / CVA content

questionnaire

Group 2 Pre-Test Case Study Post-Test Offer using

specialized HESI exam on

MI / CVA

ExperienceAdminister

questionnaire

Simulation Experience

content

S l A ( 49)S l A ( 49)Sample: Age (n=49)Sample: Age (n=49)

Age 18-24 25-31 32-38 39-45 45+

Sim 5 9 4 5 2

20% 36% 16% 20% 8%

Case 4 9 8 2 1

16.7% 37.5% 33.3% 8.3% 4.2%

Chi Square: p=.550

Sample: Nursing Program Sample: Nursing Program Sample: Nursing Program Sample: Nursing Program (n=49)(n=49)

BSN Second Degree Accelerated

Diploma

Sim 8 5 12

32% 20% 48%32% 20% 48%

Case 5 8 11

13% 26 % 46 9%13% 26.5% 46.9%

Chi Square: p(.495)

S l G d ( 49)S l G d ( 49)Sample: Gender (n=49)Sample: Gender (n=49)

Male Female

Sim 7 18

28% 72%

Case 2 22

8.3% 91.7%

Chi Square: p(.076)

Results Research Results Research Question 1Question 1Question 1Question 1

• How does the effect of an educational i t ti i th HPS i intervention using the HPS on nursing students’ knowledge of medical surgical nursing compare to the effect of an nursing compare to the effect of an educational intervention using a written case study?y

Results of ANCOVA on Results of ANCOVA on P U i HESI P U i HESI Posttest Using HESI Posttest Using HESI

Conversion ScoreConversion ScoreCo e s o Sco eCo e s o Sco eGroup Simulation Case Study

M SD M SD

Pretest (Covariate) 67.25 % 11.11 72.34 % 12.88

Observed Postest 73.16 % 10.47 67.77 % 13.80

Adjusted Posttest 74.34 % -- 66.43 % --

For test of equality of adjusted means

F(1,46)=6.02, p=.018

Results of ANCOVA on Results of ANCOVA on P U i HESI P U i HESI Posttest Using HESI Posttest Using HESI

Conversion ScoreConversion ScoreCo e s o Sco eCo e s o Sco e

7476

687072

SimC

646668 Case

6264

Pretest ObservedP tt t

AdjustedP tt tPosttest Posttest

Results of ANCOVA on Results of ANCOVA on Results of ANCOVA on Results of ANCOVA on Posttest Using HESI ScorePosttest Using HESI Score

Group Simulation Case Study

M SD M SD

Pretest (Covariate) 713.12 153.56 786.17 184.81

Observed Postest 738.00 131.01 670.08 181.83

Adjusted Posttest 750.42 -- 657.14 --

For test of equality of adjusted means

F(1,46)=4.63, p=.037

Results of ANCOVA on Results of ANCOVA on Results of ANCOVA on Results of ANCOVA on Posttest Using HESI ScorePosttest Using HESI Score

800

750

800

650

700 SimCase

600

550Pretest Observed

PosttestAdjustedPosttest

Results Research Results Research Question 2Question 2Question 2Question 2

• How does the effect of an educational i t ti i th HPS i intervention using the HPS on nursing students’ critical thinking abilities compare to the effect of an educational intervention to the effect of an educational intervention using a written case study?

Results of ANCOVA on Results of ANCOVA on P U i C i i l P U i C i i l Posttest Using Critical Posttest Using Critical

Thinking ScoresThinking Scoresg Sco esg Sco esGroup Simulation Case Study

M SD M SD

Pretest (Covariate) 700.72 156.64 770.04 185.70

Observed Postest 737.56 131.57 668.25 162.66

Adjusted Posttest 747.71 -- 657.67 --

For test of equality of adjusted means

F(1,46)=4.03, p=.051

Results of ANCOVA on Posttest Results of ANCOVA on Posttest Using Critical Thinking ScoresUsing Critical Thinking Scores

780

Using Critical Thinking ScoresUsing Critical Thinking Scores

720740760

660680700 Sim

Case

600620640

600Pretest Observed

PosttestAdjustedPosttest

Results Research Results Research Results Research Results Research Question 3Question 3

• What is the nursing student’s perspective of the simulation of the simulation educational intervention? • 1=strongly disagree• 4=strongly agree

Simulation Case

M SD M SD t p

Helped to stimulate critical thinking abilities 3.84 .37 3.50 .83 1.85

**.070

Was a valuable learning experience 3.80 .41 3.13 .68 4.2 *.0003

Knowledge gained from the experience can betransferred to the clinical setting

3.80 .41 3.46 .78 1.93

**.059

Should be included in our undergraduate education 3.76 .44 3.29 .751 2.68

*.0108

Helped me better understand concepts 3.72 0.46 3.25 0.74 2.69

*.010

Experienced nervousness during the educationali t ti

3.56 .51 1.67 .82 9.78

*.000intervention 8

Were realistic 3.56 .51 3.46 .72 .573

.569

Because of the educational intervention, I will be lessnervous in the clinical setting when providing care for

3.00 .82 2.58 .78 1.83

**.074nervous in the clinical setting when providing care forsimilar patients.

3

Can be a substitute for clinical experiences in thehospital.

2.56 .92 1.92 1.10 2.28

*.027

* Significant at p<.05 **Significant at p<.10

Student Evaluation SurveysStudent Evaluation Surveys*Significant at p<.05*Significant at p<.05g pg p

**Significant at p<.10**Significant at p<.10

3 5

4

3

3.5

2

2.5

SimCase

1

1.5Case

0

0.5

Stim CT** Valuable* Transfer toClinical**

UG Ed* Understand* Nervous* Realistic LessNervous**

Sub forClinical*Clinical Nervous Clinical

You are caring for a 60yof admitted with SOB, DOE, and You are caring for a 60yof admitted with SOB, DOE, and OrthopneaOrthopnea in in the ED. She has a h/o heart failure and HTN. the ED. She has a h/o heart failure and HTN.

• “Help me...I’m having trouble… catching… b th”my breath”

• Simulation Demonstration

Implications for Nursing Implications for Nursing EducationEducationEducationEducation

• HPS is an effective method for nursing educationeducation

• HPS can stimulate CT abilities• Strong significant differences between the • Strong significant differences between the

groups with respect to:• Understanding concepts• Valuable learning experiences• Substituting clinical experiences• Inclusion in undergraduate education• Invoking nervousness

What’s the future?What’s the future?• Needed:

• Outcome Studies• Reliable and Valid Measurement tools• Comparison Studies

The End!The End!The End!The End!

Any Questions?