assessment of ipe to move beyond attitudes: featured tools and
TRANSCRIPT
![Page 1: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/1.jpg)
Assessment of IPE to Move Beyond Attitudes: Featured tools and case studies
from the field Thursday, January 8
![Page 2: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/2.jpg)
Moderator & Presenter
Sarah Shrader, PharmD
Clinical Associate Professor, University of Kansas, School of Pharmacy
![Page 3: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/3.jpg)
Presenters
Erin Abu-Rish Blakeney, PhD, RN Faculty,
University of Washington, School of Nursing
Seattle, WA
Doug Brock, PhD Associate Professor
Department of Family Medicine and MEDEX Northwest,
University of Washington-School of Medicine
Seattle, WA
LuAnn Wilkerson, EdD Professor of Medicine and Senior Associate Dean for
Medical Education, David Geffen School of Medicine,
University of California, Los Angeles
![Page 4: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/4.jpg)
Assessment of IPE to Move Beyond Attitudes: Featured Tools and Case
Studies from the Field
Part 1
January 8, 2015
Sarah Shrader, PharmD, FCCP, BCPS, CDE (moderator)
![Page 5: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/5.jpg)
Assessment Beyond Attitudes • Intensifying interest to assess IPE beyond attitudinal
measures
• Quantitative, qualitative, and mix-methods approach
• Cochrane Review Updated 2013: “To improve the quality of evidence relating to IPE and patient outcomes or healthcare process outcomes, the following three gaps will need to be filled: first, studies that assess the effectiveness of IPE interventions compared to separate, profession-specific interventions; second, RCT, CBA or ITS studies with qualitative strands examining processes relating to the IPE and practice changes; third, cost-benefit analyses.”
Reeves S, Perrier L, Goldman J, Freeth D, Zwarenstein M. Interprofessional education: effects on professional practice and healthcare outcomes (update). Cochrane Database of Systematic Reviews 2013, Issue 3. Art. No.: CD002213. DOI:10.1002/14651858.CD002213.pub3.
![Page 6: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/6.jpg)
Evaluation Outcomes Measured
Abu-Rish, E. et al. Current Trends in Interprofessional Education of Health Sciences Students: A Literature Review. 2012;26: 444-51.
![Page 7: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/7.jpg)
Assessment Methods
Abu-Rish, E. et al. Current Trends in Interprofessional Education of Health Sciences Students: A Literature Review. 2012;26: 444-51.
![Page 8: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/8.jpg)
A Point to Ponder
• Interprofessional Activity/Program Evaluation VS. • Interprofessional Learner Assessment
![Page 9: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/9.jpg)
Level 1a: Reaction • Learners’ views on the learning experience and its interprofessional
nature Level 2a: Modification of attitudes/perception • Changes in reciprocal attitudes between participant groups.
Changes in perception or attitude towards the value and/or use of team approaches to caring for a specific patient/client group.
Level 2b: Acquisition of knowledge and/or skills • Including knowledge and skills linked to interprofessional
collaboration. Level 3: Behavioral change • Identifies individuals’ transfer of interprofessional learning to their
practice setting and their changed professional practice Level 4a: Change in organizational practice • Wider changes in the organization and delivery of care Level 4b: Benefits to patients/clients • Improvements in health or well-being of patients/clients
Kirkpatrick/Barr’s Evaluation Framework
From Barr, Koppel, Reeves, Hammick, Freeth. Effective Interprofessional Education Argument, Assumption & Evidence. 2005 Malden, MA: Blackwell Publishing,Ltd.
![Page 10: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/10.jpg)
Miller’s Pyramid of Assessment
![Page 11: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/11.jpg)
Additional Resources Canadian Interprofessional Health Collaborative. An inventory of quantitative tools measuring interprofessional education and collaborative practice outcomes. 2012 Aug. Available from http://www.chd.ubc.ca/news/featured-inventory-quantitative-tools-measure-interprofessional-education-and-collaborative-pra. National Center for Interprofessional Practice and Education Measurement Instruments https://nexusipe.org/measurement-instruments
![Page 12: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/12.jpg)
Additional Resources Valentine MA, Nembhard IM, Edmondson AC. Measuring teamwork in health care settings: A Review of Survey Instruments [Internet]. Harvard Business School. 2011 May. Report No.:11-116. Available from: http://hbswk.hbs.edu/item/6727.html. Measuring the Impact of Interprofessional Education on Collaborative Practice and Patient Outcomes: A Consensus Study. October 7, 2014. Available from: https://www.iom.edu/Activities/Global/MeasuringtheImpactofInterprofessionalEducation/2014-OCT-07.aspx
![Page 13: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/13.jpg)
Additional Resources
Blue A, Chesluk B, Conforti L, Holmboe E. Assessment and Evaluation in Interprofessional Education: Lessons Learned from a Multi-Methods Study. • Article accepted for publication in the Journal
of Allied Health • Webinar from 2014 posted on National Center
Resource Exchange (click link above).
![Page 14: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/14.jpg)
ASSESSMENT OF IPE TO MOVE BEYOND ATTITUDES: VALIDATION OF THE PERFORMANCE ASSESSMENT FOR INTERPROFESSIONAL COMMUNICATION AND TEAMWORK (PACT)
Doug Brock, PhD Erin Abu-Rish Blakeney, PhD, RN Chia-Ju Chiu, PhD, PT
![Page 15: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/15.jpg)
Acknowledgements • This work was completed as Dr. Chia-Ju Chiu’s dissertation, to develop widely applicable tools for assessing team performance.
• The work was supported by the University of Washington Macy Project Evaluation Workgroup.
• The work was funded in part by a grant from the Josiah Macy Foundation.
![Page 16: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/16.jpg)
Background
• University of Washington IPE
• Training pre-licensure students
• Focus on Kirkpatrick/Barr acquisition of skills / knowledge
• Assessment of team performance, not individuals
• Emphasis on simulation
• Josiah Macy Foundation Grant
• TeamSTEPPS® National Training program (2009)
![Page 17: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/17.jpg)
Conceptual Framework– TeamSTEPPS®
![Page 18: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/18.jpg)
Aims 1) Develop psychometrically sound measures to
assess teamwork and communication skills
2) Collect data, summarize and present construct validity evidence for the PACT tools
![Page 19: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/19.jpg)
Context—The Macy Project • Three simulated acute care cases aimed at improving team communication skills • Dyspnea in a Hospitalized Patient • Teenager with Asthma • Postoperative patient with tachycardia
• Medical school capstone included medical, nursing, pharmacy, & PA students • Pilot study—49 IP students • Validation study—306 IP students
![Page 20: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/20.jpg)
Development of the PACT
![Page 21: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/21.jpg)
PACT Toolset Validation Methods • Tools included in validation study
• PACT-Novice – Peer or untrained observer scale • PACT-Expert – Faculty or expert scale • PACT-Video – Trained raters
• Assessed reliability and validity
![Page 22: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/22.jpg)
PACT—Expert
![Page 23: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/23.jpg)
PACT—Novice
![Page 24: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/24.jpg)
PACT—Video
![Page 25: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/25.jpg)
Construct Validity Evidence 1) Content 2) Substantive 3) Structural 4) Generalizability 5) External 6) Consequential * Messick, 1995
![Page 26: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/26.jpg)
Reliability Evidence
Version Number of
Valid Records
Number of Items
ICC F Cronbach’s
Alpha
PACT-Novice
904 5 .85 6.46** .85
PACT-Expert
85 13 .76 4.09** .79
PACT-Video
112 26 .90 9.63** .91
ICC(1,k) was calculated with one-way random effects model where rater effects are random. ICC for Average Measures was reported. ** F is significant p < .001
![Page 27: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/27.jpg)
Content: Relevance & Representativeness • Scenarios, training, & tools were developed incorporating TeamSTEPPS® framework, constructs & terminology:
1. Team Structure,
2. Leadership,
3. Situation Monitoring,
4. Mutual Support,
5. Communication.
• All 5 TeamSTEPPS teamwork constructs were represented in global quality ratings in all tools • PACT-Expert & PACT-Video included behavioral markers
![Page 28: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/28.jpg)
Substantive: PACT-Novice & PACT-Expert Consistency in Responses & Processes • PACT-Novice scoring system was introduced to the raters during didactic session.
• Scenarios unfolded under faculty guidance.
• Coaching of clinical skills ensured focusing on communication and teamwork skills.
• Administered immediately following simulated scenarios & before the debriefing.
![Page 29: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/29.jpg)
Substantive: Consistency in Responses & Processes PACT-Video • PACT-Expert raters were trained through PACT-Video rater training program & were experts in TeamSTEPPS®.
• PACT-Video raters received rater training program before rating the videos.
• Strict rating protocol was developed to ensure the consistency of rating processes.
![Page 30: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/30.jpg)
Structural: Scales reflect theoretical target
• Internal consistencies of the subscales • Exploratory factor analysis
![Page 31: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/31.jpg)
Structural—PACT Expert Behavioral Marker
Rotated Factor*
Communication Leadership & Team Structure
Mutual Support
Hand-off &
S-Bar 10. Repeat back instructions or clarifications. .673
5. Applies the STEP process. .627 .355
13. Demonstrates closed-loop communication. .597
12. Demonstrates SBAR. .544 .520
9. Verbalize their activities aloud. .524
2. Refer to protocols and checklists. .362
3. Conducts briefs/huddles/debriefs. .326 .565
1. Recognize a leader .769 4. Empowers team members to speak freely and ask questions.
.658
6. Patient/Family is included. .363
8. Use Two Challenge rule or CUS. .889
7. Ask for assistance prior to task overload. .353
11. Hands off to another team member. .597 Extraction Method: Principal Axis Factoring. Rotation Method: Varimax with Kaiser Normalization.
*Only values > 0.3 are displayed.
![Page 32: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/32.jpg)
Conclusions • Validity is a continuing and iterative process of establishing evidence to support specific uses & interpretations of the tool.
• Preliminary construct validity evidence for PACT was demonstrated.
![Page 33: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/33.jpg)
Next Steps • Broad dissemination of PACT toolset
• American Educational Research Association (AERA) 2015 • Additional publications
• Strengthen validity: • Replication/Different study designs to enhance generalizability • Build external validity through associations with external
measures • Explore the consequences/impact—consequential validity—of
team assessment
• Explore use with practicing clinical teams
![Page 34: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/34.jpg)
Best Uses for PACT Toolset • TeamSTEPPS-aligned situations/scenarios
• Assess team performance (not individual)
• PACT-Novice: During teaching/training • Observers • Debriefing
• PACT-Video: Research tool for understanding team behavior and performance.
![Page 35: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/35.jpg)
PACT Toolset Resources • Link to PACT tool set: http://collaborate.uw.edu/educators-toolkit/tools-for-evaluation/performance-assessment-of-communication-and-teamwork-pact-too
• Link to “Simulation Team Training Toolkit”: http://www.collaborate.uw.edu/educators-toolkit/interprofessional-simulation-team-training-faculty-toolkit/simulation-team-trainin
![Page 36: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/36.jpg)
Creating Innovative Tools for Assessing Learners’ Interprofessional
Competencies LuAnn Wilkerson, EdD
With funding from the Josiah Macy Jr. Foundation And a team of co-investigators from Nursing and Medicine
![Page 37: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/37.jpg)
Assessing Interprofessional Competencies
IPEC Core Competencies
Assessment Tools Values/ Ethics
Roles/ Responsibilities
Interprofessional Communication
Teams/ Teamwork
OSCE stations X X X
Direct Observation X X
Knowledge Test X X X
Video Case Exam X X X
Implicit Association Test X X
Multi-Source Evaluation X X
![Page 38: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/38.jpg)
Assessing Interprofessional Competencies
IPEC Core Competencies
Assessment Tools Values/ Ethics
Roles/ Responsibilities
Interprofessional Communication
Teams/ Teamwork
OSCE stations X X X
Direct Observation X X
Knowledge Test X X X
Video Case Exam X X X
Implicit Association Test X X
Multi-Source Evaluation X X
![Page 39: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/39.jpg)
OSCE Station
Roles/Responsibilities Interprofessional communication
Teams/Teamwork
40
• Setting: Outpatient heart failure follow up clinic • Complaint: need for medication change • Roles: NP and senior medical student
– Standardized NP and senior medical student OR – Standardized senior medical student and NP student
• Can be used as a simulation or an OSCE
![Page 40: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/40.jpg)
OSCE Station
Roles/Responsibilities Interprofessional communication
Teams/Teamwork
• Setting: Outpatient Family Practice Clinic • Complaint: sudden change in vision • Roles: NP student and medical student • Checklist focus:
– Skills in handoffs – Collaborative decision making – Understanding scope of practice
![Page 41: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/41.jpg)
Interprofessional Collaborator Assessment Rubric – Individual performance
TeamSTEPPS Questionnaire – team performance
eWalk© Direct Observation Tool Interprofessional Communication
Teams/Teamwork
eWalk® (Media-X Systems Inc., Ottawa, Canada) uses a tablet database to collect, store, analyze, and report observational data
![Page 42: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/42.jpg)
eWalk® (Media-X Systems, Inc, Ottawa, Canada) allows to collect, store, analyze, and report data based on real time observation.
![Page 43: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/43.jpg)
Knowledge Test
Roles/responsibilities Interprofessional communication
Teams/teamwork
• Item bank for test construction – 49 questions keyworded to 37 discreet objectives from the
Core Competencies for Interprofessional Collaborative Practice
• Vignette-based • Multiple choice format • 7-item collaborative practice case study
– 20 questions on system-based practice and policy
• Answer key
![Page 44: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/44.jpg)
• Goal: to synthesize feedback from team members with whom a student has worked for a period of time
– Student submits the names of 6 team members – Online survey tool (REDcap) is used to contact and remind invitees to respond
• Survey uses 6 Interprofessional Collaborator Assessment Rubric categories – Communication (7 items) – Collaboration (4 items) – Roles and Responsibilities (7 items) – Collaborative Patient-Family Centered Approach (4 items) – Team Functioning (5 items) – Conflict Management/Resolution (4 items)
• Invitee selects which behaviors he/she has observed and indicates level of the described behavior demonstrated over time
– Developing -- Occasionally demonstrates – Competent -- Frequently demonstrates – Mastery -- Consistently demonstrates
Multi-source Assessment
Interprofessional Communication Teamwork
![Page 45: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/45.jpg)
Interprofessional issues addressed in The Patient or the Protocol: • Shared decision
making • Teamwork • Conflict of interest • Paternalism • Palliative care • Roles and
responsibilities • Clinical trials and
informed consent • Patient safety
Zaption® (Stigler JW, Zaption Inc.,2013 ) combines video with question/response database https://www.zaption.com
Video Case Exam
Values/Ethics Roles/Responsibilities
Teams/Teamwork
![Page 46: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/46.jpg)
Zaption® can add to any online video: • multiple lines of text • Images • drawings done with
the mouse • essay questions with
multiple-line text box • questions requiring a
number for response • questions requiring
drawing the response • multiple choice
questions
47
![Page 47: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/47.jpg)
• The IAT was developed by Tony Greenwald and Mahzarin Banaji in 1994 to study attitudes and beliefs that are “normally outside of conscious control.” – Measures the strength of associations between concepts and
evaluations or stereotypes about others. – In the early 2000’s, Nosek, Greenwalk and Banaji developed a
demonstration website to study intergroup attitudes and stereotypes. – Growing evidence that these unintended biases can be changed. – https://implicit.harvard.edu
Banaji & Greenwald, The Blind Spot: Hidden Biases of Good People, 2013.
Implicit Association Test
Roles/responsibilities Values/ethics
![Page 48: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/48.jpg)
• Goal: help students evaluate their implicit assumptions about the roles of nurses and physicians in the health care system.
• The concepts and evaluative terms for the IAT were chosen based on focus groups held with UCLA medical students and advanced practice nursing students.
• Project Implicit created the interprofessional IAT.
![Page 49: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/49.jpg)
![Page 50: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/50.jpg)
Formative Evaluation Evaluation Questions Methods
1. How do the results of the student needs assessment inform tool design, content and learning outcomes?
Focus groups with Advanced Practice nursing students and 3rd year medical students
2. What are the major competency gaps for medical and APN students and do they differ by students’ clinical level or profession?
Same as above
3. How useful are the assessment tools when used formatively?
Pilot testing 6 tools
![Page 51: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/51.jpg)
Implementation Evaluation Evaluation Questions Methods 1. Usability: How easily do students find the IPE assessment tools to complete? How can the tools and implementation be improved?
Develop usability evaluation for each tool
2. What improvements can be made in the faculty and preceptor implementation guides for the assessment tools to better prepare for developing and/or observing IP competencies?
Pilot test each tool
3. How can the implementation processes be improved or refined to optimize the product?
Pilot test each tool
4. What are the psychometric properties of each of the assessment tools? Are revisions needed to improve reliability and validity?
Analyze pilot data
![Page 52: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/52.jpg)
Outcome Study Methods Evaluation Questions Cohort study
1. To what extent do students’ performance on the new assessment tools vary based on prior experience in IP settings, prior academic performance, age, gender, health profession, or mean ratings of their SBH small group instructors?
2. To what extent does classroom team composition of System-Based Healthcare course nursing and medical students moderate results on the assessment measures? 3. Do medical students trained in IP groups differ from those trained in medical-student-only groups on the OSCE stations? 4. What are the strengths and major threats to validity related to the study design?
![Page 53: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/53.jpg)
Assessing Interprofessional Competencies
IPEC Core Competencies
Assessment Tools Values/ Ethics
Roles/ Responsibilities
Interprofessional Communication
Teams/ Teamwork
OSCE stations X X X
Direct Observation X X
Knowledge Test X X X
Video Case Exam X X X
Implicit Association Test X X
Multi-Source Evaluation X X
54
![Page 54: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/54.jpg)
Questions?
![Page 55: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/55.jpg)
Save the Date
Assessment of IPE to Move Beyond Attitudes: Featured tools and case studies from
the field
Presented by Amy Blue, PhD,
Vernon Curran, Phd and
Sara Shrader, PharmD, FCCP, BCPS, CDE
Thursday, February 12
10:00 – 11:00 A.M. Central Time
![Page 56: Assessment of IPE to Move Beyond Attitudes: Featured tools and](https://reader036.vdocuments.site/reader036/viewer/2022062302/586783501a28abc83f8bccec/html5/thumbnails/56.jpg)
Thank you!
Please visit us at www.aihc-us.org for more information on future AIHC Webinars.