uso de osces en evaluación - unam · uso de osces en evaluación john (jack) r. boulet, ph.d....

Uso de OSCEs en Evaluación

John (Jack) R. Boulet, Ph.D.Foundation for Advancement of International Medical Education and

Research

Jornadas de Educación Médica 2010“Educación por Competencias: Cambio de Paradigma”

FAIMER®

“Educación por Competencias: Cambio de Paradigma”

Universidad Nacional Autónoma de MéxicoFacultad de Medicina UNAM

February 2010

Objective Structured Clinical Examination

� Series of short stations� Candidate assessed on � Candidate assessed on

same content� Multiple evaluators� Standardization

� Conditions

FAIMER®

� Conditions� Scoring tools

OSCE� Objective

� All candidates assessed with same content and marking schememarking scheme

� Structured� Stations have a specific task or tasks� Detailed scripts for standardized patients� Candidate instructions

FAIMER®

� Clinical Examination� Measures applied clinical and theoretical

knowledge� Post-encounter exercises

OSCE Content?

� Standardized or “real” patients

� Mannequin� Part-task trainer

� Haptic models

� Computer-based

FAIMER®

� Computer-based case simulation

� Hybrid models

Assessment OSCEs in Medicine� United States Medical Licensing Examination (USMLE™)

� Step 2 CS (ECFMG Clinical Skills Assessment)� Step 3

� National Board of Osteopathic Medical Examiners� National Board of Osteopathic Medical Examiners� Medical Council of Canada� General Medical Council (UK)

� Professional Linguistics and Assessment Board� Various specialty board examinations

� American Board of Emergency Medicine� American Board of Internal Medicine

� Royal College of Physicians and Surgeons of Canada

FAIMER®

� Royal College of Physicians and Surgeons of Canada

� Dillon, G.F., Boulet, J.R., Hawkins, R.E. & Swanson, D.B. (2004). Simulations in the United States Medical Licensing Examination™(USMLE™). Quality & Safety in Health Care, 13(Suppl 1), i41-i45.

Standardized (Simulated) Patients Examinations

� Performance-based� “Standardized”� “Standardized”

� Same conditions for all test takers

� Individuals trained to portray patients with specific conditions

FAIMER®

specific conditions� Chest pain� Depression� Shortness of breath

Performance Measures� Link measures to scenario

events� Focus on observable � Focus on observable

behaviors� Incorporate multiple

measures from different sources

� Capture performance

FAIMER®

� Capture performance processes in addition to outcomes

Types of Scores� Explicit Process

� Checklists� Key actions� Key actions

� Implicit Process� Rating scales

� Timing� Sequencing

� Explicit outcome

FAIMER®

� Explicit outcome� Patient status� Complications, etc.

Why are OSCEs Important for Assessment?

� Assessment drives learning� Efficient� Efficient

� large numbers of ‘examinees’

� Provides clinical exposure� Clinical skills assessment

� Practice-based learning� Tailored feedback

FAIMER®

� Tailored feedback

� Quickly identify curricular problems

� Patient safety

Competencies Assessed with OSCEs

� Knowledge� Communication�� Counseling� Crisis management� Procedures� Physical examination� Medical decision making

FAIMER®

� Medical decision making� Leadership� Teamwork� Interviewing

Important OSCE Issues� Standardization of

conditions an content�� Choice of tasks

� Generalization

� Training� Raters, SPs

� Scoring

FAIMER®

� Scoring� QA

� Standards

Content Under-representation� Some conditions cannot be simulated very well

� Not as important for basic skills� Programming for mannequins is imperfect� Programming for mannequins is imperfect� “Real” patients can vary over time

FAIMER®

Consequences of Testing� Candidate self-selection / preparation

� want to learn� specialized preparation courses� specialized preparation courses

� Changes in curriculum� Development of training centers

FAIMER®