update milestone content review assessment tools · surgical specialty milestones specialty #...

Update Milestone content review

CCC: who, what, how

Assessment tools

3

Milestones are progressive

over time. There is no

prescribed speed.

Levels do not

correspond to PGY

or year in program.

Patient Care and Medical Knowledge Medical Knowledge

Ischemic Heart Disease Congenital Heart Disease

Cardiopulmonary Bypass, Myocardial

Protection and Temporary Circulatory

Support

End Stage Cardiopulmonary Disease

Valvular Disease

Great Vessel Disease General Competencies

Critical Care Professionalism – Ethics and Values; Personal

Accountability

Esophagus Practice Based Learning and Improvement –

Learning; Research and Teaching

Lung and Airway Interpersonal and Communication Skills

Chest Wall/Pleura/Mediastinum Systems Based Practice – Patient Safety;

Resource Allocation; Practice Management

Patient Care and Medical Knowledge Medical Knowledge

General Competencies

Professionalism – Ethics and Values; Personal

Accountability

Practice Based Learning and Improvement –

Learning; Research and Teaching

Interpersonal and Communication Skills

Systems Based Practice – Patient Safety;

Resource Allocation; Practice Management

Residents are not required or expected to reach

Level 5 by the end of training.

When selecting the best Level for a resident

◦ Selecting a middle box implies that all milestones

in that level and in lower levels have been

attained.

Selecting between levels

◦ all milestones in lower levels have been

attained as well as some milestones in the higher

level(s).

7

Thoracic Surgery Reporting Milestones Timeline

January 2014 April 2014 July 2014 October 2014

Jan-Mar 2014

-Form CCC, meet,

review & selects

assessment tools

-Develop new tools

to fill any

assessment gaps

Jul –Nov 2014

Perform assessments and

collect data Nov-Dec 2013

CCC meets and

assigns Milestones

July 1, 2014

TS Milestone Begin

January 2015

1st Milestones

document due

to ACGME

ACGME expects programs to implement Subspecialty

Reporting Milestones for AY 2014-2015

We are here…

Apr-Jun 2014

-CCC meets mo

-Implement tools,

Beta test, make

adjustments

.

Surgical Specialty Milestones

Specialty # Milestones Faculty

Development

Congenital Cardiac 9 +++

General Surgery 16 0

Otolaryngology 17 0

Colorectal 20 0

Neurosurgery 24 0

Pediatric Surgery 24 0

Thoracic 26 +++

Plastics* 30 0

Vascular 31 0

Urology 32 +

Orthopedic* 40 0

MEAN 24

Chair

Program

Coordinator

PD/

Assoc PD

Core Teaching/

Advisory Faculty

Clinical

Chair/Chief

Chief

Residents

CCC Team

●Limit the size (4-6)

●Broad representation of

Disciplines, experience,

locations

Non-faculty (e.g. RN’s, mid-

levels, perfusionists)

Educators (PhD, RN)

●Review/develop/implement assessment tools

●Review trainee portfolios

●Enforce completion of assessment data

●Review assessment data for accuracy

●Assign milestone levels for each trainee

biannually

●Develop remediation strategies

●Review faculty and trainee surveys, peer

evaluations

●Review current assessment tools and

find strengths and weaknesses

●Implement assessment tools that are

practical with maximal faculty

compliance

●Intervention strategies for struggling

residents/remediation plans

●Provide career development

●Everyone must buy in

●Improve timeliness and quality of

evals/feedback

●All members need to be fair

●Implement assessment tools that are

practical with maximal faculty compliance

●Intervention strategies for struggling

residents/remediation plans

●Provide career development

●Each CCC member is responsible for

one trainee (their assigned mentee) –

best for small programs

●Each CCC member is responsible for

one or more competencies for all

trainees

●All CCC members review all

assessment data for all trainees and

come to milestone level consensus

• Summative Assessment and Feedback

• Data/ Assessment System Development

• Assessment Methods and Validation

• Program Review and Development

Mapping experiences

Faculty/resident development

Educate

leadership

Develop/Adopt Systems

Pilot testing

Study for validity

Provide feedback

Remediation

Develop assessment curricula

Modify educational

experiences

Data manager

Professional development

Reporting mechanism

Selecting/developing effective assessment

tools

Incorporating milestone assessments

Subgrouping PGY years

Faculty development

Individualize curriculum

Time commitment

Domain Ave Time

(hrs/resident)

Program Review/Devt 17.3

Assessment

Methods/Validation

12.4

Data System Devt 6.6

Summative

Assessment/Feedback

18.3

Total 54.6

Milestones and evaluation principles

Integrating/synthesizing data

Providing timely and appropriate feedback

Thresholds of concern

Are meetings and practices “peer

protected”

Meeting minutes: Discoverable

How will differences between CCC

chair/ members /PD be managed

What is the appeals process

●Formative feedback and summative

evaluation of trainees

●Resident promotion decisions

●Curriculum and educational program

assessment and improvement

●Educational research

gmeclient

milestones

Admin home

User home

Admin home

User home

#1: Dependent on your Institution’s Graduate

Medical Committee

Being able to import and show summary data

from evaluations/tools

Generate simple counts of how many evaluators

chose a certain answer to a certain question

Connect evaluation questions to Milestones

reports: by competency, unique questions,

and/or directly verbatim

Direct observation

Case logs

Structured case discussion

Oral exams

Video assessments

Simulation

Direct observation

Pt survey

Video assessments

Simulation

Direct observation

Chart review

Pt survey

Oral exams

Video assessments

Simulation

ITS

Structured case discussion

Oral exams

Video assessment

Simulation

Moodle exams

Chart review

Case logs

Patient outcomes

Direct observation

360 assessment

Chart review

Structured case discussions

Video assessments

Record Review

Chart Stimulation

Recall

Check List

Global Rating

Standardized Patients

OSCE/ CASPE

Simulations and

Models

360 Global Rating

Learning Portfolios

ITE

Mock Oral Exam

Procedures and Case

Logs

Patient Survey

SESATS

Moodle Courses

TSC Curriculum and

Modules

Simulation / Video

Assessment

Database Patient

Outcomes

Observation of Patient

Encounters

Presentation skills

Patient evaluation

QI Review

Residents as educator

Chart audit

Implementation July 1, 2014

Documentation Jan 1, 2015

Program coordinator support is most critical

Semi-annual assessment of residents by CCC

Select and implement assessment tools

Faculty buy-in and development

Various methods in mapping and housing

assessments to the Milestones in development

syang@jhmi.edu