J. L. Marsh, MD

Chair - Orthopaedic RRC

Director of the ABOSCarroll B. Larson Chair

Residency Program Director

University of Iowa Hospitals and Clinics

Iowa City Iowa

NAS - (and other new program requirements)

Overview of what you will need to do!

Next Accreditation System (NAS) Milestones Procedural minimums Continuous data assessment Surgical skills training Greater institutional oversight No PIF, less and different site visits

We are about to experience tremendous changes to the oversight of GME

The financial implications of these

changes are uncertain at this

time

Traditional Program Review Program review scheduled PIF prepared and sent to ACGME and SV SV – 1-2 days RRC review

PIF and SVR

Board pass rates, Resident Survey, Case log data RRC actions

Initial or continued accreditation with citations 1-5 year cycle Progress report Propose probation

SV and PIF are key portions of program review

What is NAS???

• Program accreditation system without:– Mandated site visits– PIF’s– 100% mandatory PR’s– Direct resident interviews

• Program accreditation system with:– Annual or semi annual data review– Performance metrics within the data elements– Emphasis on outcomes– Focused site visits driven by data metrics– More opportunity for program innovation– Data charts, graphs and data flows!!!

Notice the word “data” is used 5

times!!

Notice the “PIF and site visit”

are going away

NAS is not all about Milestones

Milestones will be a work in progress but will not be a part of program accreditation for several years

But Milestones are one of the new requirements where increased effort and cost for programs maybe necessary and they need to start now!!

NAS represents a substantial change in program oversight

Change in focus / function of RRC More educational Less regulatory

PD’s empowered to innovate & create an excellent programCore vs. detail requirements

Improved tools for program review without a PIF Focused reviews triggered by parameters set by the RRC

The data for Program Review by RRC Trended & weighted performance metrics

– Program data– Resident and faculty scholarship (new template)– Clinical experience (enhanced case logs)– Resident Survey (new questions)– Core Faculty Survey (new) – Semi-annual Resident Evaluation

• Milestones (new)• Clinical Competency Committee (new)

– Rolling Board pass rates (Parts I & II)– Program Self-Study (new) Site Visit (every 10 years)

Many of theseare outcomes, many of them are new!

Performance thresholds based on data elements

• Weighting of data elements will provide screening criteria• RRC annual review and action only if necessary• Potential actions include:

Initial or Continued Accreditation Request more information from program Request Site visit (focused or full) on short timeline Continued Accreditation with Warning Probation

• “If there is a problem get in there and fix it”• “If the data is good…….leave them alone…innovation”

Absent – SVR and PIF document

Program Requirement Changes

• Common & Specialty Specific (no change)• Core: requirement must be met as specified; if not,

program can be cited• Detail: programs will not be assessed for compliance with

these requirements if they demonstrate good educational outcomes. These are mandatory for new programs & those that failed to meet outcomes expectations (on Probation or Continued Accreditation with Warning)

• Outcome – Some data elements are based on these

NAS Program Review

Each program reviewed at least annually

NAS is a continuous accreditation process

– Review of annually submitted data

– Supplemented by:

• Reports of self-study visits every ten years

• Progress reports (when requested)

• Reports of site visits (as necessary)

Annual Review of Data (Oct. – Nov.)

Options Available Prior to January RRC Meeting• Focused Site Visit

• Full Site Visit

• Request Clarification or Progress Report

• Send Material out to RRC for ReviewHighlighting the Problem(s) for Peer Decision

• Move to Consent Agenda

Proposed workflow prior to RRC meeting

• Minimal notification given

• Minimal document preparation expected

• Team of site visitors

• Specific program area(s) investigated as instructed by

the RRC

NAS: Focused Site Visit

• Application for new program

• At the end of the initial accreditation period

• RRC identifies broad issues / concerns

• Other serious conditions or situations identified by the

RRC

NAS: Full Site Visit

Programs (CA) meet all established performance indicator thresholds (40%) – letter from ED Continued accreditation with no RRC review

Programs (CA) fail to meet 1 established performance indicator (30%) - letter from ED Continued Accreditation but notes need for improvement - indicates the deficiency

Programs that fail to meet 2 established performanceindicator thresholds but not “High Stakes” indicators (20%) – ED reviews program for trends – if first time event letter from ED Continued Accreditation notes problems no further RRC review

Theoretical Work Flow – Consent agenda (90%)

Programs fail to meet 3 -5 established performance indicator thresholds (7%) – Two RRC reviewers assigned

Programs fail 6-9 performance indicator thresholds (3%) – Assigned a focused or full site visit

Theoretical Work Flow – RRC review

For NAS you need to do two things!

• Do well on the performance metrics

• Appoint a new committee to oversee Milestones and develop a plan to

evaluate them

The data for Program Review by RRC

Trended & weighted performance metrics– Program data– Resident and faculty scholarship (new template)– Clinical experience (enhanced case logs)– Resident Survey (new questions)– Core Faculty Survey (new) – Semi-annual Resident Evaluation

• Milestones (new)• Clinical Competency Committee (new)

– Rolling Board pass rates (Parts I & II)– Program Self-Study (new) Site Visit (every 10 years)

Many of theseare outcomes, many of them are new!

Maintain a consistent solid

performance on all of these!!

Clinical Competency Committee

Semi Annual Review of Data to assign Milestones

• CCC - Faculty time and input necessary for these individual resident evaluations which are the Milestones

• There may also be a PEC committee. – program evaluation

Milestones5 level assessments of resident knowledge,

skills, attitudes, and other attributes of performance in the six competencies in a

developmental framework from less to more advanced. They are designed to demonstrate

program outcomes by assessing resident progress through the competencies measured

in the milestone framework!

Milestones: Medical Knowledge & Patient Care

• ACL• Ankle Arthritis• Ankle Fracture• Carpal Tunnel• Degenerative Spine• Diabetic Foot• Diaphyseal Femur & Tibia

Fracture• Distal Radius Fracture• Adult Elbow Fracture• Hip & Knee Osteoarthritis

• Hip Fracture• Metastatic Bone Lesion• Meniscal Tear• Pediatric Septic Hip• Rotator Cuff Injury• Pediatric Supracondylar

Humerus FractureSmall slices of clinical care – a biopsy of resident performance!

Milestones: Medical knowledge (example)

Operationalizing Milestone reporting?

• The faculty, PD and PC time and effort to accomplish this remain uncertain

• Therefore the tradeoff for absence of SV’s and PIF’s remains uncertain

• In my opinion they are good assessments which will make a more uniform national standard to assess resident competence

There are other non NAS requirements that will have financial implications

for your department

• 6 months of PGY 1 ortho

• Mandated surgical skills training through simulation

PG-1 Year Changes 2013-2014

• ABOS certification rules developed from results of a CORD survey

• ACGME/RRC accreditation rules developed from ABOS

6 months of orthopaedic surgery

Basic surgical skills training

Good news – they are the same!

PG-1 Year Changes 2013-2014More time on orthopaedics!

So orthopaedic PGY 1’s will be on ortho for 6 months instead of 3 months

In our program we have 6 PGY 1’s so effectively this is a junior level 1.5 FTE

How much of a cost advantage for a department is this?

6 months of orthopaedics

Basic surgical skills requirements (core)

A curriculum with goals and objectives

Assessment metrics

A dedicated space for the skills training

Training in basic skills required of residents for emergency care and to prepare residents for future participation in surgical procedures

This is what is required!What will that cost?

Results of a 2011 National Orthopaedic Program Director and Resident Survey – Karam and Marsh JBJS 2012

Only 50% of residency programs have a skills lab and program.

There is high interest among PD’s in a skills curriculum.

Most PD’s have little knowledge of the budget for skills training or the cost of a skills lab

Cost is a challenge to expansion of skills programs

Interest in a curriculum?

*Percentages may not total 100% because respondents were allowed to choose more than one answer.

Lack of funding

Barrier to skills program

Members of the ABOS (AOA/CORD and AAOS) Surgical Skills Task Force

• J. Lawrence Marsh, MD – Chair (ABOS)

• James E. Carpenter, MD (ABOS)• Shepard R. Hurwitz, MD (ABOS)• Michelle A. James, MD (ABOS)• Joel T. Jeffries, MD (AOA/CORD)• David F. Martin, MD (ABOS)• Peter M. Murray, MD (ABOS)• Bradford O. Parsons, MD (AAOS)• Robert A. Pedowitz, MD, Ph.D. Co-

Chair (AAOS)

• Brian C. Toolan, MD (AAOS)• Ann E. Van Heest, MD

(AOA/CORD)• M. Daniel Wongworawat, MD

(AAOS)

1. Sterile technique and operating room set up

2. Knot tying & suturing

3. Microsurgical suturing

4. Soft tissue handling techniques

5. Casting and splinting

6. Traction

7. Compartment syndrome

8. Bone handling techniques

9. Fluoroscopy

10. K-wire techniques

11. Basic techniques in ORIF

12. Principles and techniques of fracture reduction

13. External fixation 

14. Basic Arthroscopy skills

15. Basics of Arthoplasty

16. Joint injection

17. Patient Safety

Modules (ABOS skills taskforce modules)

Modules should include:

Low cost low tech options

Modules should include:Evaluation and assessment strategies

• Guided practice until performance within time standards

• Video of performance with blinded review by expert faculty with “pass” or “needs more practice”

• OR performance ONLY after verification

January 2013

All 6 PGY 1’s

Some call on weekend no other clinical

work

Summary and Conclusions Resident satisfaction was high.

A dedicated month of surgical simulation has potential to change the paradigm of skills training for junior residents.

Considerable time invested in the planning and execution but faculty members were eager to contribute.

The greatest expense was for cadaveric specimens. With better planning more cost effective simulations, this expense could be reduced.

Overal Summary and Conclusions

NAS will take more time/effort and more yearly costLess demanding in the year of a site visitThe balance is hard to knowOther changes to PR at PGY 1 will affect finances

Less availability for all night callMore ortho time for PGY 1’s (3 vs 6 months)

Surgical simulation will cost moneyMore or less depending….This investment may be worthwhile