marita s. teng, md program director department of otolaryngology mount sinai school of medicine new...
TRANSCRIPT
Marita S. Teng, MDProgram DirectorDepartment of OtolaryngologyMount Sinai School of MedicineNew York, NY
MANAGING THE ACGME SURVEY
OPDO MEETINGNOVEMBER 11, 2011
Online survey, 34 questionsAdministered annually for
residents and fellowsTiming: mid- January thru
early JuneRequires 70% completion
rateAggregate program-level
data provided to PD and DIO if 70% response rate reached
Site visitors and RRC have data access regardless of response rate
ACGME SURVEY – WHAT IS IT?
Topics covered Duty Hours Faculty Evaluation Systems Educational Content Resources Overall Experience
ACGME SURVEY – WHAT IS IT?
From the ACGME website “Data from the survey are intended as a diagnostic
tool to focus the field staff’s questions during the on-site visit… [and] to verify and clarify the information from them. Field staff will probe and clarify any grey shaded areas, and pay particular attention to the duty hour items and to items with a substantial number of non-compliant responses.”
If residents indicate problem areas corrected, site visitor will document.
If questions/phrases misunderstood by residents, site visitor will document.
DATA FROM SURVEY
“Substantial number of noncompliant responses” 20% is a common institutional
standard Internal systematic review and
action plan requiredRRC may send warning
letters to programs for poor results
Accreditation cycle may be shortened, effectively triggering an immediate site visit
WHEN TO WORRY?
Small programsOne or two disgruntled residents can significantly alter the data
Confidentiality is affected Interns
Data often reflect their experiences on services other than Otolaryngology, and are hard to pinpoint because of varying rotation schedules and anonymity of survey
PROBLEMS SPECIFIC TO US
But no “excuses” are accepted!
Survey instructions sent out only after a intro/Q&A session
Explain purpose of survey, clarify questions
Survey form available on ACGME website – go item by item
This is not the forum for complaints
Consider NOT having the PD conduct the session Associate PD? Chief resident(s)? Program coordinator?
“SAFE SURVEYING”
Remind the residents about certain items: Duty hours Confidentiality of your
evaluation system Availability/accessibility of
Goals & Objectives Resources & systems for
communication/support education committee
meetings faculty mentors chief or senior residents institutional ombudsman
Sleep deprivation education SAFER curriculum (ppt)
“SAFE SURVEYING”
We are better than most!
Focus on areas of around 5% or greater national rates of non-compliance
OTOLARYNGOLOGY PROBLEM AREAS
80-hour week (over 4 wks)1 day off in 7 (over 4 wks)No more than Q3 call (over 4 wks)10 hours off between duty periods24 + 6 continuous hours in-houseHome call does not preclude “rest and reasonable
personal time”Called into hospital from home counts towards
80hrs
DUTY HOURSSEVEN QUESTIONS
Never
Rarely
Sometimes
Very Often
Extremely Often
How often did you break the rule that there should be a 10-hour time period provided between all daily duty periods and after in-house call?
71.9%
22.6% 4.9% 0.7% 0%
COMPILED OTOLARYNGOLOGY DATA:
DUTY HOURS
Duty hour question guide (PDF) for residents on ACGME website
Extremely
Very Somewhat
Slightly
Not at all
How sufficient is the instruction you receive from faculty and staff in your program?
59.7% 34.3%
5.3% 0.4% 0.1%
Thinking about the faculty overall, how interested are they in your residency education?
64.1% 28.6%
5.8% 1.6% 0%
Thinking about the faculty overall, how effective are they in creating an environment of scholarship and inquiry?
61.4% 30.4%
6.8% 1.4% 0%
COMPILED OTOLARYNGOLOGY DATA:
FACULTY
Consider faculty workshops Mentorship Giving feedback Teaching techniques
Resources GME offi ce Dean’s offi ce Faculty Development
Programs
FACULTY ISSUES
Extremely
Very Somewhat
Slightly
Not at all
How satisfied are you that your program treats your evaluations of faculty members confidentially?
62.5% 28.0%
5.5% 1.3% 0.4%
How satisfied are you with the way your program uses the evaluations that residents provide to improve the program?
55.0% 30.4%
10.5% 2.4% 0.6%
Overall, how satisfied are you with the written or electronic feedback you receive after you complete a rotation or major assignment?
50.4% 35.8%
10.8% 2.3% 0.7%
COMPILED OTOLARYNGOLOGY DATA:
EVALUATION
Confidentiality is compromised by nature of being a small program Consider pooling attending or rotation evaluations, then
distribute at the end of each yearEvaluations need to be meaningful
Shortening the form Requiring written comments Issues identified by residents should be followed up;
improvements should be documented
EVALUATION ISSUES
Extremely
Very Sometimes/somewhat
Slightly
Not at all
How satisfied are you with the opportunities your program provides for you to participate in research or scholarly activities?
59.6% 31.0%
7.9% 1.3% 0.2%
In your opinion, how often do your rotations provide an appropriate balance between your residency education and other clinical demands?
50.7% 37.8%
10.2% 1.2% 0%
How often has your clinical education been compromised by excessive service obligations?
0.7% 1.9% 15.8% 35.3%
46.3%
COMPILED OTOLARYNGOLOGY DATA:
EDUCATIONAL CONTENT
Time for research How protected is the time?
Balance between education and service Physician extenders – “non-educational” work Continually re-evaluate quality of didactics Faculty mentorship & teaching
EDUCATIONAL CONTENT ISSUES
Extremely
Very Sometimes/somewhat
Slightly/rarely
Not at all
How often do you work in interdisciplinary teams to care for patients?
64.8% 28.9%
5.6% 0.7% 0.1%
How satisfied are you with your program’s process to deal confidentially with problems or concerns residents might have?
58.8% 30.0%
8.5% 2.2% 0.5%
To what extent does your program provide an environment where residents can raise problems or concerns without fear of intimidation or retaliation?
64.9% 25.7%
6.8% 2.1% 0.4%
COMPILED OTOLARYNGOLOGY DATA:
RESOURCES
Interdisciplinary teams Tumor Board? Case conferences? Combined Grand Rounds? QM or QI teams?
Dealing with problems confidentially Institutional resources in the GME
offi ceDealing with problems without
fear of intimidation or retaliation Resident-only meetings
RESOURCE ISSUES
Which of the following best summarizes your opinion of your residency program?
OVERALL EXPERIENCE
Great
Good
Neutral
Negative
Very negative
0 20 40 60 80
NationalOtolaryngology
Explain and prepare residents Purpose of the survey Specific questions Can re-administer survey
internally Increase communication
Regular meetings with residents Regular meetings with faculty Lots of channels
Increase overall moraleWe are in good shape!
Take as opportunity to improve program
TAKE-HOME POINTS