2017 acgme program coordinators’ workshop€¦ · program opportunities - factors that favor the...
TRANSCRIPT
![Page 1: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/1.jpg)
2017 ACGME Program Coordinators’ Workshop
BR03: Using Data Strategically to Streamline Coordinator Work Products and Maximize Program
Outcomes
Ann Dohn, MA, DIO & GME Director, Stanford Nancy Piro, PhD, Sr. Program Manager/Education Specialist, Stanford
![Page 2: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/2.jpg)
Ann Dohn, MA Nancy Piro, PhD
Neither of the above speakers have any conflicts of interest to report.
![Page 3: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/3.jpg)
Confidential – For Discussion Purposes Only
Agenda How the role of the coordinator has increased both in importance and its impact
on the success of residency programs
How to expand your current toolbox to seamlessly align with the Academic Year Cycle of requirements
− a) Demonstration of a simple template for aligning your data with the accreditation requirements throughout the Academic Year. b) Strategic Use (and Re-Use) of Data Sets:
− 1. Surveys
− 2. Evaluations
− 3. Scorecards
− 4. Questionnaires
Q & A and participant discussion to address possible barriers to implementation
![Page 4: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/4.jpg)
Confidential – For Discussion Purposes Only
The Next Accreditation System (NAS)
ContinuousObservations(Collect Data)
Identify Opportunities for
Improvement (Find Problems)
Program Makes
Improvements (Solutions)
AssessProgram
Improvements(Program
Evaluation)
Promote
![Page 5: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/5.jpg)
Confidential – For Discussion Purposes Only
The Next Accreditation System (NAS) Overview
Internal Oversight
Graduate Medical Education Committee
Program Evaluation Committee’s Annual Program Evaluation
Milestone Evaluations by the Clinical Competency Committee
WebADS
Special Reviews
![Page 6: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/6.jpg)
Confidential – For Discussion Purposes Only
Program Responsibilities1. Annual Data Updates in ADS (externally reported)
2. Interim Program Changes (externally reported): Includes changes in resident complement or PD/PC change
3. Clinical Competency Committee (internal)
4. Milestone Reporting (externally reported to ACGME)
5. ACGME Faculty Survey (externally reported)
6. ACGME Resident/Fellow Survey (externally reported)
7. Program Evaluation Committee (internal)
8. Clinical Experience Review in annual program evaluations (internal)
9. Self Study (internal until external site visit at end of 10 years)
![Page 7: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/7.jpg)
Confidential – For Discussion Purposes Only
Managing it all: How will I pull this off?
![Page 8: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/8.jpg)
Confidential – For Discussion Purposes Only
Leveraging the Reusability of Data …
WebADS
Updated CVs
Faculty Scholarly Activity
Case Logs
Resident (360) &
Milestone Evaluations
Citation Responses
Major Program Changes
Duty HoursData
Resident Resident Scholarly Activity
Areas for Improvement
Semi Annual& Summative Evals
PEC/APE Meetings
Duty Hour Reviews
CCC MeetingsMilestone Evals
![Page 9: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/9.jpg)
Confidential – For Discussion Purposes Only
Organizing the data chaos…
![Page 10: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/10.jpg)
Confidential – For Discussion Purposes Only
Moving Streams of Data into Your Calendar Year Requirements
![Page 11: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/11.jpg)
Confidential – For Discussion Purposes Only
Creating the Annual Academic Year Timeline
![Page 12: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/12.jpg)
Confidential – For Discussion Purposes Only
Creating the Annual Academic Year Timeline
![Page 13: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/13.jpg)
Confidential – For Discussion Purposes Only
Where do I begin?
![Page 14: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/14.jpg)
Confidential – For Discussion Purposes Only
WebADS
Resident Scholarly Activity
Faculty Scholarly Activity
Faculty Scholarly Activity
Updated CVs
![Page 15: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/15.jpg)
Confidential – For Discussion Purposes Only
Data Collection for the Next Accreditation System: The Accreditation Data System (ADS) Annual Update
![Page 16: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/16.jpg)
Confidential – For Discussion Purposes Only
ACGME ADS – Reporting Window
![Page 17: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/17.jpg)
Confidential – For Discussion Purposes Only
ADS Updates – Status Check
![Page 18: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/18.jpg)
Confidential – For Discussion Purposes Only
Web ADS Major Changes – Poor Example
![Page 19: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/19.jpg)
Confidential – For Discussion Purposes Only
Web ADS Major Changes – Good Example
![Page 20: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/20.jpg)
Confidential – For Discussion Purposes Only
ACGME Citations in ADS – Revise Responses to Citations Each Year
![Page 21: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/21.jpg)
Confidential – For Discussion Purposes Only
Faculty & Resident Scholarly Activity in ADS:Annually Update for Previous Academic Year and Annually Update PD Curriculum Vitae
![Page 22: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/22.jpg)
Confidential – For Discussion Purposes Only
ACGME Letters of Notification in ADS
![Page 23: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/23.jpg)
Confidential – For Discussion Purposes Only
ACGME Letter of Notification (LON) in ADS & MedHub
![Page 24: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/24.jpg)
Confidential – For Discussion Purposes Only
Reasons for Letters of Notification from RRC
Citations: New Citations, Extended Citations, Resolved Citations Opportunities for Program Improvement Request for Progress Report Other Comments
![Page 25: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/25.jpg)
Confidential – For Discussion Purposes Only
Clinical Competency Committee
HowtheCCCdoesitsworkisdecidedbytheProgramDirector
![Page 26: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/26.jpg)
Confidential – For Discussion Purposes Only
Clinical Competency Committee
![Page 27: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/27.jpg)
Confidential – For Discussion Purposes Only
Do you provide the CCC members any evaluation data to review before the meeting? Stanford GME Survey sent to 325 Coordinators 1/15/2015
82.65% 17.35%0%
25%
50%
75%
100%
Yes No
![Page 28: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/28.jpg)
Confidential – For Discussion Purposes Only
CCC Data …
Case Logs
Resident (360) & End of Rotation Milestone
Evaluations
Duty HoursData
Resident Scholarly Activity
CCC MeetingsMilestone Evaluations
Clinical Skills
Assessment
Quality Improvement
Activities
SimLabs
In service In-service training exams
Safety Incident Reports
Progress on Milestones
![Page 29: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/29.jpg)
Confidential – For Discussion Purposes Only 29
“Each member reviews and ranks each resident prior to
the meeting.”
CCC faculty assignment and pre-work
![Page 30: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/30.jpg)
Confidential – For Discussion Purposes Only
Linking and Representing Milestone Evaluation Data
![Page 31: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/31.jpg)
Confidential – For Discussion Purposes Only
Creating a Resident Performance Profile – Visual Trends http://med.stanford.edu/gme/GME_Community/documents/conditional_formatting/conditional%20formatting%20example.swf
![Page 32: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/32.jpg)
Confidential – For Discussion Purposes Only
Data used in CCC meetings for trainee assessment – Stanford GME Survey sent to 325 Coordinators 1/15/2015
65.38%
93.85%
76.92%
44.62%
63.08%
36.92%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Aggregate DirectMilestone
Evaluations
Aggregate RotationEvaluations
Staff or patient(360) Evaluations
Technical SkillsAssessments/Mini-
CEX's
Scholarly Activities
Case Volumes
![Page 33: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/33.jpg)
Confidential – For Discussion Purposes Only
Semi-Annual Evaluations Must be a documented meeting with PD or APD and Trainee
Includes:
− Milestone / (CCC) Data
− Conference Participation
− Quality Improvement and patient safety involvement/project
− Scholarly/Research
− Procedure/Case/Patient Logs
− In-service scores
− Duty Hour Compliance
− Fatigue / Well Being
− Supervision: Adequate/issues
− Strengths and Weaknesses
− Career Counseling
![Page 34: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/34.jpg)
Confidential – For Discussion Purposes Only
Milestones
![Page 35: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/35.jpg)
Confidential – For Discussion Purposes Only
Semi-Annual Evaluations:PD can fill out the form while meeting with trainee.
35
![Page 36: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/36.jpg)
Confidential – For Discussion Purposes Only
ACGME Surveys
![Page 37: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/37.jpg)
Confidential – For Discussion Purposes Only
ACGME Residency Program Survey
![Page 38: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/38.jpg)
Confidential – For Discussion Purposes Only
ACGME Faculty Survey
![Page 39: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/39.jpg)
Confidential – For Discussion Purposes Only
Program Evaluations for Faculty & Residents
![Page 40: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/40.jpg)
Confidential – For Discussion Purposes Only
Summative Evaluations
![Page 41: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/41.jpg)
Confidential – For Discussion Purposes Only
Program Evaluation Committee / Annual Program Evaluation
![Page 42: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/42.jpg)
Confidential – For Discussion Purposes Only
Program Evaluation Committee (PEC) Must Monitor and Track (V.C.2):
1.Resident Performance
2.Faculty Development
3.Graduate Performance
4.Program Quality
5.Progress on the previous year’s action plan
![Page 43: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/43.jpg)
Confidential – For Discussion Purposes Only
Annual Program Evaluation (APE) – Pre APE
![Page 44: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/44.jpg)
Confidential – For Discussion Purposes Only
Resident Performance
The most recent aggregated written evaluations of the residents submitted by faculty and other evaluators
In-training/In-service exam scores
Procedure logs (if applicable)
Scholarly activity (publications, presentations, grant awards, etc.)
Learning portfolios: documented quality improvement activities
![Page 45: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/45.jpg)
Confidential – For Discussion Purposes Only
Faculty Development
ABMS certification status for all faculty
Updated faculty CVs
Documentation (faculty survey; attendance logs) of faculty participation in:− CME-type activities directed toward acquisition of clinical
knowledge and skills and also activities directed toward developing teaching abilities, professionalism, and abilities for incorporating the core competencies into practice
− Teaching (conferences, grand rounds, journal clubs, lecture-based CME events, workshops, directed QI projects, practice-improvement self study).
Faculty actively involved in mentor relationships with residents/fellows.
![Page 46: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/46.jpg)
Confidential – For Discussion Purposes Only
Graduate Performance
Aggregated board exam pass rates
Aggregated alumni survey results (typically, such surveys target physicians one year and five years after graduation –survey questions may inquire about such items as current professional activities of graduates and perceptions on how well prepared they are as a result of the program)
Other outcome measures
− Practice location (underserved areas)
− Academic Affiliations
− Scholarly Activity
![Page 47: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/47.jpg)
Confidential – For Discussion Purposes Only
Alumni Surveys …
![Page 48: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/48.jpg)
Confidential – For Discussion Purposes Only
Program Quality
The most recent aggregated written evaluations of the program submitted by faculty
The most recent aggregated written evaluations of the program (and/or specific rotations) submitted by residents
The most recent aggregated written evaluations of the faculty submitted by residents
Faculty’s recent scholarly activity (publications, presentations, grant awards, etc.)
![Page 49: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/49.jpg)
Confidential – For Discussion Purposes Only
Program Quality - Continued The most recent GME House Staff survey results (if
available)
The most recent GMEC Internal Review Report
Any recent communications from the ACGME or RRC
Program Report Card/Scorecard
− Trend Analyses
The most recent ACGME survey results
![Page 50: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/50.jpg)
Confidential – For Discussion Purposes Only
ACGME Program “X” Survey
![Page 51: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/51.jpg)
Confidential – For Discussion Purposes Only
ACGME Institutional Survey
![Page 52: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/52.jpg)
Confidential – For Discussion Purposes Only
Program Quality - Continued Curriculum
Overall and rotation-specific goals and objectives (Are they appropriate? Do they align with the core competencies?)
Didactic curriculum (Is there at least one regular conference targeted to the residents’ level?)
Opportunities for scholarly activity Compliance with any new standards established by the ACGME,
RRC, ABMS, etc.− Assessment Methods (Are evaluation tools appropriate? Do they align with the
core competencies?)− Resources: Personnel (PD, PC, faculty), Affiliated Training Sites,
Patient/Procedure Volume,
Learning Environment (space, call rooms, books, computers, etc.)
![Page 53: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/53.jpg)
Confidential – For Discussion Purposes Only
Progress on the Previous Year’s Action Plan
Review progress / (attempts to resolve problems) with respect to last year’s Annual Review delineating identified areas of weakness.
![Page 54: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/54.jpg)
Confidential – For Discussion Purposes Only
Annual Program Evaluation (APE) – SWOT SWOT Analysis
54
![Page 55: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/55.jpg)
Confidential – For Discussion Purposes Only
APE Guidebook
![Page 56: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/56.jpg)
Confidential – For Discussion Purposes Only
Annual Program Evaluation Action Plan Tool Action Plan
![Page 57: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/57.jpg)
Confidential – For Discussion Purposes Only
APE “SWOT” Analysis Tool Fishbone
![Page 58: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/58.jpg)
Confidential – For Discussion Purposes Only
Program Aims
AIM setting is part of the annual program evaluation Who are our residents/fellows? What do we prepare them for? − Academic / practice …− Leadership and other roles … Who are the patients/populations we care for?
AIMS are a way to differentiate programs Self-study ultimately evaluates program effectiveness in
meeting these aims Moves beyond improvement solely based on compliance
with minimum standards Assessment of relevant initiatives and their outcomes
![Page 59: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/59.jpg)
Confidential – For Discussion Purposes Only
SWOT ANALYSES – DefinitionsStrengths and Weaknesses – Internal Factors
Strengths
• Program factors that are likely to have a positive effect on (or be an enabler to) achieving your program’s aims are strengths.
• Important to acknowledge and celebrate
• What should definitely be continued (important question in an environment of limited resources)
![Page 60: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/60.jpg)
Confidential – For Discussion Purposes Only
SWOT ANALYSES – DefinitionsStrengths and Weaknesses – Internal Factors
Weaknesses• Program factors that are likely to have a negative effect on
(or be a barrier to) achieving your program’s objectives are weaknesses. Citations, areas for improvement and other information from ACGME
The Annual Program Evaluation and other program/institutional data sources
![Page 61: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/61.jpg)
Confidential – For Discussion Purposes Only
SWOT ANALYSES – DefinitionsThreats and Opportunities
Threats - Factors that pose risks.
External Factors and conditions that are likely to have a negative effect on achieving the program’s objectives, or making the objective redundant or un-achievable are called threats. • While the program cannot fully control them, beneficial to have plans to
mitigate their effect• What external factors may place the program at risk?
• What are changes in residents’ specialty choice, regulation, financing, or other factors that may affect the future success of the program?
• Are there challenges or unfavorable trends in immediate context that may affect the program? e.g., faculty burdened with heavy clinical load that prevents effective teaching and mentorship
![Page 62: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/62.jpg)
Confidential – For Discussion Purposes Only
SWOT ANALYSES – DefinitionsThreats and Opportunities
Opportunities are: Factors and contexts external to the program (institutional, local, regional and national) that can affect the program
Opportunities - Factors that favor the program, that the program may take advantage of / leverage
• External Factors that are likely to have a positive effect on achieving or exceeding your program’s objectives not previously considered are called opportunities.
What are capabilities for further evolving the program; how can the program capitalize on them?
Has there been recent change in the program’s context that that creates an opportunity?
Are these opportunities ongoing, or is there a narrow window for them? How critical is the timing?
![Page 63: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/63.jpg)
Confidential – For Discussion Purposes Only
Fishbone – Ishikawa DiagramSWOT Analysis Completed Example
To Train the
Academic Leaders of Tomorrow
AIM
![Page 64: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/64.jpg)
Confidential – For Discussion Purposes Only
There is so much data!!Data elements can be organized and leveraged for resident (CCC) and program (PEC/APE) evaluations and Web ADS to avoid duplicate work..
![Page 65: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/65.jpg)
Confidential – For Discussion Purposes Only
Additional Tools
Internal Surveys
Scorecards
Trend Analyses
![Page 66: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/66.jpg)
Confidential – For Discussion Purposes Only
GME Surveys – (Optional)
![Page 67: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/67.jpg)
Confidential – For Discussion Purposes Only
GME Surveys – (Optional)
![Page 68: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/68.jpg)
Confidential – For Discussion Purposes Only
How Can You Build a Scorecard? Easier than it looks!
![Page 69: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/69.jpg)
Confidential – For Discussion Purposes Only
External Measures
![Page 70: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/70.jpg)
Confidential – For Discussion Purposes Only
Internal Measures
![Page 71: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/71.jpg)
Confidential – For Discussion Purposes Only
Trend Analysis Example
![Page 72: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/72.jpg)
Confidential – For Discussion Purposes Only
Use Technology to Your Advantage…
Know your program requirements and follow them unconditionally
Use simple spreadsheet, calendaring and task organizational tools to manage, track and present resident performance data to your CCC
Resident education is a cyclical process –revisit and revise tools and processes each year
![Page 73: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/73.jpg)
Confidential – For Discussion Purposes Only
Recognizing time-consuming nature of work
… and need for support
![Page 74: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/74.jpg)
Confidential – For Discussion Purposes Only
The Toolbox
![Page 75: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/75.jpg)
Confidential – For Discussion Purposes Only
Electronic Toolbox for You!http://med.stanford.edu/gme/GME_Community.html Program Evaluation Committee Policy Template
Program Evaluation Checklist & Agenda
Annual Program Evaluation Guidebook: Diagrams & Action Plans
− Annual Program Evaluation Checklist
− Annual Program Evaluation Agenda (PDF / DOC)
− A Quick Method to Analyze Program Evaluations
Program Improvement Action Plan
![Page 76: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/76.jpg)
Confidential – For Discussion Purposes Only
Tools Can Be Downloaded www.gme.stanford.edu
GME Community Templates
![Page 77: 2017 ACGME Program Coordinators’ Workshop€¦ · program Opportunities - Factors that favor the program, that the program may take advantage of / leverage • External Factors](https://reader033.vdocuments.site/reader033/viewer/2022043003/5f83c7e06030b569a26e3c98/html5/thumbnails/77.jpg)
Confidential – For Discussion Purposes Only
Questions? Ann Dohn: [email protected]
Nancy Piro: [email protected]