1

Form a Better Learner: Milestones-Based Approach to Successful Remediation

Aditee Narayan, MD, MPH Betty Staples, MD

Kathleen Bartlett, MD Shari Whicker, EdD, MEd

Kathleen McGann, MD

2

Struggling Trainee Case 1 Matilda Jones, MD, PhD is a 2nd year Pediatric Cardiology fellow. She completed medical school in a reputable University and her Pediatric residency training at a well-known program with no concerns. Her test scores have always been in the low-moderate range compared with her peers. The Cardiology fellowship program director has been receiving complaints on Dr. Jones with increasing frequency. The nurses indicate that she is rude and condescending and does not address or ignores their concerns. Residents have said that she is always prepared with an answer to questions. However, they note that it is often not the correct answer. Attending evaluation comments include: 1. “Matilda is performing well below the level where she should be for her years of training. She has

made some progress, but she has a long way to go. Most importantly she needs to learn humility and to ask for help when she doesn't know something, but that means she will also need to learn to recognize when she needs help.”

2. “I do not feel as though Matilda is where she needs to be at her level as far as understanding hemodynamics, anatomy, catheter course, and other aspects of catheterization. I often felt uncomfortable and unsafe allowing her to obtain access and maneuver the catheters during the procedure. She does seem to be interested in learning and improving and was very helpful with the pre and post procedural management of the patient. I do see some improvement in her knowledge since the last time she was on cath but still feel as though it is below expected. I would like to see her spend more time with preparation for each case and understand better what the plans and goals of each case are rather than just showing up.”

3. “Many times I have been on call and Matilda will call me without the full patient story or the question we are being asked and why the echo is being requested after hours. Often times it is difficult even once I am here.”

4. “She is either not very compassionate or is not comfortable being compassionate, so explaining things to families is difficult for her.”

5. “Matilda is very defensive when approached with a concern in her deficiencies.” 6. “Matilda's performance was borderline during this rotation. She clearly worked hard, but there

were some deficiencies. In particular, her documented dictations were not processed in a timely manner. She made a good effort of trying to teach residents and students during the week. I never saw her examine a patient during the rotation, so I am not able to comment on this aspect of her performance.”

7. “Overall I think that Matilda struggles to compare and compete with her peers regarding her level of understanding of cardiology. She seems uncomfortable and/or unwilling to take a leadership role with teaching. She comes across as perturbed when challenged by attendings. Her compassion and consideration for families is very hard to assess. She doesn't seem to look at the "big picture" often and rarely contributes to help put things in perspective for the team of learners that she is mentoring on the team.”

3

Struggling Trainee Case 2

Sam Smith is a 2nd year Pediatric resident who is very committed to a career in Genetics. You just received an alert on an evaluation from his first ED rotation in November of 2nd year, which was triggered by a below-expectations score on the medical knowledge and documentation components of the rotation. Here are the specific comments from the ED rotation: “Sam’s knowledge of common conditions that present to the pediatric emergency department is lacking. As a result of his less than expected medical knowledge, he struggled with efficient initial evaluations of patients and development of appropriate ED plans. Without the appropriate medical knowledge, it is very difficult to perform hypothesis-driven, targeted initial H&Ps. Sam spent far more time in patient rooms during initial evaluations when compared to his peers. When the ED got busy, Sam was not able to adjust the pace/efficiency with which he worked.” This feedback is surprising to you because Sam has performed above the national mean on his ITE for the past two years in a row and has never had a “below expectations” score on prior evaluations. You have not personally worked with him since early in his intern year. You remember his dedicated care of patient with an inborn error of metabolism. At that time you found him to be very bright and extremely thorough, though sometimes late getting his notes done. In reviewing his recent evaluations, Sam typically gets 3s or “meets expectations” on a 5-point Likert scale. Here are some specific comments from the past few rotations: 1. Inpatient General Peds (June of intern year): “His presentations were disjointed; he functioned as

a collector/reporter of data, and offered very little in terms of data interpretation or plan development. He struggled with multi-tasking and had a difficult time juggling some of the day-to-day responsibilities as an intern. It seemed particularly hard for him to think forward in terms of discharge planning and advancing his patients' care.”

2. Continuity Clinic (fall of 2nd year): “STRENGTHS: Demonstrates professionalism and respect with families and coworkers. Acquisition of primary care fund of knowledge has improved. Has been noted to see patients for others who are behind. Documentation is SUPERB, thorough and pleasing to look out, making essential information from all sources available to others who happen to be seeing a patient he recently saw. Open and appreciative of teaching and feedback. SUGGESTIONS FOR FUTURE: Continue to work on streamlining presentations when discussing with attendings--scattered at times. Multitasking continues to be a struggle at times-- keep running list of "to-do" tasks, prioritize the most important to do NOW to keep patient visit moving toward completion. Challenge yourself to combine some physical exam components with history taking to cut down on some encounter times as they are sometimes very lengthy.

3. PICU (July of 2nd year): “Sam was eager to learn and did a good job of collecting data on the critically ill patients in the unit. Areas for improvement include assimilation of this data and working on succinct, focused presentations.”

4. Inpatient subspecialty rotation (September of 2nd year): “Terrific job by Sam on the cardiology team. He works well with the team. He knows the patients. He is learning about cardiology and applies what he knows.”

4

---This Page Intentionally Left Blank---

5

Selected Subcompetencies

Patient Care A. Gather essential and accurate information about the patient B. Organize and prioritize responsibilities to provide patient care that is safe, effective, and efficient* C. Provide transfer of care that ensures seamless transitions* D. Make informed diagnostic and therapeutic decisions that result in optimal clinical judgment E. Develop and carry out management plans Medical Knowledge A. Demonstrate sufficient knowledge of the basic and clinically supportive sciences appropriate to

pediatrics B. Critically evaluate and apply current medical information and scientific evidence for patient care

(see PBLI C below) Practice-based Learning and Improvement A. Identify strengths, deficiencies, and limits in one’s knowledge and expertise B. Systematically analyze practice using quality improvement methods with the goal of practice

improvement C. Locate, appraise, and assimilate evidence form scientific studies related to their patient’s health

problems (also covers MK B above) D. Participate in the education, of patients, families, students, residents, and other health

professionals D. Interpersonal and Communication Skills A. Communicate effectively with patients, families, and the public, as appropriate, across a broad

range of socioeconomic and cultural backgrounds B. Demonstrate the insight and understanding into emotion and human response to emotion that

allow one to appropriately develop and manage human interactions* C. Communicate effectively with physicians, other health professionals, and health related agencies D. Work effectively as a member or leader of a health care team or other professional group E. Professionalism (sub-competencies are integrated) A. Demonstrate humanism, compassion, integrity, and respect for others based on the

characteristics of an empathetic practitioner B. Demonstrate a sense of duty and accountability to patients, society and the profession F. Systems-based Practice A. Coordinate patient care within the health system relevant to their clinical specialty B. Work in interprofessional teams to enhance patient safety and improve patient care quality G. Personal and Professional Development* A. Develop the ability to use self-awareness of knowledge, skills, and emotional limitations to engage

in appropriate help-seeking behaviors B. Demonstrate trustworthiness that makes colleagues feel secure when one is responsible for the

care of patients C. Provide leadership that enhances team functioning, the learning environment and/or health care

system/environment with the ultimate intent of improving care of patients D. Recognize that ambiguity is part of clinical medicine and respond by utilizing appropriate

resources in dealing with uncertainty

6

Selected Subcompetencies + associated milestones PC-A

7

PC-B

8

PC-C

9

PC-D

10

PC-E

11

MK-A Demonstrate sufficient knowledge of the basic and clinically supportive sciences appropriate to pediatrics

Not yet Accessible Level 1 Level 2 Level 3 Level 4 Level 5

Does not know or remember the basic content knowledge of common pediatric problems and illnesses.

Understands the basic content knowledge of pediatrics, but is still learning to apply it to clinical situations.

Able to analyze and categorize knowledge in a way that allows the generation of a meaningful differential diagnosis.

Able to evaluate knowledge and use it appropriately in a given clinical encounter to develop meaningful clinical management plans.

Learns from experience; analyzes a situation, evaluates what worked well and what did not work well in the past, and creates, adapts, or extrapolates information appropriately to new clinical situations and encounters.

MK-B (see PBLI C below)

12

PBLI-A

13

PBLI-B

14

PBLI-C (also covers MK-2)

15

PBLI-D

16

ICS-A

ICS-B

17

ICS-C

18

ICS-D

19

PROF-A

PROF-B

20

SBP-A

21

22

SBP-B

PPD-A

23

PPD-B

24

25

26

PPD-C

27

PPD-D

28

Corrective Action Worksheet

Case #_______ Corrective Action or Enhanced Learning Plan (circle one)

1 2 3 4

Subcompetency Current Milestone Level

Desired Milestone Level

Strategies for Improvement

29

Remediation Policy First and foremost, the foundation for the remediation process is in education and enhanced learning rather than discipline. The leaders of the residency program understand that there is a great deal to learn over the course of residency, and we are invested in helping each individual become the best pediatrician possible. Our mission is to train physicians to take excellent care of children, and we recognize this goal as our obligation to our residents as well as our responsibility to society. Fundamental to education, the process must include both teaching and assessment, and the assessment process must be detailed, thoughtful and ongoing. Residents are assessed based on the Pediatric Milestones as developed by the ACGME. Milestone assessments are incorporated in various rotation evaluations, self-assessments, and advisor assessments. Additional milestones based tools may also be used as needed. Residents are expected to be at appropriate milestones as they progress through residency. Inability to perform at the appropriate level may indicate need for remediation. When we think assessment, many may just think of written evaluations. Please keep in mind that assessment includes a multi-rater approach, including faculty, peer, nursing and parent evaluations. Additionally, we use in-training exams, procedure logs, direct observation and individualized learning plans with self-reflection. All of these many components are discussed in the context of the regular advisor/advisee meetings which occur at least two times a year. Advisors openly discuss their perspective on the resident’s progress and elicit the resident’s perspective as well. Advisors attend a three times a year competency meeting (November, Feb/March, and May) where the progress of each resident is discussed in the context of the 6 core competencies and milestones. Members of the clinical competency committee (CCC) include the resident advisors (program director, associate program directors, Vice Chair of Education, and continuity clinic director), the two med/peds program directors, the chief residents, Director of Medical Education and representatives from the ED and PICU. The advisor presents data reviewed during the previous advisor meeting. Based on the CCC meeting, advisors will complete a milestones assessment for each resident twice a year. This represents an overall assessment by the committee. If further issues arise, the advisor will meet again with his/her advisee. When issues of competency arise, there is a well-established step wise plan to address these issues with the goal being the ultimate success of the trainee. Each step of this plan is rooted in 3 principles:

1. The process will be education based not discipline based. 2. The process will be absolutely transparent to the individual engaged. The resident will be aware of what the

issues are, what the plan is to address the issues, what the timeline is, and what the potential consequences of failure to address could be.

3. The process will be absolutely confidential to those not involved in the plan. It will not be shared with colleagues or faculty who are not relevant. As long as duration of training is not affected, it will never be shared with other institutions, medical boards or employers as this is peer review documentation protected by state statute.

Generally, the first step is an Enhanced Learning Plan. All residents construct an individualized learning plan at the start of the academic year. When there is an area of concern, residents work with their advisor directly to construct a strategy to address the concern. This written plan will have a timeline for completion (generally 2-3 months) and is completely internal to our program. If the issue is not corrected within the context of an Enhanced Learning Plan, if the issue requires higher level of intervention or if the issue relates to professionalism, we proceed to a Corrective Action Plan under the guidelines of the Duke GME office. Development of a corrective action involves the resident, advisor, program director and the Chief Resident as your advocate with oversight by the CCC. In the event that the program director is the advisor, the Vice-Chair for Education, assists with the plan. Corrective actions also involve a written document which clearly describes the issues of concern, develops a strategy for remediation, and a timeline for completion with a meeting at the mid-

30

point. The trainee is informed that the options at the end date of the corrective action are completion of the corrective action, continuation of the corrective action, or termination from the program in the extreme case that the issues are severe and not fixable after multiple attempts to do so. The corrective action process includes a mechanism for the trainee to appeal the program’s decision to Duke DIO. There are three points that are important to recognize

1. The competency committee does not presume to be the best resource to address all issues. The department has been very generous with its support and commitment to assist with necessary resources available in the community or elsewhere in the institution, including but certainly not limited to- expert coaching in communication, professionalism, test taking strategies, mental health, and medical health assessment.

2. Our program has never and will never be a “one and done” sort of system. Residents are given ample opportunity to remediate, including multiple corrective actions, if necessary. Ultimately, we must recognize our dual responsibility to society, as well as the trainee, to send out physicians who will be competent providers. The only exception to this approach is a violation of the policies of the North Carolina Medical Board or Duke institutional policy regarding trainees which can be found in the GME Trainee Manual and Benefits Guide at https://gme.duke.edu/sites/default/files/files/2010-2011%20GME%20Trainee%20Manual.pdf.

3. The process will remain educationally based, transparent and confidential.

31

TEMPLATE for PROGRAM TRAINING DIRECTOR

Consider giving this to the trainee in person. Strongly consider an additional person from the program. Options might include the faculty advisor or faculty advocate of the trainee’s choosing present, the Vice Chair of Education and/or chief resident. We also strongly advise asking the resident to include a spouse or family member if this is a “high stakes” corrective action. This document should be the result of the Clinical Competency Committee not the program director “alone.” The program director and advisor should then provide the information to the trainee to help interpret it for them. (Blue italicized print are instructions to PTD and should be removed before Plan is finalized). Date: __________________ Dear XXX, This document is meant to provide a summary of our discussion today in which we indicated to you that would be plan on ___________ (routine or adverse) Corrective Action from ____________(date) to _____________(date) (typically 2-3 months). Based on (identify relevant assesments), your performance was judged to fall short of expected competencies in __________: (consider using the ACGME competency language with specific sub-competencies):

• A • B • C

These concerns are translated into the context of the Pediatric Milestones for further clarification. (What follows is a link to the document entitled The Pediatrics Milestone Project which was prepared as a joint initiative of the ACGME and American Board of Pediatrics: https://www.abp.org/abpwebsite/publicat/milestones.pdf) At this time, the following Pediatric Sub-Competencies are found to be below that expected for your level of training for the XX and YY rotations.

Patient Care: Communication: Medical Knowledge:

• List deficient sub-comptencies under each competency with the trainee’s current milestone attainment

As a result of this/these issue(s) the trainee will be expected to: List specific sub-competencies and expected milestone level attained by conclusion of corrective action To achieve these goals we would recommend the following strategies and impose the following requirements: List required activities and recommended activities

32

The following methods may be used to assess Milestones levels: list assessment tools Recognizing that remediation can be stressful, it is recommended that you consider contacting the Personal Assistance Service (PAS) at 416-1727 or an independent health professional/ counselor/ therapist. Dr. YYY at Employee and Occupational Health or the GME office can provide you with some names if you prefer.

If you believe that you have a health problem that interferes with your ability to perform your job, it is important that you access the following resources as quickly as possible. These include:

o Personal Assistance Service (PAS) 416-1727 (free and confidential counseling) o Employee Occupational Health and Wellness 684-3136 (you can arrange a consultation with

them) o Duke’s Reasonable Accommodation Process 684-8247 (you may request an evaluation for

accommodation of a disabling impairment that limits your ability to accomplish your job) At the midpoint of this Corrective Action Plan (___________(specify date)) you will meet with _______________________ (specify who; usually the program director and perhaps faculty advisor/advocate) to discuss your progress and receive a written evaluation. At the end of this Corrective Action Plan and after receiving and reviewing ____________________ (specify what information you will use to judge performance) Dr. ___________(usually the program director) will meet with you to discuss your progress. At that point, the program may

- Remove Corrective Action and return to your regular progress within the residency or fellowship with a new completion date of ____________ should all the above areas be adequately remediated based upon successful performance.

- Continue Corrective Action for another specified period of time with continued close monitoring if you

have made positive but still incomplete progress and are still not at the expected level of competency. This may require additional remedial months and additional activities which will be outlined at that time.

- Decide to “non–renew” your contract at the end of the contract year or “terminate” your contract

immediately. If any of this is true, add the following: You should be aware that your anticipated date of completion may be delayed by _____________ months. The Board of___________ will be notified of your extension of training.

If you wish the corrective action reviewed you must request the review in writing or by email of Dr. ZZZ, the ACGME Designated Institutional Official and Associate Dean for GME within 7 days. Additional information about the Corrective Action and Hearing Procedures was provided to you at Orientation and available at the GME web site (http://www.gme.duke.edu/).

33

Your signature below signifies that you received this document and had the opportunity to ask questions. ______________________________________________ ______________________________ Program Training Director Date ______________________________________________ ______________________________ Faculty Advisor (if applicable) Date ______________________________________________ ______________________________ Trainee Date ______________________________________________ Others present during meeting? CC: YYY, MD, DIO, and Director Graduate Medical Education