Chief Resident Orientation Agenda

Friday, April 30, 2010, 12:00-4:30

Brennan Center, Riverside East (Parking will be Validated for those Parking in the Red Ramp)

Time Topic Presenter

12:00 - 12:10 Lunch and Introductions Dr. Louis Ling, Associate Dean for GME

12:10 - 12:30

Navigating UMMC, F Mira Jurich, GME Coordinator, Fairview

Health System and

Lynne Peterson, Manager of Physician

Recruitment

12:30 -1 :15 Surviving as Chief Resident Chief Resident Panel

- Tacjana Friday MD

- Diane Horvath-Cosper MD

- Kaz Nelson MD

1:15 - 1:30 ~ BREAK ~

1:30 - 2:30 Conflict Resolution Rosie Barry, Program Director, Office of

Human Resources

2:30 - 3:00 Knock ‘em Dead Presentations Kaz Nelson, M.D.

Chief Resident, Psychiatry Residency

Program

3:00 - 3:15 ~ BREAK ~

3:15 - 4:15 Residents in Difficulty -

Remediation

Phillip Rauk M.D.

Director, Obstetrics & Gynecology Residency

Program

4:15 - 4:30 Support Resources for Residents

and Fellows

Dr. Marilyn Becker, Director of Learner

Development

Adjourn

Fairview Health Services

Chief Resident Retreat6/11/09

Chief Resident RolesHospital Perspective

• Supervision of junior trainees• Assist hospital with policy and procedure

compliance – role model appropriate behavior• Committee participation• Interface with program directors• Adverse event/root cause analysis• Resource to faculty, hospital, administration,

trainees, students

Chief ResidentKey Hospital Issues - 2010

• Hand washing• Immediate BBFE reporting and follow-up• Site marking for all procedures• Order authentication within 24 hours• Timely discharge planning• Immediate dictated discharge summaries• Response to documentation queries

PROFESSIONALISM

Fairview Health ServicesCode of Professional Behavior

Code of Professional Behavior

• Place the patient at the center of all we do• Apply the best science we know• Model the highest degree of professionalism• Actively engage as a collaborative member of the

care team• Be aware of, and comply with the rules

Patient Centered Care• Be available and approachable• Provide all needed information to patients

and staff regarding their treatments and their choices

• Advocate for the patient and family• Respect confidentiality• Do our best to meet patient needs within

the constraints of science, ethics, and available resources

Apply the Best Science

• Maintain professional knowledge by continuing education, reading, learning from colleagues

• Consult appropriately• Acknowledge that I am an educator for

patients, families, and colleagues• Disclose conflicts of interest

Model Professionalism• Share knowledge proactively• Communicate effectively and respectfully• Challenge others respectfully• Avoid speaking negatively about other health

care providers• Model appearance and deportment in a manner

which instills confidence and provides comfort• Refrain from sexual contact or romantic

relationships with current patients• Avoid conduct or activities which could impair

judgment and ability to act competently

Team Collaboration• Actively engage in team conversations,

meeting, rounds• Share helpful information• Listen carefully and well• Communicate effectively with referring

physicians• Respond to colleagues and staff in a timely

manner• Manage hand-offs

Comply with Rules

• Know and follow pertinent hospital policies

• Monitor my own behavior, and the behavior of others

• Provide honest feedback and coaching

Questions?For more information:

Jim Breitenbucher, M.D.VP Medical Affairs & Clinical Operations

612-273-6086, jbreite1@fairview.org

Mira JurichCoordinator, Graduate Medical Education

612-273-7482, mjurich1@fairview.org

Fairview Health Services

Presented By:Lynne Peterson

Fairview Physician Recruitment

Fairview Services

• Practice Management

• Moonlighting Opportunities

Practice Management• Getting Started

• Developing Your Curriculum Vitae

• Interviewing & What To Expect

• Compensation & Negotiating Your First Position

Getting Started -Initiating the job Search

Self Examination of Interests– Type of Practice– Practice Considerations– Personal Considerations

Resources: What are they?– How to begin and where to look

Timeline: – When should you start– How long does it take?

Developing Your Curriculum Vitae

What’s important to include– Must haves– What’s important highlight

What doesn’t have to be included

What employer’s look for

Interviewing & What To ExpectPreparing for the interview

The interview– Timing– What to expect

Assessing the practice

Follow-up to the interview

Negotiating Your First Position

Receiving an offer– Tips and Etiquette

Contracts/Employment Agreements– Contractual terms– What to look for– What to avoid– Review of the Agreement

Compensation

Industry Terms

Compensation Methodologies

Market data– What to expect in the first year (or two)– Beyond the guarantee

Timing• Getting Started (20 min)

• Developing Your Curriculum Vitae (20 min)

• Interviewing & What To Expect (30 min)

• Compensation & Negotiating Your First Position (30 min)

Forums for Education TopicsEvening gatherings

Designated practice management forums

Lunch-n-Learn

Open to other ideas

Moonlighting Opportunities• House Officer

– General– Surgical

• Urgent Care

• Hospitalist

• Peds BMT

Moonlighting Opportunities

• Employed by Fairview in a casual status.– Compensated at an competitive hourly rate– Malpractice Paid– Employer FICA contribution– Hours can be tracked and reported to

residency program.

Moonlighting Requirements• Meet specialty requirements for the

specified role

• Completed at minimum 1 year of residency training.

• Able to obtain a MN License and DEA

Contacts

Practice Management

Lynne PetersonFairview Physician Recruitment612-672-2285lpeters1@fairview.org

Moonlighting

Dianne GustafsonFairview Provider Staffing Office612-672-7219dgustaf1@fairview.org

Intro – 5 mins

1. Logistics2. About our office3. About the class 4. About “fluency” in conflict

‐ what does it take to be “fluent” in a language‐ practice, context, immersion, knowing structure, grammar, etc.

‐ assumption – they already have tools in dealing with conflict – goal of this workshop, provide more tools (hammer and nail metaphor), to work towards conflict fluency

Normalizing conflict issues at the U:  When OE first announced a conflict session at 5pm on a Friday, it was full by Monday at 8 a.m.

1

This is normally an 8 hour class with plenty of time to practice, but not today!

2

This module is an introduction.  In this  course there is a recognition that  conflict is not always “resolvable”.  That’s a different approach.

The Conflict Fluency course focuses on skill building RE conflict fluency, ie, the ability to interact with and engage in conflict in intentional and effective ways – stress ‘effective’ means engaging and managing conflict in a way that helps achieve the goals you are striving for.

Given the nature of conflict, this workshop deals with competencies needed to help us deal constructively with conflict and lessen the possible destructive impact of conflict interactions.

Fluency: how many of you know or have learned a second language.  What does fluency in a second language require?  (practice)

Are you fluent in the language of conflict … where do we learn it and what do we learn?

3

Large Group Discussion:What is the negative side of conflict?  Identify specific examples, usually easy to do

Optional:  Small Group Activity‐What are the positive side or benefits of conflict?Not as simple to do this.  Has the group experienced any of these positives?

4

We will be looking at three key skill areas today.

NAMING – what is and what is not conflict.  Have you experienced situations where some thought there was a conflict and others disagreed?

experience in our bodies, our minds and our psyche

FRAMING – what we think it’s about

TAMING – what to do with it – conflict styles

5

The “Sage Handbook on Conflict Communication” has a sample of 8 different definitions of conflict from various scholars

Description of intractable – 6 months and may never be resolved.  The conflict you’re thinking about may or may NOT be intractable.

Remember: we want to engage, manage, live with and accept the conflicts 

Emotional upset is key and critical to recognize and work with.  Some conflict guidance tells us to minimize the emotions, or stick to the issues and put aside the emotions.  The critical skill is how to use the emotions that are there to “tack” or to help move the parties back towards discussion.  (‘Tacking’ is(a sailing term RE how to navigate from point A to point B when the wind is not directly at your back – it is a matter of ziggingand zagging in a generally forward direction to reach point B.) 

The 3 “in’s” – interaction, incompatibility, interference.

6

This slide illustrates the impact of effectively listening, and especially using Reflecting and Mirroring behaviors/skills to communicate that you have paid attention to, and truly ‘heard’, the other person.

7

Before slide:   Think back to the conflict you just discussed.  What are 

1. 3 things that made it difficult.  (Pause and let them write)2. 3 things that would have helped.3. List what you think the conflict is about.

Discussion/Lecturette

‐ What is the conflict about?  Give an example

‐ Usual focus on material – usually works; but be aware of possible symbolic and relational issues

‐ Ongoing, “intractable” conflicts tend to be at the relational and symbolic levels –different interventions needed

‐ Best approaches tend to reach all 3 levels

8

Taming – we all have general patterns how we go about engaging, managing and resolving conflict.  Those patterns, and any pattern of behavior, can be attributed to personality, biology, and culture.  This diagram is an oversimplification – but it helps to illuminate what’s going on.

Personality – MBTI – the way we like things to be, our preferencesBiology – brain research on male and female brains and empathyCulture – the way we were taught how to think about and approach conflict and other people.

This builds on the bio‐psycho‐social concept of psychology – how we’re raised, male‐female and how our brain works.

A lot of our patterns of behavior are unconscious and, as animals, we are very good at reading these behavioral signals.  Brain research – Alex Pentland’s Honest Signals is an example of how information is available to those who pay attention.

In this training, we focus on culture – based on the belief that how we go about dealing with conflict is something we learned growing up in our families, our communities etc.

We have options – theses three things identify the basis of our habits..

9

Patterns of shared basic assumptions and behaviors that help us approach challenges/problems that are learned/taught to all members (implicitly and/or explicitly) as the correct way to perceive, think and feel out of our awareness – rare that we are aware of that filter

10

Culture is in constant flux.  As conditions change, cultural groups adapt.  Think about the places that you have worked – did the culture change from place to place – what was similar and what was different?  How do new people joining your group change the culture?

1. Iceberg model ; just as 9/10’s of things are under the surface, so two 9/10’s of culture is out of conscious awareness.  Eye behavior in interviewing for example.

11

Imagine a continuum – direct/indirect – where would you be?

Expressive /restrained – where would you be?

12

Discussion Style: a direct, rational approach.  Emphasizes precision in language and is described as “say what you mean and mean what you say.”

Engagement Style: a direct, emotional approach.  This style sees the overall sincerity of the person as grounded in a concern illustrated through intense expression of emotion.  This could be seen as ‘positive’ or ‘negative’ emotion, e.g. excitement or anger.

Accommodation Style:  This style emphasizes not “letting the conversation get out of control.”  Intense emotion is seen as dangerous.  Stories and metaphors, intermediaries and keeping conflict low is emphasized.  ‘beating around the bush’

Dynamic Style:  In this style, credibility is built through a trust because people are letting their emotions out.  The message will be indirect and expressed through metaphor or humor, but the emotion is shared.

ACTIVITY:  If the group has completed the actual instrument, refer to the Strengths and Weaknesses of the conflict styles on page 13 of the interpretive Guide Activity and have them combine with the activity.

Have people with a partner discuss where they see themselves, their experience with each quadrant, and what they need to develop.

ACTIVITY 2: Have participants identify their primary client’s style and how they interact with that.

13

Emphasize – we shift all the time, depending on who the person is, what the conflict is about, how we feel that day, etc.

14

The most basic communication skills – basic but probably undeveloped.  Also probably unconscious.

ASK:  How many have had actual training in these skills? 

When so much depends on these skills, why do we spend so little time developing them?

15

Basics of listening

Attending = giving attention to the speaker; our own experience, culture, etc impacts how we attend and what we hear

Monitoring = self management; processing mentally before choosing our action; asking what is my most appropriate and constructive response to this input?

Responding = acting on our choice after weighing the input, possible words and actions

16

These are BEHAVIORS we use in RESPONDING mode; all are appropriate when the time is right

17

18

Knock ‘em dead Presentations

Kaz Nelson, MD

Is it time to ditch power 

point?

The Casper Pigeon 

Project

Can you pass the elevator test?

Less Than

85% Normal 

body weight

Avoid Weak Verbs

0

2

4

6

8

10

12

14

Met/Met Val/Met Val/Val

Percent with SchizophreniformDisorder at Age 26

1. Slides the Audience will See

2. Notes Only you will see

3. Handout to be taken away

Beneficence Autonomy

Psychosis vs. Delirium

Kaz Nelson, MD

Psychosis vs. Delirium

Kaz Nelson, MD

University of Minnesota Dept of Psychiatry

April 30th, 2010

Bipolar 

Type 1Bipolar 

Type 2

Mania 

(with and without hypomania)

Hypomania  

(without mania or psychosis and some depression.)

Euthymia

Mania

Depression

Typical Course

January 2009 END NOTES

May I have your attention? By Jon Hallberg, M.D.

Confessions of a reformed PowerPoint user.

I think I owe several of you an apology. Over the years, I’ve given dozens of presentations to groups large and small on topics ranging from bubonic

plague to steroid use in athletes. Perhaps you were present for one of them. If so, it’s to you I owe the apology.

I gave many of those talks using royal blue slides with yellow lettering. (I was told this color scheme would make them especially easy to read.) I loaded the slides with as much information as I possibly could, although I

tried to limit the number of bullet points to six per slide. (I was told this was the optimal number.) I made sure my slides would be easy to print. (Six per

page.) I often read directly off my slides, as you read along with me—or ahead of me. I wonder now, how in the world did I keep your attention? (I

suspect I didn’t.)

As I think back on some of those talks, I cringe. They must have been

awful—dull, text rich and image poor. Where was the story? The pull? The hook? What was I thinking? And when did I fall into the trap of giving visually boring presentations? I can tell you. It was 1997, the year I

discovered PowerPoint. But this is about to change. I’ve become a reformed PowerPoint user. And here’s why.

I discovered Presentation Zen: Simple Ideas on Presentation Design and

Delivery by Garr Reynolds. If you give presentations, you need to read this

book. For me, a single read-through changed the way I give talks and view them. I’ve been so taken by Reynolds’ message, I’m now on mission to

improve the quality of medical presentations. Encouraging you to read his

book is the simplest way I can do that.

I stumbled on Presentation Zen by accident. Early in 2008, I read another

great book, Daniel Pink’s A Whole New Mind: Why Right-Brainers Will Rule the Future. I was so intrigued, I went to Pink’s website. There, I saw a link

to his next book, The Adventures of Johnny Bunko: The Last Career Guide

You’ll Ever Need, billed as the first U.S. business book in manga, or

Japanese comic book form. For a description of the Bunko book, I was

directed to a slide show. I navigated through the more than 100 slides in

about five minutes. They were simple and stark, mainly black and white with a little red for accent. They contained few words (sometimes only one), and

each slide presented no more than a single idea. I wondered who created this thing. I clicked on another link and found out it was a guy named Garr

Reynolds.

Reynolds is an expert on presentation design and delivery who lives in

Japan. He loves simplicity, elegance, and white space. Reading his book

(itself a thing of beauty), you immediately begin to see why most of our presentations are really awful. We cram too many words (and graphs and

charts and data) onto our slides, and as speakers, we literally read off of

them. (I think this is often the fault of conference organizers who ask for a copy of our slides ahead of time.)

In this slim book, just over 200 pages, Reynolds covers such ideas as

creativity, crafting a story, simplicity, being present, and connecting with the audience. He shares several sample presentations, covering everything from sustainable food to aromatic chemistry. (If a presenter can make the

properties of tetravalent carbon visually interesting, then those of us in medicine can surely make an update on congestive heart failure more

engaging.) Reynolds also recommends a number of other books and websites, including my new favorite, the TED (for Technology, Education,

and Design) conference site. (If you want to see how master presenters make superb use of PowerPoint and other visual tools, check out

www.ted.com/talks.)

So why should physicians care about improving their PowerPoint presentations? As long as medical schools and medical conferences continue

to offer lecture-like teaching, PowerPoint will continue to be the medium

through which information is shared. And if that’s going to be the case, we presenters have a responsibility to improve our presentations. I can’t think

of a better place to start this sea change than by reading Presentation Zen.

MM

Jon Hallberg is medical director of the new University of Minnesota Physicians’ Mill City Clinic.

Presentation ZenHow to Design & Deliver Presentations Like a Pro

By Garr Reynolds(info@garrreynolds.com)

This brief handout, highlights many of the key points made in my recent presentations and seminars onpresentation design. You will also find a bibliography of suggested readings and links to websites referred toin the presentations.

Zen and effective presentationsIn the presentation, we discussed the current state of business presentations today which, more often thannot, incorporate the use of PowerPoint in ways that actually undermine the speaker’s good intentions. Bullet-point filled slides with reams of text become a barrier to good communication. We have become accustomedto a “PowerPoint culture” in which a disconnect exists between the audience and the presenter. Manypeople, including many top business leaders, are fed up with PowerPoint. But it is not PowerPoint’s fault —PowerPoint is just a tool.

"Technical knowledge is not enough. One must transcend techniques so that the art becomes an artless art, growing out of the unconscious." — Daisetsu Suzuki

If we apply some basic, accessible concepts borrowed from the world of Zen, we can improve oureffectiveness and allow our content to connect in more powerful ways. One key concept is simplicity.However, simplicity is not merely a means to more effective communication. Rather, it is a consequence ofour “Letting Go” of bad habits and much of what we have learned about multimedia presentations in the eraof PowerPoint. Other important concepts include: The beginner’s mind; Being fully in the momentModeration, or “the middle way;” Minimization of chaos and clutter.

“Making the simple complicated is commonplace; making the complicated simple, awesomely simple, that's creativity.” — Charles Mingus

In a nut shell: PowerPoint culture causes both audiences and presenters to suffer. And content suffers too.The root of the suffering is attachment to old PowerPoint habits and misunderstandings about how best toconnect to an audience. Lose your attachment to the “normal” way PowerPoint is used and lose poorpresentation habits to move to a higher level of effectiveness.

Effective use of multimediaWhen designing our presentations and creating the supporting visual aids, we should keep in mind the wayour audience will actually process our presentation. We must design our visuals and use PowerPoint in waysthat take advantage of how people process information. Much can be learned, then, from a review of the keyfindings in the field of cognitive science concerning how people learn best in multimedia presentation settings.Below, cognitive scientist, Dr. Richard Mayer, summarizes the three assumptions of multimedia learningtheory.

“Cognitive scientists have discovered three important features of the human information processing systemthat are particularly relevant for PowerPoint users: dual-channels, that is, people have separate informationprocessing channels for visual material and verbal material; limited capacity, that is, people can payattention to only a few pieces of information in each channel at a time; and active processing, that is,people understand the presented material when they pay attention to the relevant material, organize it into acoherent mental structure, and integrate it with their prior knowledge.”

— Rich Mayer, in an interview with Sociable Media, Inc.

2

(1) Review of Key Findings

• Multimedia Effect. Narration with pictures (visuals) is better than narration alone.• Modality Principle. People learn better when words are presented as narration rather than text.• Redundancy Principle. People learn better from narration & graphics rather than narration, graphics, & text.• Coherence Principle. People learn better when extraneous visual material is excluded.

(2) Practical Implications for better PowerPoint Presentations

• Presentations must be both verbal & visual.• Too much slide information overloads people’s cognitive systems.• Can your visuals be understood in 3 seconds? If not, redesign them to support your talk.• Slide design & delivery must help people organize, integrate information.

Organization & Preparation Tips

PowerPoint is not inherently a bad tool. In fact, if presenters just avoid a few of the most commonPowerPoint pitfalls, their presentations will greatly improve. Below, many of the items discussed in thepresentation are highlighted in brief.

(1) Start with the end in mind. Before you even open up PowerPoint , sit down and really think about theday of your presentation. What is the real purpose of your talk? What does the audience expect? In youropinion, what are the most important parts of your topic for the audience to take away from your, say, 50-minute presentation? Remember, even if you've been asked to share information, rarely is the mere transferof information a satisfactory objective from the point of view of the audience. After all, the audience couldalways just read your book (or article, handout, etc.) if information transfer were the only purpose of themeeting, seminar, or formal presentation.

(2) Plan in “analog mode.” That is, rather than diving right into PowerPoint (or Keynote), the bestpresenters often scratch out their ideas and objectives with a pen and paper. Personally, I use a largewhiteboard in my office to sketch out my ideas (when I was at Apple, I had one entire wall turned into awhiteboard!). The whiteboard works for me as I feel uninhibited and free to be creative. I can also step back(literally) from what I have sketched out and imagine how it might flow logically when PowerPoint is addedlater. Also, as I write down key points and assemble an outline and structure, I can draw quick ideas forvisuals such as charts or photos that will later appear in the PowerPoint. Though you may be using digitaltechnology when you deliver your presentation, the act of speaking and connecting to an audience — topersuade, sell, or inform — is very much analog.

(3) Good presentations include stories. The best presenters illustrate their points with the use of stories,most often personal ones. The easiest way to explain complicated ideas is through examples or by sharing astory that underscores the point. Stories are easy to remember for your audience. If you want your audienceto remember your content, then find a way to make it relevant and memorable to them. You should try tocome up with good, short, interesting stories or examples to support your major points.

(4) It’s all about our audience. There are three components involved in a presentation: the audience, you,and the medium (in our case, PowerPoint). The goal is to create a kind of harmony among the three. Butabove all, the presentation is for the benefit of the audience. However, boring an audience with bullet pointafter bullet point is of little benefit to them. Which brings us to point number five, perhaps the mostimportant of all.

“Respect those who come to you with open ears and foster a sense of community.” — P.T. Sudo

3

(5) Reduce the text on your slides to an absolute minimum. The best slides may have no text at all.This may sound insane given the dependency of text slides today, but the best PowerPoint slides will bevirtually meaningless with out the narration (that is you). Remember, the slides are suppose tosupport/supplement the narration of the speaker, not make the speaker superfluous. Yes, it is true that manypeople often say something like this: “Sorry I missed your presentation, Steve. I hear it was great. Can youjust send me your PowerPoint slides?” Well, you could. But if they are good slides, they may be of little usewithout you.

(6) Do not read the text word for word off the slide. Audiences can read, so why do presenters insiston reading long lines of text from slides? Also, it is very difficult — if not impossible — to read a slide andlisten to someone talk at the same time. So again, why all the text on slides these days? One reason may bethat it is convenient for the speaker when organizing the presentation to write out his/her thoughts onebullet point at a time. But as Yale professor and visual communications specialist, Edward Tufte points out ina September Wired Magazine article “…convenience for the speaker can be punishing to both content andaudience.” Speakers also may be thinking that their wordy slides will make for better handouts, a common“handout” technique. However, the confining, horizontal orientation of a slide (one slide after another)makes for difficult writing and reading. Which brings us to the next point below.

(7) Written documents (research papers, handouts, executive summaries, etc.) are for theexpanded details. Audiences will be much better served receiving a detailed, written handout as a takeawayfrom the presentation, rather than a mere copy of your PowerPoint slides. If you have a detailed handout orpublication for the audience to be passed out after your talk, you need not feel compelled to fill yourPowerPoint slides with a great deal of text.

Remember: (1) your slides should contain only a minimum of information; (2) your slide notes, which onlyyou see, will contain far more data; and (3) your handout will have still far more data and detail.

Slide (PowerPoint) Tips

(1) Keep it simple. PowerPoint was designed as a convenient way to display graphical information thatwould support the speaker and supplement the presentation. The slides themselves were never meant to bethe “star of the show.” People came to hear you and be moved or informed (or both) by you and yourmessage. Don't let your message and your ability to tell a story get derailed by slides that are unnecessarilycomplicated, busy, or full of what Edward Tufte calls "chart junk." Nothing in your slide should besuperfluous, ever. Your slides should have plenty of "white space" or "negative space." Do not feel compelledto fill empty areas on your slide with your logo or other unnecessary graphics or text boxes that do notcontribute to better understanding. The less clutter you have on your slide, the more powerful your visualmessage will become.

Simplicity is the ultimate sophistication. — Leonardo da Vinci

(2) Avoid using Microsoft templates. Most of the templates included in PowerPoint have already beenseen by your audience countless times (and besides, the templates are not all that great to begin with). Youcan make your own background templates which will be more tailored to your needs or you can purchaseprofessional templates on-line (for example: www.powerpointtemplatespro.com).

(3) Avoid using PowerPoint Clip Art or other cartoonish line art. Again, if it is included in thesoftware, your audience has seen it a million times before. It may have been interesting in 1992, but today theinclusion of such clip art often undermines the professionalism of the presenter. There are exceptions, ofcourse, and not all PowerPoint art is dreadful, but use carefully and judiciously.

(4) Use high-quality graphics including photographs. You can take your own high-quality photographswith your digital camera, purchase professional stock photography, or use the plethora of high-quality imagesavailable on line (be cautious of copyright issues, however). Never simply stretch a small, low resolutionphoto to make it fit your layout — doing so will degrade the resolution even further.

4

(5) Use object builds and slide transitions judiciously. Object builds (also called animations), such asbullet points, should not be animated on every slide. Some animation is a good thing, but stick to the mostsubtle and professional (similar to what you might see on the evening TV news broadcast).

(6) Synchronize your speaking with the builds and transitions. In other words, show the next item(new slide or new build) at the same time you begin talking about it. This requires practice, but it takes only ashort time to get the hang of it. Watch the evening news on TV and you’ll notice that bullet points andgraphics appear at the same time or just after the reporter speaks on the particular item.

(7) Use video and audio when appropriate. You can use video clips within PowerPoint without everleaving the application or turning on a VCR. Using a video clip not only will illustrate your point better, it willalso serve as a change of pace thereby increasing the interest of your audience. You can use audio clips (suchas interviews) as well. Something to avoid, however, is cheesy sound effects that are included in PowerPoint(such as the sound of a horn or applause when transitioning slides). The use of superfluous sound effectsattached to animations is a sure way to lose credibility with your audience.

(8) Limit your ideas to one main idea per slide. If you have a complicated slide with lots of different data,it may be better to break it up into 2-3 different slides (assuming no side-by-side comparisons are needed).

Delivery Tips

(1) Move away from the podium — connect with your audience. If at all possible get closer to youraudience by moving away from or in front of the podium.

(2) Remember the “B” key. If you press the “B” key while your PowerPoint slide is showing, the screenwill go blank. This is useful if you need to digress or move off the topic presented on the slide. By having theslide blank, all the attention can now be placed back on you. When you are ready to move on, just press the“B” key again and the image reappears. (The “.” key does the same thing).

(3) Use a remote-control device to advance your slides and builds. A handheld remote will allow youto move away from the podium. This is an absolute must. (http://www.keyspan.com/products/).

(4) Make good eye contact. Try looking at individuals rather than scanning the group. Since you are using acomputer, you never need to look at the screen behind you — just glance down at the computer screenbriefly. One sure way to lose an audience is to turn your back on them.

Perfect adequacy (non-attachment) teaches us to transcend ourselves so that we may respond to the inner requirements of those around us. — Robert Linssen

(5) Take it slowly. When we are nervous we tend to talk too fast. Get a videotape of one of yourpresentations to see how you did — you may be surprised at the pace of your talk.

(6) Keep the lights on. If you are speaking in a meeting room or a classroom, the temptation is to turn thelights off so that the slides look better. But go for a compromise between a bright screen image and ambientroom lighting. Turning the lights off — besides inducing sleep — puts all the focus on the screen. Theaudience should be looking at you more than the screen. Today’s projectors are bright enough to allow youto keep many of the lights on. If you are presenting to a small group, then you can connect your computer toa large TV (via the s-video line-in). With a TV screen, you can keep all or most of the lights on.

5

Suggested readings

Presentations

Beyond Bullet Points: Using Microsoft PowerPoint toCreate Presentations That Inform, Motivate, and Inspireby Cliff AtkinsonMicrosoft press

The Short Road to Great Presentationsby Peter Reimold & Cheryl ReimoldIEEE Press

Looking Good in Presentations, Third Editionby Molly W. Joss, Roger C. ParkerThe Coriolis Group

Presentations That Get Results: 14 Reasons Yours May Notby Marian K. WoodallProfessional Business Communications

Design/Visual Communication

Multimedia Learningby Richard MayerCambridge University Press

The Elements of Graphic Designby Alexander WhiteAllworth Press

7 Essentials of Graphic Designby Allison GoodmanHowDesignBooks

ZEN

The Zen of Creativity : Cultivating Your Artistic Lifeby JOHN DAIDO LOORIBallantine Books

Websites

www.garrreynolds.comwww.sociablemedia.comwww.edwardtufte.comwww.presentersuniversity.comwww.presentationcoach.com

Contact Information for Garr Reynolds

Email: info@garrreynolds.comWeb: www.garrreynolds.com

Sample slides on last page.

6

Current Commentary

A Model for Instituting a ComprehensiveProgram of Remediation for At-RiskResidentsRini Banerjee Ratan, MD, Adrienne G. Pica, MPH, and Richard L. Berkowitz, MD

Residents who are performing be-low a level appropriate for theirstage of training pose challenges forthe departments and institutionsthat train them. This problem maybe encountered all over the UnitedStates and in every area of medi-cine. However, programs designedto help these residents improve andovercome their deficits have not yetbeen described in the literature.This article offers a model for insti-tuting a comprehensive program ofremediation into a residency train-ing program. A sample letter of no-tification outlining such a program isincluded, which can be modified foruse by other programs. Possiblebarriers as well as strategies toguide the successful developmentand implementation of a remedia-tion program are discussed. Themodel provides a guide and tools toassist program directors and othersinvolved in medical education increating and tailoring a remediationprogram that suits the needs of the

at-risk resident as well as the pro-gram and institution.(Obstet Gynecol 2008;112:1155–59)

Individuals in residency trainingprograms who perform below a

level appropriate for their stage oftraining compromise the ability of

health care teams to provide excel-lent patient care and pose chal-lenges for the departments and in-stitutions that train them. Althoughmost residents proceed throughtheir training with readily solvableproblems, those with more chal-lenging difficulties may have anundesired effect at multiple levels.Some faculty members who per-ceive a resident’s deficiencies mayrespond by heightening their su-pervision or scrutiny of the individ-ual’s work, whereas others maylimit their interactions, such as thesurgeon who will not permit theresident to scrub on her cases. Sim-ilarly, fellow residents may be re-luctant to hand over the care ofacutely ill patients to a peer who isconsidered to be a “weak link” in

the chain. Junior colleagues andmedical students may expressconcern that their education isbeing compromised while underthe tutelage of a resident recog-nized to have deficiencies. Fi-nally, the resident himself mayfeel overwhelmed, ostracized, orpersecuted. This problem may beencountered all over the UnitedStates and in every area of medicine.

Such individuals have frequentlybeen described as “problem resi-dents,” and they are “at-risk” forseveral adverse outcomes. Previ-ous articles have described thecharacteristics of these residentsand indicated that the prevalenceof this problem varies from 5.8 –9.1%, but programs designed tohelp them improve and overcometheir deficits have received lessattention in the literature. Ourgoal in this article is to provideprogram directors and others in-volved in graduate medical edu-cation with a model for institutinga comprehensive program of re-mediation into a busy residencyprogram.1– 4

IDENTIFYING THE PROBLEMThe first step in the process is toidentify those characteristics of anat-risk resident. Residents maystruggle for a variety of reasons,including knowledge, skill, and be-havioral deficits, attitudinal prob-lems, lack of confidence, psychiat-

From the Department of Obstetrics and Gynecology,New York Presbyterian-Columbia University Medi-cal Center, and Center for Education Research andEvaluation, Columbia University Medical Center,New York, New York.

Corresponding author: Rini Banerjee Ratan, MD,161 Fort Washington Avenue, New York, NY10032; e-mail: rr2172@columbia.edu.

Financial DisclosureThe authors have no potential conflicts of interest todisclose.

© 2008 by The American College of Obstetriciansand Gynecologists. Published by Lippincott Williams& Wilkins.ISSN: 0029-7844/08

...it is important that

appropriate intervention

be instituted at a nascent

stage in training.

VOL. 112, NO. 5, NOVEMBER 2008 OBSTETRICS & GYNECOLOGY 1155

ric issues, medical illness, substanceabuse, or interpersonal conflicts.1

In fact, a multiplicity of issues isoften the case. Very often residentsat risk will have encountered simi-lar difficulties earlier in their lives.It may be useful to examine theirperformance reports from medicalschool and even college to look forfootprints of previous concerns.Interestingly, Papadakis et al6

found that unprofessional behaviorin medical school was a predictorof future disciplinary action by astate medical board.

If a pattern of substandard per-formance is identified by multipleevaluators, it is important that ap-propriate intervention be institutedat a nascent stage in training. Ide-ally, faculty members and peersshould complete written or elec-tronic evaluations of residents atthe end of each clinical rotation.This allows program directors to

monitor performance on a longitu-dinal basis and to determine whensomeone consistently falls belowan acceptable threshold. If con-cerned, program directors shouldconduct interviews with facultymembers, staff associates, andother residents to further define theprecise nature of the problem. Theprogram director should then holda formal meeting with the at-riskresident to discuss the situation andissue a warning, if appropriate. Aninitial course of action and firmtimeline for execution and fol-low-up should be determined. Theprogram director should write abrief note documenting the salientpoints discussed during the meet-ing to be placed in the at-risk resi-dent’s file. The Box, “Sample Noteto File” (see Appendix 1 online atwww.greenjournal.org/cgi/content/full/112/5/1155/DC1) provides anexample of such a note.

DEVELOPING ACOMPREHENSIVE PROGRAMOF REMEDIATIONIf the problems persist after a frankdiscussion between the program di-rector and resident has occurred,we encourage the program directorto discuss the case within his or herdepartmental education or promo-tions committee to generate con-sensus on the most appropriatesubsequent course of action. Thisstrategy provides a forum to furtherdefine the issues to be addressedand is a form of due process, whichconsequently helps to insulate theprogram director and institutionagainst possible accusations of dis-crimination. Once consensus on re-mediation has been established, atailored plan for the resident-at-riskcan be designed (Fig. 1). Essentialsteps that must be undertaken in-clude the following:

Fig. 1. Blueprint for Developing a Program of Remediation.Ratan. Resident Remediation Program. Obstet Gynecol 2008.

1156 Ratan et al Resident Remediation Program OBSTETRICS & GYNECOLOGY

1. Documentation in theResident’s FileAll performance issues should beclearly documented in the resi-dent’s file. This should includeminutes from the Education Com-mittee meeting as well as all evalu-ations of the resident from faculty,peers, midlevel providers, nurses,and medical students. The fileshould also include all notes writ-ten and dated by the program di-rector documenting meetings withthe at-risk resident. In cases whereconcerns about the resident’s per-formance warrant consideration ofprobation or other disciplinary ac-tion, it may be judicious to notarizeany documentation included in thefile that is not signed and dated bythe resident.

2. Notification andInvolvement of InstitutionalGraduate Medical EducationCommittee and DesignatedInstitutional OfficialIf the department decides on for-mal disciplinary action such as pro-bation, the institution’s graduatemedical education committee mustbe informed and the case discussedwith the hospital’s designated insti-tutional official before proceeding,particularly if the program is con-

sidering extension of a resident’stenure, or termination due to per-formance issues.

3. Discussion With theInstitution’s Legal CounselIt is also essential to meet with theinstitution’s legal counsel to ensurethat due process procedures arestrictly followed and documented,as required by the AccreditationCouncil for Graduate Medical Ed-ucation (ACGME).7 It may be use-ful to address possible ramificationsregarding the resident’s future re-quests for licensure or hospitalprivileges, such as whether the res-ident would be obligated to answer“Yes” on questions regarding disci-plinary actions frequently found onsuch forms.

4. Development of aRemediation CurriculumWe recommend designing a com-prehensive remediation curriculumby putting together in modularfashion components that addresseach of the deficiencies identifiedin the resident’s performance. Indi-vidual items from that curriculumcan then be combined in the futureto tailor programs for residentswith different constellations of per-formance issues. When developing

the curriculum, it may be helpful toidentify the ACGME competencythat is addressed by each compo-nent of the program to ensurealignment with the goals and objec-tives for the overall residency pro-gram. Table 1 shows an outline of asample curriculum that may be de-signed. The areas for improvementare listed as objectives, and thenactivities to strengthen the particu-lar areas of deficiency are identifiedaccordingly.

Residents at risk may have prob-lems identified in several ofthe ACGME competencies. Toaddress problems with medicalknowledge, weekly tutorial sessionswith a faculty mentor may be as-signed. Figure 2 provides an over-view of how we have designed suchtutorial sessions for obstetrics. Inadvance of each hourly session,reading and self-study material areassigned, which the resident is ex-pected to complete outside of dutyhours. For residents who have de-ficiencies identified in the ACGMEcompetency domains of interper-sonal and communication skillsand professionalism, we have en-rolled residents in workshops thatinclude strategies for handling con-flict, exercises on how to recognizeand eliminate self-defeating ac-

Table 1. Sample Remediation Curriculum

Objective Activity ACGME Competency

Improve fund of knowledge inobstetrics and gynecology

Weekly instruction in obstetrics, usingspecifically designed curriculum with obstetricsfaculty

Medical knowledge

Weekly instruction in gynecology, usingspecifically designed curriculum withgynecology faculty

Reading program/self-studyImprove surgical skills Modified operating schedule with an attending

physicianPractice-based learning and

improvement/medical knowledge

Address concerns aboutinterpersonal behavioral issues

Medical evaluation by occupational healthservices

Interpersonal and communicationskills/professionalism

Interpersonal skills courseAppointments with psychiatrist/counselor during

probationary period

ACGME, Accreditation Council for Graduate Medical Education.

VOL. 112, NO. 5, NOVEMBER 2008 Ratan et al Resident Remediation Program 1157

tions, and lessons on improving self-esteem and confidence. Other struc-tured interventions that have beenproven useful in addressing issues ofprofessionalism include behaviormodification programs, commonlyreferred to as “charm school” andanger management workshops.

NOTIFICATION ANDEVALUATION OF THERESIDENTOne of the most difficult parts ofthe process is the actual notificationof the resident of the ongoing con-cern regarding performance, theneed for an official program ofremediation, and possible out-comes if objective improvement isnot noted. Several factors shouldbe taken into consideration whendeciding the manner in which tobreak this news. The timing of themeeting is important—the residentshould be reasonably well-rested,not postcall. An initial “fitness-for-duty” evaluation, administeredby occupational health services,should be considered, and may bemandated by the institution, to ruleout any possible organic cause forpoor performance, such as sub-stance abuse or psychiatric illness.As part of the resident’s due pro-cess, written notification of the cor-rective or disciplinary action mustbe provided and should be re-viewed by legal advisors in ad-vance of the meeting. This letter

should outline a brief history of theproblem and carefully delineateeach deficiency noted in the resi-dent’s performance, along with thespecific corrective action proposed.(The Box, “Template of Letter No-tifying Resident of Program of Re-mediation,” provides a templateletter that programs may modify touse those items most appropriate fortheir resident-at-risk [see Appendix 2online at www.greenjournal.org/cgi/content/full/112/5/1155/DC2].)

After carefully orchestrating allof the above factors, a confidentialmeeting should be held with theresident at risk, in the presence of awitness and details of the plannedremediation program reviewed, asoutlined in the letter that is latersent to the resident’s home. Theresident should be given the oppor-tunity to ask questions, and thetone of the discussion shouldbe positive, emphasizing that thiscourse of action is being recom-mended in the resident’s best inter-est, to provide the additional skillsand knowledge necessary to per-form at the level expected in thetraining program.

Immediately after the meeting,the resident may be evaluatedmedically. No prior notice shouldbe given if valid drug testing is tobe conducted. We also offer resi-dents at risk a visit with a psychia-trist from the institution’s “resident-in-crisis” program.

Assistance should be soughtfrom the entire faculty in support-ing a resident in remediation, byoffering heightened attention toclinical performance, as well asby providing encouragement andemotional support during the pro-gram of remediation. Similar infor-mation should then be relayed tothe house staff to allow discussionof the effect of remediation on theirtraining and to actively enlist thehelp of fellow residents to supporttheir colleague during this difficulttime. The resident at risk shouldmeet with the program directorbriefly the next day, to provide theopportunity to ask questions thatmay have arisen after time for re-flection and to ensure appropriateemotional stability.

IMPLEMENTATION ANDEVALUATION OF THEREMEDIATION PROGRAMAs the remediation program is be-ing implemented, it should be keptin mind that any tutorial sessions,additional surgeries, or other ed-ucational experiences that are in-corporated into the remediationcurriculum, with the exception ofself-study assignments, must occurwithin the structure of the 80-hourwork week. It must also be deter-mined whether the resident cancomplete the program of remedia-tion while continuing to participatewithin the residency and whetherthe individual is able to teach jun-ior residents and medical studentssatisfactorily during this process.Junior trainees may not feel com-fortable working under the supervi-sion of a resident who is acknowl-edged to have significant deficits.The progress and emotional well-being of all residents or studentsworking closely with the resident atrisk should be monitored.

A heightened level of supervi-sion must be instituted and docu-mented for the resident undergoing

Fig. 2. Tutorial sessions in obstetrics. Q&A, question and answer.Ratan. Resident Remediation Program. Obstet Gynecol 2008.

1158 Ratan et al Resident Remediation Program OBSTETRICS & GYNECOLOGY

remediation, and a formal systemfor evaluating progress that in-cludes timely feedback is essential.5

A strict time line should be deter-mined and followed. We requestweekly “progress reports” fromthe resident’s faculty mentors andshare this information with the res-ident at frequent regular meetings.The Box, “Faculty Evaluation ofResident in Program of Remedia-tion” (see Appendix 3 online atwww.greenjournal.org/cgi/content/full/112/5/1155/DC3) includes asample of the specific evaluationform that we developed to assessthe resident’s progress at the com-pletion of the first 8 weeks of aremediation program.

CONCLUSIONRemediation forces a departmentinto self-examination of both fac-ulty and trainee needs and respon-sibilities. It requires significant in-vestment by the department andprogram administration. Facultymembers must devote additionaltime and effort beyond their re-quired clinical activities to remedi-ate the at-risk resident, and otherresidents will be asked to acceptadditional burdens to assist theircolleague. After implementing a re-mediation program, the leadershipof a department may experiencesome social repercussions. Evenresidents whose performance is ex-

cellent may begin to feel they are“under a microscope.” Some fac-ulty members may disagree with aconsensus for remediation of a sin-gle resident. However, such a pro-cess also provides an opportunityfor a department to gain a reputa-tion for mentoring and supportingresidents during times of stress anddifficulty.

Appropriate notification, feed-back, evaluation, and documenta-tion of training expectations are allcritical elements of any program ofremediation. The ACGME now re-quires that programs develop anddistribute learning goals that arelinked to the six competencies.Clearly defined objectives can beused to set standards, help a depart-ment identify a resident’s deficien-cies, and support the decision forremediation. We encourage pro-grams to develop a remediationprogram as a way of advocating fortheir residents. Our proposed pro-gram should be applicable to allresidency training programs, re-gardless of specialty. The modeloutlined here is intended to reducesome of the “start-up” time spentby the program directors when de-veloping remediation programsand help answer the question“Where do I begin?” The Councilon Resident Education in Obstet-rics and Gynecology School forProgram Directors is another ex-

cellent resource and includes sev-eral lectures on resident remedia-tion. Although it is a dauntingenterprise, designing a comprehen-sive, yet tailored program of reme-diation is the right thing to do—forthe at-risk resident, for our patients,and for our entire profession.

REFERENCES1. Reamy BV, Harman JH. Residents in

trouble: an in-depth assessment of the25-year experience of a single familymedicine residency. Fam Med 2006;38:252–7.

2. Yao DC, Wright SM. The challenge ofproblem residents. J Gen Intern Med2001;16:486–92.

3. Smith CS, Stevens NG, Servis M. Ageneral framework for approaching res-idents in difficulty. Fam Med 2007;39:331–6.

4. Beckmann CR. CITROG (Committeeon In-Training Examination for Resi-dents in Obstetrics and Gynecology)examination remediation indicator: thenext steps. Am J Obstet Gynecol 1996;174:1942–3.

5. Ende J. Feedback in clinical medicaleducation. JAMA 1983;250:777–81.

6. Papadakis MA, Hodgson CS, TeheraniA, Kohatsu ND. Unprofessional behav-ior in medical school is associated withsubsequent disciplinary action by astate medical board. Acad Med 2004;79:244–9.

7. Accreditation Council for Graduate Med-ical Education. ACGME InstitutionalRequirements. Effective: July 1, 2007.Available at: http://www.acgme.org/acWebsite/irc/irc_IRCpr07012007.pdf.Retrieved August 18, 2008.

VOL. 112, NO. 5, NOVEMBER 2008 Ratan et al Resident Remediation Program 1159

Education

The in-training examination in obstetrics and gynecology: An attempt to establish a remediation indicator

Frank W. Ling, MD,” Sarina J. Grosswald, EdD,b Douglas W. Laube, MD,’ Andrea M. Carpentieri, MA,b and Carlyle Crenshaw, MDd

Memphis, Tennessee, Washington, D.C., Madison, Wisconsin, and Baltimore, Maryland

OBJECTIVE: With the use of a university- and community hospital-based faculty, we attempted to determine at what performance level remediation would be recommended. STUDY DESIGN: The Committee on In-Training Examinations for Residents in Obstetrics and Gynecology Task Force on Standard-Based Scoring sent the 1991 examination to 16 university- and 12 community hospital-based faculty members. Given a standardized definition of a “borderline third-year resident,” each faculty scored each item on the examination on whether that hypothetic resident would or would not correctly answer the item. RESULTS: The mean expectation of correct responses on the 397-item test was 236 (59%). This was identical to the score obtained if 2 SDS were subtracted from the actual mean for all third-year residents taking the examination. University- and community hospital-based faculty members had generally similar expectations of this defined resident. CONCLUSION: Although poor examination results should not be recommended as the sole determinant for promotion, it appears that 2 SDS below the mean may be an appropriate score below which remediation could be recommended. (AM J OBSTET GYNECOL 1995;173:946-50.)

Key words: Remediation indicator, performance expectations

The first in-training examination for residents in obstetrics and gynecology administered by the Council on Resident Education in Obstetrics and Gynecology was given in 1970. Since then it has become a highly visible and widely used educational tool. Originally designed for both self-assessment and program assess- ment, the examination appears to be evolving into an evaluation tool that program directors can use in de- termining the cognitive knowledge of individual resi- dents in the program and the residents as a group.1-6 Anecdotal reports suggest that differing levels of expec- tation are held by different program directors. With no standardized reference point for what is an inappropri- ately low score for an individual resident, the Commit- tee on In-Training Examination for Residents in Ob-

From the Department of Obstetrics and Gynecology, UniversiQ of Tennessee, Memphis,” The American College of Obstetricians and Gynecologists,” the Department of Obstetrics and Gynecology, Univer- sity of Wisconsin,’ and the Department of Obstetrics and Gynecology, Uniuerszty of Maryland.”

Received for publication June 14, 1994; revised December 14, 1994; accepted December 15, 1994. Reprints not available from the authors. Copyright 0 I995 by Mosby-Year Book, Inc. 0002-9378/95 $5.00 f 0 6/l/62898

946

stetrics and Gynecology, appointed by the Council on Resident Education in Obstetrics and Gynecology, es- tablished a task force to study the possible implications of establishing a standard-based scoring system to de- velop a rational basis for remediation on the basis of a specific examination performance level. The purpose of this report is to describe these efforts.

Material and methods The Committee on In-Training Examination for

Residents in Obstetrics and Gynecology Task Force on Standard-Based Scoring was formed in April 1991 to establish a score for the examination below which a third-year resident might need remedial instruction. Standard-setting methods based on judgments about expected performance involve several basic steps: (1) selecting the judges, (2) defining “borderline” knowl- edge and skills, (3) collecting judgments, and (4) com- bining the judgments to choose a passing score.’ Three commonly used methods for making judgments for educational measurement are the Nedelsky, Angoff, and Ebel methods. The Nedelsky8 and Angoff methods compute passing scores from the identified expected scores for each test question. The Ebel method requires

Volume 173, Number 3, Part 1 Am J Obstet Gynecol

Ling et al. 947

Expected Score - Phase 1

-- Phase 2

12 3 4 5 6 7 8 9 10 11 12 13 14 15

Fig. 1. Predicted score by individual judges. Phase 1 (university residency-based faculty judges) mean 236; phase 2 (community hospital-based faculty judges) mean 225; overall mean 231.

a two-stage approach, first classifying questions into groups, and then making judgments about the groups.7, 9

For this study a modification of the Angoff method was used. In the Angoff method a judgment of the prob- ability (0.0 to 1.0) that a borderline student would get the item correct is made for each question.g Because this was an exploratory study it was decided that the judges would be asked simply whether they felt a borderline third-year resident in obstetrics and gynecology should know the correct answer; thus the probability would be either 0.0 or 1 .O. The results could be assumed to reflect the level of knowledge the judges expected from a bor- derline test-taker. These results were then compared with the actual test scores for third-year residents.

In a two-phase process, university-based faculty and community hospital-based faculty were each asked to identify performance expectations. In July 1991, 16 university faculty members were selected to participate in the first phase of this study. They were chosen irrespective of subspecialty interests but were recruited by members of the Committee on In-Training Exami- nation for Residents in Obstetrics and Gynecology for their active involvement in resident education. Partici- pants reviewed each item of the 1991 examination and indicated whether they felt that a borderline third-year resident would anwswer the question correctly.

A borderline third-year resident in need of remedia- tion was defined as “the resident who has a basic fund of knowledge accumulated primarily from cursory, su- perficial reading and informal teaching at rounds or conferences. Problem-solving skills and integration of data are incomplete. As a result, clinical decision-mak-

ing is neither prompt nor always appropriate. Techni- cal (surgical) skills can range from poor to excellent, but when asked about indications for surgery, alterna- tives of treatment or rationales for certain parts of procedures, background information is not complete. This resident is often not a good teacher of students and junior residents since he/she does not have that firm of knowledge base from which to teach.”

In a second phase of the project community hospital program directors were recruited to complete the same exercise carried out by the university faculty. Council on Resident Education in Obstetrics and Gynecology Re- gional Representatives identified community hospital program directors in their regions who were willing to participate in the second phase of this study. The purpose of this phase was to compare responses from these community hospital-based faculty members with those previously obtained from the university faculty. In February 1992 materials and instructions similar to those sent to the university faculty in phase 1 were sent to the community hospital faculty members. The results of each of the two phases of the study were tabulated and compared with actual third-year results.

Results Fig. 1 illustrates the range of expected scores to be

obtained by the hypothetic borderline third-year resi- dent on the 397-item examination. In phase 1 of the study, with 16 university faculty participating, the high- est expected score was 346 items (87% correct) and the lowest expected score was 116 (29% correct). The mean expectation of correct answers was 236 (59%). In phase 2 of the study, the responses from the community

948 Ling et al. September 1995 Am J Obstet Gym01

hospital program directors, 12 faculty members pro- vided responses with a high expectation of 336 (85% correct) and a low of 105 (26% correct). The mean expectation for this group was 225 (57% correct).

When responses to the individual questions were analyzed, at best 49% (46% phase 1, 49% phase 2) of the questions on the test had strong agreement from the raters as to whether the items could be answered by the borderline third-year resident. Agreement was de- fined as having 12 of 16 respondents in phase 1 of the study and nine of 12 in phase 2 voting in the same direction on a single item. Because of the varied topics addressed by the individual items, the fact that there is only 46% to 49% agreement is not unexpected. The 397 items form a reliable examination for the content of obstetrics and gynecology (r = 0.99).

Actual performance on the examination by third-year residents resulted in a mean examination score of 279 (71%, corrected for item deletions), with an SD of 22.87. Subtracting 2 SDS from the third-year mean score produces a score of 233, or 59%, the same mean produced by the combined scores of the faculty raters.

Comment In the overall evaluation of residents in obstetrics and

gynecology, little standardized data are available by which program directors can compare their residents with those of other programs. The Council on Resident Education in Obstetrics and Gynecology In-Training Examination remains the only instrument that is used widely by residency programs in obstetrics and gynecol- ogy. To assess an individual resident’s cognitive knowl- edge, a program director is logically drawn to compar- ing the resident’s score with national averages for resi- dents at a comparable level. As an outgrowth of this desire to evaluate by means other than local standards, program directors have specified various performance levels as “passing” for residents in their individual programs. With no such national standard, residents in different programs are being judged on potentially widely disparate levels of expectation. For example, some program directors have publicly stated that all the residents in their program are expected to score above the 50th percentile each year. This is clearly an incor- rect application of the test scores. Others have specified a certain percent of questions answered correctly as a minimal performance expectation. Such performance levels are arbitrary and potentially harmful to both the individual resident and the program as a whole. The study described is the first attempt to provide a nation- ally determined remediation indicator for an individual resident taking the in-service examination.

The two independent groups of faculty members, the first from university faculty and the second from com- munity hospitals, were very close in their assessment of

performance expectations for the hypothetic borderline third-year resident. The 59% score, the mean of the 16 responses of university faculty, was not significantly different from the 57% mean score from the 12 com- munity hospital faculty members who participated (t = 0.488, not significant). The average of 58% was very similar to the 59% score calculated by subtracting 2 SDS from the actual mean scored by all third-year residents taking the 1991 examination. A very simple interpretation of these results would suggest that 2 SDS below the mean is an appropriate remediation indicator for third-year residents taking the examination. Such a recommendation, simple as it appears, is, however, fraught with potential pitfalls. Any cutoff score identi- fied should be approached with caution.

For example, if 2 SDS below the mean was used as a remediation indicator for the 1992 examination, 67 examinees, representing 5.5% of the third-year resi- dents, would have fallen below this standard. Third- year residents scoring below 2 SDS below the mean would be considered statistical outliers on the examina- tion but not necessarily individuals who require clinical remediation (i.e., residents may be performing well clinically yet score statistically low on the examination). Therefore correlation of resident clinical performance with these numbers on the examination is necessary in determining whether the “hypothetic borderline third- year definition” used is appropriate for these current residents or any future residents who might be evalu- ated with this remediation indicator. Certainly a resi- dent whose clinical performance is commensurate with his test score would be appropriate for remediation.

There are several other issues that can be raised regarding this potential remediation indicator. For ex- ample, a more rigorous attempt to analyze each item (i.e., a process that includes a group discussion of each item followed by a second rating) might generate slightly different results, because raters interact in rat- ing rather than rating in isolation.‘(’ Further, a rigid Angoff method, a process that allows for degrees of potential accuracy on each item rather than a yes-no decision as used here, might also yield different results. However, the increased time and effort required to proceed in a more comprehensive fashion could only be

justified by the expectation that such a process would identiEj a more clearly homogeneous group of poten- tially borderline residents. Given the exploratory nature of this study, this was felt to be excessively rigorous. Identifying expectations of the various subspecialists could be used to determine whether expectations out- side one’s own subspecialty area makes a significant difference. Because expectations may vary depending on local faculty and the strength of their own teaching and residents, wider representation of faculty members and wider subspecialty representation may be needed,

Volume 173, Number 3, Part 1 Am J Obstet Gynecol

Ling et al. 949

or repeated analysis, using different faculty around the country, may be appropriate to reduce year-to-year variation.

If the process described for third-year residents is considered an appropriate one, further analyses could be conducted for other levels of residency (i.e., first, second, and fourth year). An argument could be made that establishing a score to represent a remediation indicator at each level of training would help identify as early as possible in the training process residents with cognitive deficiencies and the areas in need of reme- diation.

So long as the single instrument most widely used to evaluate a resident’s progress in accumulating cognitive knowledge is not a strictly criterion-based one, there will continue to be a desire to set some standards of expecta- tion for the examinees. Establishing a national remedia- tion indicator has many implications for both individual residents in training and faculty members involved in resident education. Program directors seek national standards to which residents can be held accountable, but by the same token residents should be held to levels of expectation that are realistic and firmly grounded in sound, scientific basis. Both faculty and residents must keep in mind that the goal of the residency process is to ensure a highly competent, empathetic physician . . . a qualitative end point. Therefore limited expectations should be placed on the quantitative in-training exami- nation. Qualitative evaluation must also be a significant element in the assessment of a resident. Assuming that a reasonable standard of performance has been estab- lished, the resident can and should be held to those local standards with appropriate probationary status and ad- ditional educational programs implemented. Measure- ment of outcomes of these supplementary educational efforts will be more readily available in the future with the newly implemented standard score.

For residents in need of remediation on the basis of the statistical indicator, a number of techniques can be applied to assist with improvement in cognitive areas. The substance of these efforts can be derived through use of the Test Item Summary Booklet. Most of the information necessary for cognitive remediation is found in a combination of four components of the booklet: (1) the answer to the question, (2) the ex- panded key word phrase, (3) the objective key (Council on Resident Education in Obstetrics and Gynecology objectives, fourth edition”), and (4) the item-specific text references (usually two). Although the test booklets are no longer returned, it should be pointed out that the expanded key word phrase coupled with the objec- tive key provide enough information so that the frame- work of the question can be deduced. In effect, every- thing is there except the actual question.

The Test Item Summary Booklet can be used with the

information provided as a basis for oral examination or discussions, a departmentally administered written ex- amination covering the reference material, and anno- tation of the reference material by the individual resi- dent. This technique can be particularly useful consid- ering the key word phrase and attendant references can be developed into a short annotated bibliography of missed questions.

For the resident in question, approximately 3 hours per week for study and preparation time, coupled with 1 to 2 hours per week for program director assessment and review, should be sufficient. With this guideline, if 10 to 20 missed items per session were assigned from the Test Item Summary Booklet, the resident would be expected to read all pertinent reference material for each of these items. By use of the key word phrase and educational objective, the resident is expected to sum- marize the readings or provide the oral discussion or annotated write-up. The program director and resident together subsequently discuss the material and select potential areas for better clinical exposure or further cognitive support. With this technique the resident could have covered the entire examination in 20 weekly sessions, assuming that he or she reviewed 20 items per session. This approximates 60 hours spanning the 20- week period. In addition, mentor time with different faculty would be approximately one third to two thirds of this time commitment over the same 20-week period.

In continued efforts to improve the use of in-training examination scores, the 1993 performance reports in- cluded the addition of a standard score. Establishing the 1992 examination as the base year for comparison, comparison of scores can now be made from year to year. Selected items from the 1992 examination serve as standard items for establishing the basis for equating. Scores are statistically adjusted to produce a mean of 500 and an SD of 100 for third-year examinees. Ap- proximately two thirds of third-year examinees will score between 400 and 600, and 95% will score between 300 and 700. Scores for future tests (beginning with 1993) are scaled to third-year residents. With 500 used as a benchmark, any examinee’s score can be assessed relative to the mean for third-year examinees.

This new scoring allows several advantages. For ex- ample, a score of 550 will mean the same level of performance each year; if a resident scores 550 one year and 600 the next, it is statistically correct to say that there was an improvement of 50 points. It is expected that this enhancement will offer objective data for following up the progress of individual and national scores, thereby reducing the use of arbitrary “pass” levels on a local basis.

The study described is an initial attempt at establish- ing a consistent level of expectation. The results suggest that further study in this area would be useful. Until

950 Ling et al. September 1995 .km J Obstet Gynecol

further data are available, program directors should assess the implications and appropriateness of any local

7 policy for remediation.

1.

2.

3.

Russell KP. In-training examinations for residents in ob- stetrics and gynecology: report of the 1970 examination. Obstet Gynecol 1970;36:953-6. Russell KP. The 1971 in-training examination in obstetrics 9 and gynecology. Obstet Gynecol 1973;41:148-54. Meskaukas JA, Newton M, Russell KP. The 1973 in- 10. training examination in obstetrics and gynecology. Obstet Gynecol 1974;44:463-8.

4.

5.

Newton M. The 1974 in-training examination in obstetrics and gynecology. Obstet Gynecol 1975;46:356-8. Easterling WE Jr. In-training examinations for residents in obstetrics and gynecology, 1975 to 1978. AM J OBSTET GYNECOL 1979;133:733-41.

6. Visscher HC. Validity and purpose of in-training exami-

REFERENCES 8

11.

nation in obstetrics and gynecology, 1979-1982. Obstet Gynecol 1984;63:253-9. Livingston SA, Zieky MJ. Passing scores: a manual for setting standards of performance on educational and oc- cupational tests. Princeton, New Jersey: Educational Test- ing Service, 1982. Plake BS, Melican GJ. Effects of item content on intra- judge consistence of expert judgment via the Nedelsky standard setting method. Educ Psycho1 Meas 1989;49:45- 51. Angoff WH. Scales, norms and equivalent scores. Prince- ton, New Jersey: Educational Testing Service, 1984. Smith RL, Smith JK. Differential use of item information by judges using Angoff and Nedelsky Procedures. J Educ Meas 1988;294:259-74. Council on Resident Education in Obstetrics and Gynecol- ogy. Educational objectives: core curriculum for residents in obstetrics and gynecology. 4th ed. Washington: Council on Resident Education in Obstetrics and Gynecology, 1992.

Correction In the article by Eskenazi et al., entitled “Fetal growth retardation in infants of

multiparous and nulliparous women with preeclampsia” (AM J OBSTET GYNEUIL 1993; 169: 1112~8), in Table III on pages 1116 and 1117, under the heading “Case patients versus control patients,” the subheading “x”’ should have been “Multiparous x’.“’

Academic Incivility:

Resources for Dealing with Harassment The University of Minnesota is committed to a working and learning environment that is respectful, collegial, and free of harassment. Harassment can include offensive, intimidating, or hostile behavior that interferes with a student’s ability to work or study, such as, but not limited to, threatening or demeaning language. If you or someone you know has experienced offensive, intimidating or hostile behavior that interferes with your ability to work or study, you don’t have to face these challenges alone. There are services here to support you. First Step Contacts for Personal, Academic or Career Concerns

Contact:

Jan Morse Student Conflict

Resolution Center (SCRC) Phone:

612-626-0689 Email:

morse005@umn.edu Web:

http://www.sos.umn.edu

Contact:

Mary Tate Minority Affairs and

Diversity and Medical School EOAA Unit Liaison

Phone: 612-625-1494

Email: tatex001@umn.edu Office:

B608 Mayo

Contact:

Marilyn Becker Learner Development

Phone: 612-626-7196

Email: becke024@umn.edu

Office: B624Mayo

Contact:

(RAP) Resident Assistance Program

Phone: 651-430-3383 (local);

1-800-632-7643 (toll-free) Web:

www.sandcreekeap.com

Primary sources of assistance include:

Your Program Director or Faculty Advisor. They are your essential partners in a successful educational experience. If you encounter a problem and feel comfortable approaching them, do it and do it early.

Student Conflict Resolution Center (SCRC). If you want to talk to someone outside of your department, you can contact the SCRC. Consultations are confidential - no one will know you contacted SCRC without your permission. SCRC works with hundreds of students and offers information, coaching, and intervention. You can reach them by phone 612-624-SCRC, by email sos@umn.edu or in person (211 Eddy Hall).

Mary Tate. The Director of the Medical School Office of Minority Affairs and Diversity is the Equal Opportunity and Affirmative Action liaison. For questions or concerns regarding matters of allegations of mistreatment, sexual harassment, or discrimination, the Office of Minority Affairs and Diversity may assist in finding solutions. You can reach her by phone 612-625-1494, by email tatex001@umn.edu, or in person B608 Mayo.

Marilyn Becker. The Medical School Director of Learner Development. Dr. Becker assists residents and fellows with learning/performance concerns across the GME competencies and residency/fellowship requirements; provides assessments and referrals for special services [disability evaluation, ESL tutoring, personal/couple counseling, health/wellness assistance]; and is available for consultation on academic/training process difficulties. You can reach her by phone 612-626-7196, by email becke024@umn.edu, or in person B624Mayo.

Resident Assistance Program (RAP). The Resident Assistance Program (RAP) is a confidential counseling service designed to offer residents and their immediate family members a professional, external resource to address a variety of stressors, at no cost to the client. In many cases, these stressors are affecting personal lives and impacting a resident’s ability to meet professional expectations in the workplace. You can reach them by phone 651-430-3383 (local) OR 1-800-632-7643 (toll free); or the web www.sandcreekeap.com.

For more information on campus resources, visit http://www.sos.umn.edu/stafffaculty/academic_civility.php Delaying or avoiding a situation can make it worse. Don’t put off addressing a problem until you’re falling behind in your coursework or considering leaving your program or job. You don’t have to face it alone. See also Resident Dispute Resolution Policy at: http://www.med.umn.edu/gme/residents/parta/disciplresdisputeresolpol.html As always, if you believe there is imminent danger to a student or others, please call 911.

2010-2011

Graduate Medical Education Committee (GMEC)

~~ RREESSIIDDEENNTT LLEEAADDEERRSSHHIIPP CCOOUUNNCCIILL ~~

Meeting Schedule

Unless otherwise indicated, GMEC Resident Leadership Council (RLC) meetings are

held the 4th Tuesday of every month from 2:30-3:30pm in Room B620 Mayo

Memorial Building.

As voting members of the GMEC, RLC members are strongly encouraged to attend

the Graduate Medical Education Committee meeting immediately following the

Resident Council meeting (3:30-4:30 PM in Room B620 Mayo Memorial Building).

July 27, 2010

August 24, 2010

September 28, 2010

October 26, 2010

November 23, 2010

December Meeting CANCELED

January 25, 2011

February 22, 2011

March 22, 2011

April 26, 2011

May 24, 2011

June Meeting CANCELED

GMEC Coordinator:

Carla Nelson

612-625-7634

nelso081@umn.edu

DEADLINE FOR SUBMISSION OF GMEC RLC AGENDA ITEMS

To allow committee members adequate time to review the GMEC RLC meeting agenda and supporting documentation, the meeting agenda and action items will be E-mailed to committee members by 4:30 PM on the third Wednesday of each month. To this end, agenda topics and their supporting documentation must be submitted by 4:30 PM on the third Tuesday of each month. Submit items to Carla Nelson, GMEC Coordinator, via E-mail at nelso081@umn.edu or via fax at 612-624-0150. Items may also be delivered directly to Carla’s office, Room B-654 Mayo Memorial Building. Topics and supporting documentation received after the deadline will be held until the next GMEC RLC meeting. You must plan accordingly. Please take the time to familiarize yourself with the agenda and supporting documentation in advance so that we may have a more productive meeting.

2010-2011 Submission Deadlines

Meeting Date Submission Deadline

Tuesday, July 27, 2010 Tuesday, July 20, 2010 Tuesday, August 24, 2010 Tuesday, August 17, 2010 Tuesday, September 28, 2010 Tuesday, September 21, 2010 Tuesday, October 26, 2010 Tuesday, October 19, 2010 Tuesday, November 23, 2010 Tuesday, November 16, 2010

* December Meeting Canceled * --- Tuesday, January 25, 2011 Tuesday, January 18, 2011 Tuesday, February 22, 2011 Tuesday, February 15, 2011 Tuesday, March 22, 2011 Tuesday, March 15, 2011 Tuesday, April 26, 2011 Tuesday, April 19, 2011 Tuesday, May 24, 2011 Tuesday, May 17, 2011

* June Meeting Canceled * ---

2010-2011

GRADUATE MEDICAL EDUCATION COMMITTEE (GMEC)

Meeting Schedule

GMEC meetings are held the fourth Tuesday of every month from 3:30-4:30pm

in Room B620 Mayo Memorial Building unless otherwise indicated. Given the

nature of work conducted by the group it is important that voting members

attend. If you are unable to do so, please designate someone to attend in your

absence.

July 27, 2010

August 24, 2010

September 28, 2010

October 26, 2010

November 23, 2010

December Meeting CANCELED

January 25, 2011

February 22, 2011

March 22, 2011

April 26, 2011

May 24, 2011

June Meeting CANCELED

GMEC Coordinator:

Carla Nelson

612-625-7634

nelso081@umn.edu

DEADLINE FOR SUBMISSION OF GMEC AGENDA ITEMS

To allow committee members adequate time to review the GMEC meeting agenda and supporting documentation, the meeting agenda, consent agenda and action items will be E-mailed to committee members by 4:30 PM on the third Wednesday of each month. To this end, agenda topics and their supporting documentation must be submitted by 4:30 PM on the third Tuesday of each month. Submit items to Carla Nelson, GMEC Coordinator, via E-mail at nelso081@umn.edu or via fax at 612-624-0150. Items may also be delivered directly to Carla’s office, Room B-654 Mayo Memorial Building. Topics and supporting documentation received after the deadline will be held until the next GMEC meeting. You must plan accordingly. Please take the time to familiarize yourself with the agenda and supporting documentation in advance so that we may have a more productive meeting.

2010-2011 Submission Deadlines

Meeting Date Submission Deadline

Tuesday, July 27, 2010 Tuesday, July 20, 2010 Tuesday, August 24, 2010 Tuesday, August 17, 2010 Tuesday, September 28, 2010 Tuesday, September 21, 2010 Tuesday, October 26, 2010 Tuesday, October 19, 2010 Tuesday, November 23, 2010 Tuesday, November 16, 2010

* December Meeting Canceled * --- Tuesday, January 25, 2011 Tuesday, January 18, 2011 Tuesday, February 22, 2011 Tuesday, February 15, 2011 Tuesday, March 22, 2011 Tuesday, March 15, 2011 Tuesday, April 26, 2011 Tuesday, April 19, 2011 Tuesday, May 24, 2011 Tuesday, May 17, 2011

* June Meeting Canceled * ---

TEACHING RESOURCES/MATERIALS FOR CHIEF RESIDENTS

1. Academic Internal Medicine (CDIM/APDIM) Residents as Teachers: http://www.im.org/RESOURCES/EDUCATION/RESIDENTS/LEARNING/INTERPERSONA

LCOMMUNICATIONSKILLS/Pages/Residents-as-Teachers.aspx 2. Practical Professor (University of Alberta and Alberta Rural Physician Action Plan): http://www.practicalprof.ab.ca/ 3. University of California, Irvine Residents as Teachers: http://www.residentteachers.com/Content/Curriculum.asp 4. American Academy of Pediatrics, Residents as Teachers: http://www.aap.org/sections/ypn/r/resident/pdfs/resasteachers.pdf 5. American College of Surgeons: Successfully Navigating the First Year of Residency: http://www.facs.org/education/essentials.doc 6. American College of Emergency Physicians: http://www.acep.org/practres.aspx?id=40272&ekmensel=c580fa7b_90_378_40272_1 7. Resident Well Being: http://www.med.umn.edu/gme/residents/wellness/home.html 8. Resident Educator Development (RED): http://www.med.umn.edu/gme/residents/reseducdevel/home.html