february 11, 2015 family & community medicine chaney...

Medical Education Planning & Development Basics February 11, 2015

Family & Community Medicine Chaney Conference Room Workshop Agenda

Time Didactic Exercise Ref/Handouts 12:15-12:30

Introductions; Why do this; overall program; expectations, projects, etc

Discussion/Q & A Introduce project to work on through in workshop

Project Plan worksheet

12:30-12:40

Learner-centered ed; learner needs; Dreyfus level of learners—Novice—Master; Pathman Model

Discussion regarding residents as learners Table, pg 23

Kern Ch 2 Pathman Model

12:40-1:10

Developing Learning Objectives for learners and course/activity: learner, process and pt outcomes objectives

Write learning objective; share with peers; group feedback Table pg 78

Kern Ch 4 Learning Objectives Primer

1:10-1:40

Adult learning theory; competency based education; educational strategies; matching ed methods to objectives Promote Effective Presentation Skills March 11

Determine strategies for your activity; share w/ peers; feedback from group Follow up session: Effective Presentations Mar 18

Kern Ch 5 Ten Aspects of Learning for Effective Teaching (Viggiano)

1:40-2:00

BREAK

2:00-2:45

Learner assessment & Feedback: formative and summative; feedback strategies; summative assessment strategies

Practice with feedback strategies; Develop learner assessment strategies for activity Follow up sessions: EPAs & Milestones and Assessment & Feedback sessions June 5; Dealing with Difficult Learner Sept 9

Kern Ch 7 Preceptor pocket card

2:45-3:15

Implementation and Program evaluation; resources needed; timeline; overall program evaluation; stakeholders; methods for program evaluation; logic model

Develop implementation plan; identify needed resources; potential barriers & challenges; design overall program evaluation plan

Kern Ch 6/8 Logic model Data collection plan

3:15-3:35

Scholarly work: Making it Count Twice Recap from Viggiano presentation Jan 23.

Review examples of publications/posters/MedEdPortal publications Follow up session: Ed Scholarship Mar 11; Abstract Writing May 13

Kern Ch 9 MedEdPortal

3:35-4:00

Debrief: Key learning points; next steps Schedule on-site coaching

Each participant feedback key learnings; areas of uncertainty Flesh out activity plan template and review with table-mates

2/11/2015

1

Faculty Development

Medical Education Planning & Development

February 11,2015

Nancy Davis, PhDRuth Weber, MD, MSED

1

Workshop Outline

• Introductions/Overview of Fac Dev program

• Learner centered education

• Learning objectives

• Educational strategies for adult learners

• Learner assessment & feedback

• Implementation & program evaluation

• Scholarly work: make it count twice

• Debrief/next steps

2

2/11/2015

2

Your Needs‐‐Introductions

• What are your specific gaps?

• Your project

• Program timelines

• Interim activities

3

LEARNER CENTERED EDUCATION

4

2/11/2015

3

Learner Needs Assessment

• Perceived needs

– Individuals preferences

– Practice attributes

• Actual needs

– Practice/systems data

– Observation/feedback

– Regulation/mandates

5

Dreyfus Model: Novice Expert

Dreyfus and Dreyfus, 1982

6

2/11/2015

4

Developmental Stages of Learners

LevelTraining

LevelExample

Novice Beginning student

• Introductory lectures, web exercises

• Group work on case studies

Advanced Beginner

Advanced student

• Students apply concepts in a “project” at the academic health center

•Teacher is model and “coach”

Competent Post graduate training

• Apply concepts to his or her own panel of patients in interprofessional team

Proficient Early practice• Regularly review and improve care for patients

Expert Advanced practice

• Develop novel ways to understand and improve systems of care

Medical School Years 3&4

Residency

Medical School Years 1&2

Fellowship and Practice

7

Methods for Stage of LearnerPathman/PRECEDE Examples

Stage/Method Awareness Agreement Adoption Adherence

Predisposing Lecture

Grand rounds

Enabling Champions

Clinical precepting

Problem‐based small group

Clinicalprecepting

Simulation

Reinforcing Role‐play

Feedback in practice

Reminders

Audit/Feedback

Davis DA, et al. The case for knowledge translation: shortening the journey from evidence to effect. BMJ 2003;327:33

8

2/11/2015

5

DEFINING COMPETENCE

Competency is defined as the observable behavior that combines knowledge, skills, values, and attitudes related to a specific activity

9

Medical Knowledge

Patient Care

Interpersonal Communication

Professionalism

Systems‐based Practice

Practice‐based Learning and Improvement

ACGME Core Competencies

10

2/11/2015

6

Family Medicine Competencies

• Disease prevention and health promotion

• Critical thinking skills in patient care

• Patient safety

• Advocating for individual and community health

• Coordination of team based care

11

EDUCATIONAL DEVELOPMENT

12

2/11/2015

7

Educational Development Cycle

1. Problem Identification

& Needs Assessment

1. Problem Identification

& Needs Assessment

2. Goals & Objectives2. Goals & Objectives

3. Educational Strategies

3. Educational Strategies

4. Implementation4. Implementation

5. Evaluation & Feedback5. Evaluation & Feedback

13

LEARNING OBJECTIVES

14

2/11/2015

8

Your Overarching Goals and Objectives

Program (activity) goal/aim

What do you hope to accomplish?

Program Objectives

15

Goals/Objectives: SMART

16

2/11/2015

9

Interface of Learning Objectives and Teaching Strategiescase studies

projectsexercises

projects critiquesproblems simulations

case studies appraisalscreative exercises

problems develop plans EVALUATION

Teaching Strategies Related to Learning

Objectives

exercises constructs appraisecase studies simulations assess

exercises critical incidents choosepractice discussion SYNTHESIS estimate

demonstrations questions arrange evaluatequestions projects test collect judgediscussion sketches compose measure

review simulations ANALYSIS construct ratetest role play analyze create revise

assessment microteach calculate design scorereports compare formulate select

lecture learner APPLICATION contrast organize valuevisuals presentations apply criticize manage

video writing demonstrate debate planaudio dramatize diagram prepare

examples COMPREHENSION employ differentiate proposeillustrations describe illustrate distinguish set upanalogies discuss interpret experiment

Cognitive Processing Dimension

explain operate inspect→ KNOWLEDGE express practice inventory

define Identify perform question list recognize schedule relaterecord restate shop test

Learning Objectives

repeat translate sketchuse

Adapted from: Bloom, B.S. (Ed.), Engelhart, M.D., Furst, E.J., Hill, W.H., & Krathwohl, D.R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook 1: Cognitive domain. New York: David McKay.

Ellen F. Goldman, EdD 11-1-2010

17

Learning Objectives ExamplesCognitive Level Action Verbs Behavioral Learning Objective Example Teaching Strategies Examples Assessment Strategies

Knowledge

Learners will

“know” something

new

Define

Describe

List

Record

Repeat

Learners will be able to list the five risk factors

of MI

Lecture

Video

Audio

Written exams

Oral exams

Pre/Post

Comprehension

Learner will be

able to “convey”

their new

knowledge.

Discuss

Explain

Express

Identify

Recognize

Restate

Translate

Learners will be able to explain the Krebs Cycle Question

Discussion

Learner presentations

Writing

Written exams

Oral exams

Pre/Post

Learner presentations

Writing critique

Analysis

Learners will be

able to “analyze

and interpret” new

information

Analyze

Calculate

Compare

Contrast

Criticize

Diagram

Differentiate

Distinguish

Experiment

Question

Learners will be able to analyze a fishbone

diagram for cause and effect of a problem

Problems

Exercises

Case Studies

Critical incident analysis

Discussion

Assessment of:

Problems

Exercises

Case Studies

Critical incident analysis

Root Cause Analysis

Evaluation

Learners will be

able to “evaluate”

situations or

concepts based on

what they’ve

learned

Appraise

Assess

Choose

Estimate

Evaluate

Measure

Revise

Score

Select

Value

Learners will be able to evaluate best treatment

plan based on efficacy and cost

Case Studies

Projects

Exercises

Critiques

Simulations

Appraisals

Assessment of:

Case Studies

Projects

Exercises

Critiques

Simulations

Appraisals

2/11/2015

10

1. Create a stem…

– After completing the lesson, the learner will be able to . . .

– After this unit, the learner will have . . .

– By completing the activities, the learner will . . .

– At the conclusion of the course/unit/study the learner will . .

2. Add an action verb

– Use verbs from Bloom’s taxonomy list

– Determine the actual product, process, or outcome.

Writing Educational Objectives

http://www.educationoasis.com/curriculum/LP/LP_resources/lesson_objectives.htm

19

• Goal/Aim: 25% of all Family Medicine residents will complete a longitudinal QI project over the next academic year.

AVOID CHOOSE• Vague verbs

“Know how to …” Explain, describe, discuss

• General terms Be as specific as possible

Example of Program/Activity Objective

20

2/11/2015

11

Examples of Learner ObjectiveNot so good:

The resident will understand quality improvement methodology and the relevance to their future careers in medicine.

BETTER:

After completing the year‐long curriculum, each resident will be able to:

• Define steps of a PDSA cycle (knowledge)

• Explain the importance of quality improvement in medicine (comprehension)

• Perform a RCA (application)

• Diagram a process map (analysis)

• Design a QI project (synthesize)

• Evaluate performance data (evaluation)

21

Another Good Example

After completion of this course the learner will be able to:

define Root Cause Analysis (RCA);

recognize which clinical situations require RCA;

explain why RCA is important;

demonstrate mutual respect on interprofessional teams;

and have performed an interprofessional mock RCA.

What is the highest level of cognitive processing this faculty expects her learners to achieve?

22

2/11/2015

12

Your Project

Develop SMART Objectives for:

Your activity

Your learners

23

EDUCATIONAL STRATEGIES

24

2/11/2015

13


projectsexercises





Objectives












Learning Objectives




25

Educational Strategies

• Content ‐‐‐Flows from learning objectives

– Schedule of events

– Written material

– Resources for additional learning

• Educational Methods

– Cognitive, affective, psychomotor

– Use multiple educational methods

– Feasible with available resources

26

2/11/2015

14

Cognitive Objectives

• Readings

• Lecture

• Discussion

• Technology based materials

• Problem based learning

27

Affective Objectives

• Targeted Reading

• Exposure

• Facilitation of openness, introspection and reflection

• Role models

28

2/11/2015

15

Psychomotor Objectives

• Supervised clinical experience

• Simulations

• Role‐plays

29

Learner Centeredness

• Formal/informal assessment of learner needs

• Tailoring of content/methods to meet learners’ needs

– Modify /add to curriculum

– What objectives are critical, desirable but not critical, feasible

30

2/11/2015

16

Special considerationsPractice‐Based Learning and Improvement

• Locates, appraises and assimilates evidence from scientific studies related to patients health problems

• Demonstrates self‐directed learning

• Improves systems in which the physician provides care

Methods:

Independent learning projects

Individual learning plans

Self assessment/chart audits

31

Special considerationsSystems‐Based Practice and Teamwork

• Provides cost‐conscious medical care

• Emphasizes patient safety

• Advocates for individual and community health

• Coordinates team based care

Methods:

• inclusive health care teams

• feedback on cost of care

• work in disease management programs

• participate in QI and safety teams

32

2/11/2015

17

Special considerationsProfessionalism

• Demonstrates professional conduct and accountability

• Demonstrates humanism and cultural proficiency

Methods:

• Role modeling

• Facilitated reflection/discussion of experiences embodying professionalism

• Peer evaluations

• Honor Council

33

Your project…..

Look at your objectives…..

What educational methods will be most effective and feasible?

What resources are available?

What time is involved?

Will your curriculum involve PBLI, SBP and/or professionalism?

34

2/11/2015

18

LEARNER ASSESSMENT & FEEDBACK

35

Feedback

• Individual Learner– Formative

• Improve performance

– Summative• Grades/promotion

• Curriculum/Program– Formative

• Improve effectiveness

– Summative• Resources, External requirements, Promotion, Publication

36

2/11/2015

19

One Minute PreceptorMicroskill Strategies

1 Assess prior knowledge Assess learner knowledge/experienceEstablish goals for encounter

2 Get a commitment “What” questions: what is going on? What would you like to do?

3 Probe for evidence Encourage learner to think out loudWhy do you think that’s so?

4 Give positive feedback Reinforce what was done rightUse specific, behavioral examples

5 Identify and correct mistakes Specific and focused: “That drug is often a good choice, but not with renal failure.”

6 Teach the general rule Chose a rule at learner’s levelConsider competencies other than “knowledge” eg, communication, professionalism

7 Encourage reflection and promote self directed learning

“What do you feel you did well?”“What do you need to study?”

Irby DM. The One‐Minute Preceptor. First presented at the annual Society of Teachers of Family Medicine Predoctoral meeting. Orlando FL Feb 1997

37

Standardized Students

• Break into dyads

• One person is student, one is faculty member

• Five minutes to roll play

• Group discussion about how it went….

38

2/11/2015

20

Your project

• What will be evaluated?

– Think of your objectives…..

– Measurable

• What forms/implements will you use?

• What resources will you need?

• Who will do the evaluation?

39

IMPLEMENTATION PROGRAM EVALUATION

40

2/11/2015

21

Logic model and common types of evaluation

Needs/asset assessment: What are the characteristics, needs, priorities of target population?

What are potential barriers/facilitators?

What is most appropriate to do?

Process evaluation:How is program implemented?

Are activities delivered as intended? Fidelity of implementation?

Are participants being reached as intended?

What are participant reactions?

Outcome evaluation: To what extent are desired changes occurring? Goals met?

Who is benefiting/not benefiting? How?

What seems to work? Not work?

What are unintended outcomes?

Impact evaluation: To what extent can changes be attributed to the program?

What are the net effects?

What are final consequences?

Is program worth resources it costs?

41

Start with the end in mind

Learner Outcomes

Educational Strategies• Content• Formats

Resources needed

Input

Output

Outcome

42

2/11/2015

22

Logic Model

43

44

2/11/2015

23

Evaluation Data Collection Plan

Questions Indicators Data collection

Sources Methods Sample Timing

Needs Target learner needs met?

Process Target reachedProgram run well?Satisfied?

Outcomes

Impact

Goals met?Desired changes?

Changes attributable to program?ROI?

45

Logic Model and Reporting

46

2/11/2015

24

SCHOLARLY WORK: MAKE IT COUNT TWICE

47

Dissemination Considerations

• Who else is interested in what you’re doing?

• What are your career goals?

• What do you enjoy? Play to your passions

• Where do you excel?

• Who are your professional peers?

• How do they like to learn about new innovations?

48

2/11/2015

25

Dissemination Opportunities Presentations

• Clinical– UKSM‐W DFMC

– KAFP: posters and lectures

• Practice Improvement

• Educational– STFM: workshop, seminar, poster

– RPS

– NCFMR

49

Dissemination Opportunities

Publications– Med Ed journals and sites

• STFM

• Academic Medicine

• Medical Teacher

• Teaching & Learning in Medicine

– General medical journals and sites• ABFM

– Specialty journals

– Quality journals

50

2/11/2015

26

Dissemination Time/Effort• Get help!

• Collaborate

• Create realistic timeline

• Peer review

51

52

2/11/2015

27

Project work

Coaching

Next session:Effective Presentation SkillsMarch 18

53


projectsexercises





Objectives












Learning Objectives




54

Educational Project Plan Sample

Name:___________________________________________________________________________ Activity Goal/Aim: To teach PGY1 residents appropriate techniques counseling patients regarding tobacco cessation.

Educational Design Learning Objective Educational format/strategy Learner Assessment strategy Resources needed

1. Take accurate tobacco history

Lecture/pre-reading Pre/post test Content experts

2. Identify patients who are receptive to counseling

Case-based session Pre/post test

3. Develop process for systematic approach to counselling.

Group exercise to create evidence-based process for our clinic

Observation; critique plan Examples of approaches

4. Refer patients to resources, eg Quit Line;

Assignment to research available resources, eg on-line, clinic, local agencies

Critique plan Internet search; local agency resources

5. Appropriately prescribe medications for tobacco cessation

Lecture/discussion/cases Observation; case evaluations

Program Evaluation Plan: Brief description. Design and attach logic model.

Implementation Timeline Milestone Date

Due Develop learning objectives 3/1 Create cases 4/1 Consult with Dr X, content expert 4/15 Meet with Nancy/Ruth 4/30 Consult with behavioral health professional 5/1 Create tobacco cessation resource tool 6/1 Develop pre/post tests 6/15 Meet with Nancy/Ruth 6/30 Evaluation plan (logic model) completed 7/1

Educational Project Plan

Name:___________________________________________________________________________ Activity Goal/Aim: Educational Design Learning Objective Educational format/strategy Learner Assessment strategy Resources needed 1.

2.

3.

4.

5.

Program Evaluation Plan: Brief description. Design and attach logic model.

Implementation Timeline Milestone Date

Educational Project Plan

Name_________________________________________________________Date___________________

1. Goal/Aim: What is the gap or problem you want to address through your activity?

2. Objectives: What are the learning objectives? Use the term “Performance expectations” to guide you. What do you want learners to do after the activity? Think SMART: specific, measureable, actionable, relevant and time bound.

3. Educational design: What learning methods/formats will you use? Devleop 2-3 bullet points to outline your educational idea, project or innovation and what content you want to include.

4. Learner Assessment: How will you assess your learners? What methods will you use to assess what your learners have accomplished?

5. Program evaluation: How will you evaluate your activity? How will you know its impact and how it might be improved?

6. Implementation: How will you implement? Who will be your partners? What resources will you need? From whom will you need buy-in?

Timeline: Project your planning, implementation and evaluation timeline.

Questions Indicators Data collection

Sources Methods Sample Timing

Needs Target learner needs met?

Process Target reached

Program run well?

Satisfied?

Outcomes

Impact

Goals met?

Desired changes?

Changes attributable to program?

ROI?

Educational Evaluation Data Collection Plan

Medical Education Planning & Development Basics February 11 Workshop Outline

Time Didactic Exercise Who Ref/Handouts 12:15-12:30

Introductions; Why do this; overall program; expectations, projects, etc

Discussion/Q & A Introduce project to work on through in workshop

ND

12:30-12:40

Learner-centered ed; learner needs; Dreyfus level of learners—Novice—Master; Pathman Model

Discussion regarding residents as learners Table, pg 23—Consider this when developing obj

ND Kern Ch 2

12:40-1:10

Developing Learning Objectives for learners and course/activity: learner, process and pt outcomes objectives

Write learning objective; share with peers; group feedback Table pg 78

ND/RW Kern Ch 4

1:10-1:40

Adult learning theory; competency based education; educational strategies; matching ed methods to objectives Promote Effective Presentation Skills March 11

Determine strategies for your activity; share w/ peers; feedback from group

RW Kern Ch 5 Ten Aspects of Learning for Effective Teaching (Viggiano)

1:40-2:00

BREAK

2:00-2:45

Learner assessment & Feedback: formative and summative; feedback strategies; summative assessment strategies Promote EPAs & Milestones and Assessment & Feedback sessions June 5 & Dealing with Difficult Learner Sept 9

Practice with feedback strategies; Develop learner assessment strategies for activity

RW Kern Ch 7

2:45-3:15

Implementation and Program evaluation; resources needed; timeline; overall program evaluation; stakeholders; methods for program evaluation; logic model

Develop implementation plan; identify needed resources; potential barriers & challenges; design overall program evaluation plan

ND Kern Ch 6/8

3:15-3:35

Scholarly work: Making it Count Twice Recap from Viggiano presentation Jan 23. Promote Abstract Writing session on May 13

Review examples of publications/posters/MedEdPortal publications

RW/ND Kern Ch 9 MedEdPortal

3:35-4:00

Debrief: Key learning points; next steps Each participant feedback key learnings; areas of uncertainty Flesh out activity plan template and review with table-mates

ND/RW

*Refer to Kern Example Curriculum: Primary Care GYN for Internal Medicine Residents *Promote relevant upcoming Fac Dev sessions

One Minute PreceptorMicroskill Strategies

1 Assess prior knowledge Assess learner knowledge/experienceEstablish goals for encounter

2 Get a commitment “What” questions: what is going on? What would you like to do?

3 Probe for evidence Encourage learner to think out loudWhy do you think that’s so?

4 Give positive feedback Reinforce what was done rightUse specific, behavioral examples

5 Identify and correct mistakes Specific and focused: “That drug is often a good choice, but not with renal failure.”

6 Teach the general rule Chose a rule at learner’s levelConsider competencies other than “knowledge” eg, communication, professionalism

7 Encourage reflection and promote self directed learning

“What do you feel you did well?”“What do you need to study?”

Irby DM. The One‐Minute Preceptor. First presented at the annual Society of Teachers of Family Medicine Predoctoral meeting. Orlando FL Feb 1997

Stage/Method Awareness Agreement Adoption Adherence

Predisposing Lecture

Grand rounds

Enabling Champions

Clinical precepting

Problem-based small group

Clinical precepting

Simulation

Reinforcing Role-play

Feedback in practice

Reminders

Audit/Feedback

Pathman/PRECEDE Model: Effective Methods for Different Learner Stages

Davis DA, et al. The case for knowledge translation: shortening the journey from evidence to effect. BMJ 2003;327:33

My Te4Q Educational Project Plan Name:____________________ Date: _________________________ Email: ____________________ Please complete Sections 1-3 (via participant application) prior to workshop 1. GOAL/AIM: What is the gap or problem you want to address through your educational program? What are the Quality Improvement/Patient Safety competencies you want your learners to exemplify at the completion of your educational program? 2. OBJECTIVES: What are the learning objectives? Use the term ‘performance expectation’ to guide you here. What do you want learners to do after the activity? Think SMART: specific, measurable, actionable, relevant and time bound

2. EDUCATIONAL DESIGN: What learning methods/formats will you use? Develop 2-3 bullet points to outline your educational idea, project or innovation and what QI/PS content you want to include. 3. LEARNER ASSESSMENT: How will you assess your learners? What methods will you use to assess what your learners have accomplished? 4. PROGRAM EVALUATION: How will you evaluate your initiative? How will you know its impact and how it might be improved? 5. IMPLEMENTATION: How will you implement? Who will be your partners? What resources will you need? From whom will you need ‘buy-in”?

How many learners will participate?

Timeframe: Estimated Implementation Date: Confidence Level: How confident are you that you will be able to complete your

Association of American Medical Colleges

project? VERY Moderately Unlikely What barriers to you anticipate?

Additional Activity(ies): What educational QI activities aside from this project will you implement?

© 2014 AAMC. May not be reproduced without permission.

1

Wesley Family Medicine Residency Name: Academic Year: Date: Goals: can be broad, sweeping, philosophical, lofty ideas. For example, “feel more confident doing skin excisions” Objectives: must be narrow, specific, concrete. Try to specify how much of what by when. Eg “Schedule extra 2 slots for Friday Colonoscopy Sessions by XXXX, 20XX” You may have multiple objectives to achieve one goal. Forces For: what are the things that will work in your favor to help you achieve your goals: a supportive partner, time coming up an a rotation Forces Against: what factors in your life may conspire against you to make it difficult to attain your goals” lifelong pattern, family responsibilities...

GOALS OBJECTIVES Forces FOR Forces AGAINST

Clnical: Skills/Procedures

Medical Knowledge

GOALS OBJECTIVES Forces FOR Forces AGAINST

Leadership/Scholarly Work

Personal: Self-Care, Relationships, Self-Expression

Professional Development: Professionalism, Life-long learning, managing Information, Licensing, Certifications, Conference

Other Domain:_______________________

2

february 11, 2015 family & community medicine chaney...

Documents