S H A R O N K R A C K O V , E D . D . A N T O I N E T T E P E T E R S , P H . D .

H E N R Y P O H L , M . D . A A M C A N N U A L M E E T I N G , S A N F R A N C I S C O : 2 0 1 2

Advanced Thinking about the Role of Feedback in Competency-Based

Education

Workshop Agenda

Part 1: Workshop introduction • Workshop goals and objectives • Definition of feedback • Interactive group discussion: When feedback succeeds or

fails Part 2: Role of feedback in learning

• How does the strategic use of outcome-based objectives, deliberate practice and feedback foster learning?

• How does the level of feedback affect the attainment of competency?

Part 3: Hands-on practice Part 4: Wrap up and take-home messages

Workshop Goals and Objectives

At the conclusion of this workshop, participants will be able to:

• Recognize the advantages of linking outcome objectives

with opportunities for deliberate practice and learner outcomes

• Practice using formative assessment & feedback to help learners build knowledge & skills based on learning objectives

• Practice giving feedback about task, process & self-regulation, differentiating each from feedback about “self”

• Practice giving feedback that targets the appropriate cognitive stage in the context of incremental learning

• Identify appropriate timing for feedback given the cognitive stage of learning & level of feedback

What is feedback?

Information about one’s performance received from an external source

Interactive group discussion

When have you given or received feedback

that: • Led to change? • Failed?

What models or feedback formats have you

used?

When feedback fails…

• Feedback may fail because: • It was not given effectively (not timely or specific;

related to the self vs. the task or process; no plan for next steps; no help with reflection)

• The student rejected it (did not trust the content or the feedback giver; was not committed to the goal)

• After feedback, students may: • Work harder and improve • Maintain the status quo • Give up, change goals, or regress

To answer this question, we will be looking at:

Relationship between learning objectives & achieving outcomes

Bloom’s taxonomy, Miller’s triangle, Dreyfus and Dreyfus scale

Role of deliberate practice in the attainment of competency & expert performance

Role of formative assessment, reflection & feedback in the incremental attainment of knowledge, skills and attitudes

How does the strategic use of outcome-based objectives, deliberate practice and feedback

foster learning?

PRACTICE FORMATIVE ASSESSMENT SUMMATIVE ASSESSMENT (PROFICIENT/EXPERT)

REFLECTION + MENTORING

Achieving Outcome-based Objectives Through Deliberate Practice

Remembering: Can the student recall or remember the information? define, duplicate, list, memorize, recall, repeat, reproduce, state

Understanding: Can the student explain ideas or concepts? classify, describe, discuss, explain, identify, locate, recognize, report, select, translate, paraphrase

Applying: Can the student use the information in a new way? choose, demonstrate, dramatize, employ, illustrate, interpret, operate, schedule, sketch, solve, use, write.

Analyzing: Can the student distinguish between the different parts? appraise, compare, contrast, criticize, differentiate, discriminate, distinguish, examine, experiment, question, test.

Evaluating: Can the student justify a stand or decision? appraise, argue, defend, judge, select, support, value, evaluate

Creating: Can the student create new product or point of view? assemble, construct, create, design, develop, formulate, write

BLOOM’S

TAXONOMY

Does (Action) Observation, Portfolio, Logs, Peer Assessment

Shows How (Performance) Clinical and Practical Assessment Knows How (Competence) Written Assessment Knows (Knowledge) Written Assessment

Miller GE Shumway JM Harden RM

MILLER’S

TRIANGLE

Five levels of performance

1. Novice: Uses rules to determine actions

2. Advanced beginner: Develops strategies to deal with situational cues

3. Competent: Develops new rules & reasoning procedures to decide on a plan of action

4. Proficient: Recognizes patterns & reacts appropriately

5. Expert: Sees intuitively what needs to be achieved & how to do it

Dreyfus Model Expands on Miller

Ericksson’s Principles

• Continued deliberate practice (experiential learning) is necessary to attain expertise

• Conditions leading to improvement:

– Well defined tasks or objectives – Detailed and immediate feedback on performance – Opportunities to improve by performing the same or

similar tasks repeatedly

REMEMBERING

UNDERSTANDING

APPLYING

ANALYZING

EVALUATING

CR

EA

TIN

G

KNOWS

KNOWS

HOW

SHOWS

HOW

NOVICE

ADVANCED

BEGINNER

COMPETENT

PROFICIENT

ERICKSSON +MILLER+BLOOM+

DREYFUS

REPETITION FOR IMROVEMENT

DELIBERATE PRACTICE

MILLER BLOOM DREYFUS

TASK + PROCESS ++ SELF SELF-REGULATION +++

How does the level of feedback affect the attainment of

competency?

WHEN? WHAT? HOW OFTEN?

How does feedback affect the attainment of competency?

GOOD JOB!

DEFINITELY

IMPROVED!

Large and small group work

1. View video 2. Identify need for improvement 3. Write feedback statements 4. Role play 5. Critique feedback 6. Write group feedback 7. Debrief with large group

Wrap up and take home messages

• Wrap up: Hypothesize how the scenarios might impact:

• Students’ achievement of outcomes (task) • Students’ growth as learners (process) • Students’ abilities to self regulate

• Take-home messages:

References

1. Anderson, L. W., and Krathwohl, D. R. (Eds.): A taxonomy for learning, teaching and assessing: A revision of Bloom's Taxonomy of educational outcomes: Complete edition, New York: Longman, 2001 2. Batalden P, Leach D, Swing S, Dreyfus H, Dreyfus S: General competencies and accreditation in graduate medical education. Health Affairs 21:103-111, 2002 http://socrates.berkeley.edu/~hdreyfus/html/paper_socrates.html 3. Bloom B: Taxonomy of Educational Objectives, New York: Longmans Geen, 1956 4. Brosvic GM, Epstein ML, Dihoff RE, Cook MJ. Acquisition and retention of Esperanto: the case for error correction and immediate feedback. Psych Record 2006;56:205-18. 5. Dweck CS. Mindset: The New Psychology of Success. NY: Ballantine Books, 2006. 6. Ende J: Feedback in Clinical Medical Education. JAMA 250:777-781, 1983 7. Ericsson KA Krampe RTh, Clemens T-R. The role of deliberate practice in the acquisition of expert performance. Psychol. Review. 1993; 100 (3): 363-406 8. Hattie J and and Timperley H: The power of feedback, Review of Educational Research. 2007 77: 81-112 9. Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine 1990; 65: Supplement, pp S63-S67

Workshop organizers

Sharon Krackov, Ed.D. skrackov@mac.com

Antoinette Peters, Ph.D. toni_peters@hms.harvard.edu Henry Pohl, M.D. pohlh@mail.amc.edu