learner assessment win may. what is assessment? process of gathering and discussing information from...

Learner Assessment

Win May

What is Assessment?Process of gathering and discussing information from multiple sources

to gaindeep understanding of what learners

know, understand and can do

Purpose of AssessmentSubsequent learning

improve

Assessment Results

Learner-Centered Assessment

• What is it that my learners need to know and be able to do during their professional lives?

• What skills and knowledge will stand the test of time, given the rapidly evolving nature of knowledge?

Characteristics of Learner-Centered Assessment

• Promotes high expectations• Respects diverse talents and learning styles• Promotes coherence in learning• Integrates education & experience• Actively involves learners• Provides prompt feedback• Fosters collaboration

Huba & Freed, 2000

Competence in Medical Education

Habitual and judicious use of – Communication– Knowledge– Technical skills– Clinical reasoning– Emotions– Values– Reflection

in daily practice for the benefit of individuals & communities being served.

Epstein & Hundert, 2002

Assessment of Competence

• Insight into actual performance

• Capacity to – adapt to change– find and generate new knowledge– improve overall performance

Fraser & Greenhalgh, 2001

Pyramid of Competence

Goals of Assessment

• Optimize capacities through self-directed learning

• Identify incompetent physicians

• Provide basis for selecting applicants for advanced training

Types of Assessment• Formative

– Guide future learning– Promote reflection– Motivate– Shape values

• Summative– Certify– Promote

Which is the best method?

• Valid• Reliable• Appropriate to the objective• Feasible• Acceptable to faculty and learners• Cost-effective• Impacts future learning and practice

Types of Learner Assessment• Written exercises

– Case-based module– Multiple choice questions– Short-answer questions– Structured essays– Key feature and script-concordance questions– Paper/Thesis/Project

• Assessments by supervisors– Chart-stimulated recall– Product review– Standardized oral examination– Global ratings– Structured direct observation

Types of Learner Assessment

• Clinical simulations– Standardized patients

• Multi-station examinations• Unannounced standardized patients

– High-tech simulators

• Multi-source assessments– Portfolio– Peer assessment– Patient assessment– Self-assessment

Some methods to assess cognitive skills

Assessment Method

Domain Strengths Limitations

MCQ Knowledge, Problem-solving

Many content areas; High reliability; Computer grading

Difficult to write; Cueing;

SCT Knowledge, Problem-solving

Clinical problem-solving;

Computer grading

Transfer not yet proven

Table of Specifications

No. Content

Area

Recall of facts

Application/

Interpretation

Synthesis/

Problem-solving

Total

1. Oro-pharyngeal cancer

1 2 2 5

2. Myocardial infarction

1 1 3 5

Multi-method Assessment

• Each method has limitations

• Essential to use multiple methods

• Allows inclusion of wider content areas

• Multiple points of view

Outcomes - Based Education

3 circles of outcomes

Inner circle• What the doctor is able to do

Middle circle• How the doctors approach their practice

Outer circle• Doctor as a professional

What the doctor is able to do

• Apply skills of history and PE

• Communication skills

• Health promotion & disease prevention

• Procedural skills

• Investigative skills

• Management skills

How doctors approach their practice

• Apply basic sciences to clinical practice

• Use critical thinking, problem solving, clinical reasoning

• Incorporate appropriate attitudes, ethics and understanding legal issues

• Information retrieval and handling skills

Doctor as professional

• Role of doctor within health care system

• Aptitude for professional development and transferable skills

Summary• Assess competence in an integrated,

coherent, and longitudinal fashion• Know limitations of each method• Use multiple methods• Implement longitudinally• Provide frequent and constructive

feedback