Integrating teaching and assessment of clinical reasoning

Dr. Amy Warren BSc., BVSc. (hons), PhD. DACVP

Associate Professor Veterinary Pathology, University of Calgary, Canada

Our current understanding of clinical reasoning…..

• System 1 and system 2 dual reasoning theories likely accounts for much of the cognitive processes of clinical reasoning – Experts with familiar cases tend to use system 1 reasoning – Novices tend to use system 2 reasoning – Experts are triggered to use system 2 reasoning in unfamiliar

cases, cases with dissonant data or as a “checks and balance” to their reasoning

• Experts possess an extensive and multidimensional

knowledge base = SCRIPTS – Initial exposure of clinicians to a case typically stimulates initial

System 1 retrieval of multiple illness scripts

Great… so how do we best convert a novice to an expert?

DVM student Competent clinician

Adult learning

Adults: • Organize knowledge in schemes • Have a great deal of prior knowledge • Make sense of knowledge by active

manipulation • Optimally learn when experiential and

contextually bound • Are intrinsically rather than extrinsically

motivated to learn • Excessive cognitive load hampers learning

Adult learning

Adults: • Organize knowledge in schemes • Have a great deal of prior knowledge • Make sense of knowledge by active

manipulation • Optimally learn when experiential and

contextually bound • Are intrinsically rather than extrinsically

motivated to learn • Excessive cognitive load hampers learning

Encapsulation theory Co

ncep

tual

Adapted from Schmidt (1990) Acad Med 65(10):611-621

Stage 1: Development of elaborated causal networks Relatedness of concepts Cause and consequence of pathophysiologic processes

Stage 2: Compilation of elaborated networks into abridged ones

Extensive and repeated application High level simplified causal models using diagnosis to explain signs and symptoms

Stage 3: Emergence of Illness scripts Stage 4: Storing patient encounters as Instance scripts Ex

perie

ntia

l

DVM program years 1 and 2

DVM program years 3 and 4

Internship and residency programs

Knowledge schemes: setting the scene for clinical reasoning

• Basic sciences provides understanding between pathophysiology and clinical signs

• Teaching of knowledge “schemes” enhances memory organization and diagnostic success

• Multiple studies support that students that

understand the pathophysiology of a clinical sign have better diagnostic reasoning and accuracy – Analytical reasoning (system 2) RELIES on

understanding of basic principles

How this causes this.

Developing “knowledge structures”

• Build on existing knowledge schemes – Instructors add and restructure mis-

conceptions – Case discussions and two way feedback

• Experts (teachers) model knowledge

structures – Instructor self-awareness of knowledge that

may be encapsulated

• Concept maps, algorithms and schemes help novices to organize information

• Biomedical and clinical sciences must emphasize links to clinical signs/ presentation

Circulating immune complexes deposit in basement membrane

Complement, humoral and cell-mediated immune cell

damage to glomerulus

Increased leakiness of glomerulus approximately

same MW as albumin

PROTEINURIA

Type III hypersensitivity

Glomerulonephritis

Antibody-antigen

structure and production

Glomerular microanatomy and renal physiology

Adult learning

Adults: • Organize knowledge in schemes • Have a great deal of prior knowledge • Make sense of knowledge by active

manipulation • Optimally learn when experiential and

contextually bound • Are intrinsically rather than extrinsically

motivated to learn • Excessive cognitive load hampers learning

Experiential learning model

Adapted from Merriam (1999)

Case-presentation and interpretation (e.g., proteinuria)

Immediate feedback from instructor and correction of

misconceptions

Instructor creating cognitive dissonance to challenge current understanding

Concepts taught in a classroom setting (e.g., pathophysiology of GN)

Circulating immune complexes deposit in basement membrane

Complement, humoral and cell-mediated immune cell damage

to glomerulus

Increased leakiness of glomerulus approximately same

MW as albumin

PROTEINURIA

Type III hypersensitivity

Experiential learning is additive

• Case revisited with increasing complexity – Each cycle students gain

understanding based on prior knowledge structures

• Analytical reasoning supports early reasoning – Mnemonics and heuristic aids – Zones of proximal development

• Instructor role – To relate old knowledge to new – Create cognitive dissonance – Correct misconceptions

Context matters

• Learning is fundamentally situated – Biomedical and clinical knowledge

needs to be embedded in clinical context

• Teaching in context enhances

transference into practice – Context helps to intrinsically

motivate student learning

• We access knowledge in a similar way to it is learned. – If we learn in organ systems rather

than presenting signs- we access the information in a systemic fashion.

Adult learning

Adults: • Organize knowledge in schemes • Have a great deal of prior knowledge • Make sense of knowledge by active

manipulation • Optimally learn when experiential and

contextually bound • Are intrinsically rather than extrinsically

motivated to learn • Excessive cognitive load hampers learning

Cognitive load theory

• Intrinsic load (the complexity of the task) • Extraneous load (the way learning tasks are presented) • Germane load (load associated with actual learning)

Maximizing germane learning in clinical reasoning

• Reducing intrinsic load – Introduce multiple steps to problem solving – Sequence cases so that increasing complexity – Gradually increase the fidelity of the case

• Reducing extraneous load – Move the goal post closer (partial diagnosis, identify

principles) – Put multiple information sources in one location – Provide a worked through example – Replace written explanations with visual aids

• Maximizing germane load – Encourage comparison between multiple cases and identify

unifying principles – Vary types of cases (mixed practice) – Self explanation of concepts to maximize prior knowledge

Encapsulation theory Co

ncep

tual

Adapted from Schmidt (1990) Acad Med 65(10):611-621

Stage 1: Development of elaborated causal networks Relatedness of concepts Cause and consequence of pathophysiologic processes

Stage 2: Compilation of elaborated networks into abridged ones

Extensive and repeated application High level simplified causal models using diagnosis to explain signs and symptoms

Stage 3: Emergence of Illness scripts Stage 4: Storing patient encounters as Instance scripts Ex

perie

ntia

l DVM program years 3 and 4

Internship and residency programs

Fostering illness scripts and clinical reasoning in clinics

Clinical reasoning learning is enhanced by: • Repeated and directed exposure of students to

cases – Repeated exposure fosters System 1 reasoning

• Instructor modeling of reasoning (rounds) – Encourage both System 1 and System 2 reasoning – Encouraging students to use System 2 reasoning as a check

and balance to their System 1 intuition

• Repetitively asking “what”, “why” and “how” questions forces students to use system 2 thinking

• Integration of prior biomedical knowledge – Fosters analytical reasoning skills

• Immediate instructor feedback • Teaching in context enhances transference into

practice

Relating it all back • Repeated case exposure optimizes illness script

acquisition and system 1 reasoning

• Case exposure needs to be active

• Deliberate application and integration of biomedical knowledge

• Teaching of knowledge in schemes enhances knowledge retrieval

• Analytical and non-analytical reasoning occurs simultaneously and additively and should be encouraged

• Teaching multiple reasoning methods and encouraging redundancy minimizes diagnostic error

Assessment of clinical reasoning: How to test what’s in the “black box”?

• How do we measure what we can’t define?

• Reasoning skills are not directly measureable, they are inferred from behaviors

• Context specificity of reasoning • Clinical reasoning is non-linear

Balancing the reliability, validity and feasibility of assessment methods.

Oral/ Viva voce/ long cases • Work up case and then presents to

assessor • Assessor asks clarifying questions • Can be structured in an OSCE style

format to increase reliability

PROS: • Use of “real” cases has high context

validity • Allows examination of the entire

case approach

CONS: • Resource intensive • Can be low reliability and subjective • Competence in one context is not

predictive of overall competence

Example rubric or OSCE checklist 1. Summary statement (1 pt) 2. Problem list (3 pts) 3. Most likely Dx (1 pt) 4. Supporting data (2 pts) 5. Next steps in the diagnosis/

treatment plan

Global rating score

MCQ • Simple knowledge (recall)

based questions do little to assess decision making

• Clinical scenarios increases the higher order questions

The most common cause of UP:UC ratio < 10.0 in a dog is: A. Pyuria B. Hemoglobinuria C. Glomerulonephritis D. Renal tubular nephrosis

A 4-year old MC Dalmatian dog presents with polyuria and polydipsia. He is moderately azotemic, has a USG of 1.014, hypoalbuminemia and hypercholesterolemia. His UP: UC is 15. The most likely diagnosis is: A. Pyuria B. Hemoglobinuria C. Glomerulonephritis D. Renal tubular nephrosis

PROS: • Validity, reliability and feasibility are

high • Familiar testing modality for test

takers and writers

CONS: • Difficult to write non-knowledge

based questions • Assumes linearity • Context specificity is low

Knowledge

Analytical

Script concordance test • Clinical scenarios • Test takers rate how useful

the information provided is compared to expert opinion

A 9 year old FS Labrador Retriever presents with a 4 hour history of panting, acting anxious and retching non-productively. The patient is no longer able to stand. The temperature is 38.7, heart rate of 60 bpm with weak peripheral pulses. The capillary refill time is 3 seconds and the mucus membranes appear “muddy”. If you are considering a diagnosis of… With the information provided, this diagnosis becomes… GDV -2 -1 0 +1 +2 Food Bloat -2 -1 0 +1 +2 Oral FB -2 -1 0 +1 +2

PROS: • Good context specificity • Allows for the ambiguity of clinical

reasoning

CONS: • Very difficult to establish reliability

and validity • Expert group difficult to define

Key features test • Clinical scenario based questions • Focus on the difficult steps in

diagnosis and management where most likely to make errors

PROS: • Good context specificity • Allows for the non-linearity of cases • Focus on the difficult aspect of

reasoning in a case allows more breadth

CONS: • Validity and reliability are dependent

on proper blueprinting of the exam and large case number (>40)

A 4 year old German Shepherd dog presents to emergency after a normal evening of eating dinner and playing in the back yard with sudden onset of panting and restlessness. The heart rate is 180 bpm and the mucus membranes are pale with a refill time of 3s. The abdomen is distended but not painful. Your top differential at this time (name one) What is your next course of action (choose as many as you like): • Treat with metacam and monitor at

home • Perform a rectal examination • Take an abdominal radiograph • Take a thoracic radiograph • Perform an abdominocentesis • Pass a nasogastric tube • Treat with fluids

Which is best? • Multiple different testing methods at

multiple time points • Context is important the higher fidelity

the reasoning becomes • Concentrate less on a single best

clinical reasoning approach but more on the “boundaries” of a range of acceptable performances

• Non-linear process- as such qualitative and mixed methods illuminate the non-linearity

Acknowledgments Collaborators: • Dr. Kent Hecker • Dr. Tyronne Donnon • Dr. Catherine Wagg • Dr. Nicole Fernandez • Dr. Heather Priest • Research technician- Jason Abboud • DVM class of 2014 and 2015

Funding • Zoetis • UCVM veterinary educational

research fund • UCVM clinical research fund