the clinical communication and learning skills unit hea-01 workshop supporting students who are...

The Clinical Communication The Clinical Communication and Learning Skills Unitand Learning Skills Unit

HEA-01 WorkshopHEA-01 Workshop Supporting students who are Supporting students who are

struggling academically:struggling academically:one medical school’s approachone medical school’s approach

Jo Brown & Dason EvansJo Brown & Dason Evans

1212

The Clinical Communication and The Clinical Communication and Learning Skills UnitLearning Skills Unit1212

Plan for the dayPlan for the day10:00-11:0010:00-11:00

IntroductionsIntroductions

Defining the problemsDefining the problems11:00-11:1511:00-11:15 CoffeeCoffee

11:15-12:3011:15-12:30Why bother?Why bother?

What is done elsewhere?What is done elsewhere?12:30-1:3012:30-1:30 LunchLunch

1:30-2:301:30-2:30 The Academic support programme at The Academic support programme at Barts and the London.Barts and the London.

2:30-3:452:30-3:45 How useful is this model to other How useful is this model to other institutionsinstitutions

3.45-4.003.45-4.00 CloseClose


Defining the problemDefining the problem


ExamplesExamplesStudent 1: an excellent student academically, Student 1: an excellent student academically, fails her first Clinical Skills OSCE. fails her first Clinical Skills OSCE.

She has a deep learning style for theoretical She has a deep learning style for theoretical knowledge but an inappropriate learning style for knowledge but an inappropriate learning style for clinical skills.clinical skills.

We provided her with sessions on “how to learn We provided her with sessions on “how to learn clinical skills”.clinical skills”.


ExamplesExamplesStudent 2: fails his third year theory exams with a Student 2: fails his third year theory exams with a very poor score. He had excellent A-levels, good very poor score. He had excellent A-levels, good year one exams and reasonable year two exams.year one exams and reasonable year two exams.

He turns out to have an excellent short term memory He turns out to have an excellent short term memory and a superficial learning style. The volume of work is and a superficial learning style. The volume of work is finally too much to cram and he fails.finally too much to cram and he fails.

We provided him with sessions encouraging a regular We provided him with sessions encouraging a regular habit of self directed learning.habit of self directed learning.


ExamplesExamplesStudent 3: a previously excellent student fails Student 3: a previously excellent student fails her third year clinical skills OSCE. her third year clinical skills OSCE.

She has no insight into the fact that she is clinically She has no insight into the fact that she is clinically depressed. Her attendance has been poor, but she is depressed. Her attendance has been poor, but she is academically strong as she starts bookwork when she academically strong as she starts bookwork when she wakes at 3am every morning.wakes at 3am every morning.

We arranged an urgent GP and psychiatry We arranged an urgent GP and psychiatry appointment.appointment.


ExamplesExamplesStudent 4: an international student fails her Student 4: an international student fails her clinical skills exam, saying she didn’t know clinical skills exam, saying she didn’t know what to expectwhat to expect

More to come!


ExamplesExamplesStudent 5: Fails his clinical skills exam scoring Student 5: Fails his clinical skills exam scoring poorly on communication skills stationspoorly on communication skills stations

He finds talking to patients difficult, as he isn’t sure He finds talking to patients difficult, as he isn’t sure exactly what information he needs to gather and how exactly what information he needs to gather and how he might go about this in a time-limited fashion.he might go about this in a time-limited fashion.

We went with him to the wards on two occasions and We went with him to the wards on two occasions and using video tape review helped him gain structure and using video tape review helped him gain structure and confidence in talking to patients.confidence in talking to patients.


Your examplesYour examples


Common problems at Common problems at Barts and the LondonBarts and the London


Why Bother?Why Bother?


Why do we bother?Why do we bother?Ethical and moral duty to Ethical and moral duty to students.students.Socio-political culture of Socio-political culture of accepting and responding accepting and responding to individual differences.to individual differences.Obligation and Obligation and responsibility to future responsibility to future patients.patients.– ((philosophy of ASP – Not philosophy of ASP – Not

a revision course)a revision course)– GMCGMC

Producing “Life Long Producing “Life Long Learners.” Learners.”

Belief that students who Belief that students who have been supported have been supported through difficulty may end through difficulty may end up better doctors.up better doctors.– Coping mechanismsCoping mechanisms– empathyempathy

School philosophy to:School philosophy to:– meet the needs of students meet the needs of students – produce graduates… who produce graduates… who

are fit for the nation's are fit for the nation's present and future health, present and future health, research and health care research and health care needs .needs .

Financial Financial – benefits (widening access)benefits (widening access)– penalties (attrition rates)penalties (attrition rates)


Group Work 1Group Work 1

Group 1Group 1 How are struggling students identified How are struggling students identified in your institution?in your institution?

Group 2Group 2 How does your institution work out the How does your institution work out the reasons that students struggle?reasons that students struggle?

Group 3Group 3 What interventions does your institution What interventions does your institution put in place to support struggling put in place to support struggling students?students?

Group 4Group 4


Group work 1Group work 1


LunchLunch


1212

Support for Students with Support for Students with Academic DifficultiesAcademic Difficulties


Overview of PresentationOverview of Presentation

Macro

Micro

Macro

•Background to the school•Background to the A.S.P.•Model

•Specific Case History

•Validation of programme

•Looking to the future


Background to Barts and the LondonBackground to Barts and the London

An ancient and amalgamated medical schoolA spiral curriculumCurriculum revised in 1999:– Problem Based Learning running throughout

the curriculum– Early clinical experience in the community– Explicit teaching of Communication and

Clinical Skills


AssessmentAssessment

Clinical skills OSCE

Data interpretation

Written Assessment

Continuous assessment

Year 1

Year 2

Year 3

Year 4

Year 5


Background and History To ASPBackground and History To ASP Began in 1999 when committed teachers began to give detailed feedback to students about their exam results and offer clinical skills workshops.

Previously informal and variable support from senior tutors, concerned and interested faculty

Formalised and funded in 2002, the ASP has grown to provide student centred support for learning.

Now represented on all major educational committees and produces an annual report.


Formalised & embedded in curriculumFormalised & embedded in curriculum

ASP

CCLSU

Assess

Teaching

Education Directorate

Dean for Education

Medical School

Dean for Student Affairs and senior tutor system


The systemThe system

Indirect

Direct

Educational Assessment interviewFree discussionSemi-structured interviewFine-tooth analysis of assessment results

Poor Performance in Summative Assessment

Principally JOSCE & 3rd year EMQ

Senior Tutor Referral

Student request

Poor performance in continuous assessment

Other

No actionE.g. precipitating problem resolved or obtaining academic support elsewhere

Ongoing Individual TuitionAimed at supporting self-directed Learning either in clinical skills or theoretical background to medicine. Always student led.

Problem List Action Plan Learning

Contract

Short Courses: Learning style

Aimed at encouraging a change in learning style,

1 how to learn clinical skills2 Reflective communication

skills3 exam (eg MCQ) technique4 anxiety management5 others

Specific SuggestionsTo Students, Senior tutors, tutors or firms

Referral ElsewhereMedical referralEducational psychologistLearning support unit

Referral Assessment Intervention


Educational Assessment InterviewEducational Assessment Interview

Free discussion about Free discussion about reasons for poor reasons for poor performanceperformance

How he/she learnsHow he/she learns– KnowledgeKnowledge

– SkillsSkills

– Self directed learningSelf directed learning

– Collaborative learningCollaborative learning

Fine tooth-comb analysis Fine tooth-comb analysis of assessment resultsof assessment results

Screening questionsScreening questions– MotivationMotivation

Scale of 1-10Scale of 1-10What sort of doctor?What sort of doctor?Enjoy the course?Enjoy the course?

– AttributionAttributionHave you received enough Have you received enough teaching?teaching?Was the exam a fair exam?Was the exam a fair exam?Is mark representative of Is mark representative of performance?performance?

– Pastoral Pastoral Medics in the family?Medics in the family?Received enough support?Received enough support?

– Working with patientsWorking with patients– Problems attendingProblems attending


The systemThe system

Indirect

Direct

Educational Assessment interviewFree discussionSemi-structured interviewFine-tooth analysis of assessment results

Poor Performance in Summative Assessment

Principally JOSCE & 3rd year EMQ

Senior Tutor Referral

Student request

Poor performance in continuous assessment

Other

No actionE.g. precipitating problem resolved or obtaining academic support elsewhere

Ongoing Individual TuitionAimed at supporting self-directed Learning either in clinical skills or theoretical background to medicine. Always student led.

Problem List Action Plan Learning

Contract

Short Courses: Learning style

Aimed at encouraging a change in learning style,

1 how to learn clinical skills2 Reflective communication

skills3 exam (eg MCQ) technique4 anxiety management5 others

Specific SuggestionsTo Students, Senior tutors, tutors or firms

Referral ElsewhereMedical referralEducational psychologistLearning support unit

Referral Assessment Intervention


Number of students

0

10

20

30

40

50

60

70

1999 April 2000-March 2001

April 2001-March 2002





Questions on processQuestions on process


Case studyCase study

ReferralReferral


Case Study: Referral ICase Study: Referral I

An international student fails her clinical An international student fails her clinical skills exam, saying she didn’t know skills exam, saying she didn’t know what to expect.what to expect.

Came in the lowest 5% of cohortCame in the lowest 5% of cohortFailed 12 out of 19 stationsFailed 12 out of 19 stations

Previously received A and B grades for Previously received A and B grades for her written examsher written exams


ScoreScore

AVESCORE

30

20

10

0

Std. Dev = 5.40

Mean = 60.7

N = 233.00


Number of stations passedNumber of stations passed

STATPASS

20.519.518.517.516.515.514.513.512.511.510.59.58.57.5

50

40

30

20

10

0

Std. Dev = 2.57

Mean = 14.7

N = 233.00



Educational Assessment Educational Assessment InterviewInterview


Case Study: Diagnostic ICase Study: Diagnostic I

Fine tooth analysisFine tooth analysis

Clinical examination skills

0 1 2 3 4 5 6 7 8 9 10

Respiratory examination /10

Cardiovascular examination /10

Abdominal examination (RIF) /10

Breast examination /10

Sensory examination of legs /10

Thyrotoxicosis /10

cohort mean

case score


Practical Skills

0 1 2 3 4 5 6 7 8 9 10

Blood pressure /10

Urine testing /10

Ophthalmoscopy /10

Phlebotomy /10

cohort mean

case score


Communication Skills

0 1 2 3 4 5 6 7 8 9 10

History of chest pain /10

History of chronic cough /10

History of PR bleed (Crohn’s) /10

Risk factors for CHD /10

Low er abdominal pain /10

Type 1 diabetes: non-adherence /10

Loss of consciousness /10

Diagnosis of type 2 diabetes /10

Alcohol intake /10

cohort mean

case score


Semi-structured interviewSemi-structured interview

General questionsGeneral questions– Lives in international hallsLives in international halls– Rarely mixes with students of other Rarely mixes with students of other

nationalitiesnationalities– Rarely uses conversational EnglishRarely uses conversational English– Home sick, isolated and unsure if she really Home sick, isolated and unsure if she really

wants to study medicine in the UKwants to study medicine in the UK



Learning Methods – declarative knowledgeLearning Methods – declarative knowledge– Works 3-4 hours a week SDLWorks 3-4 hours a week SDL– Relies on memory of reading from text booksRelies on memory of reading from text books– Reads with TV or radio onReads with TV or radio on

Learning Methods - skillsLearning Methods - skills– Has used clinical skills lab (alone)Has used clinical skills lab (alone)– Uses one skills book and handoutsUses one skills book and handouts– Clerks 2 patients a week “if I am pushed”Clerks 2 patients a week “if I am pushed”– Found third year unstructuredFound third year unstructured



Specific questionsSpecific questions– Is motivated to be a DrIs motivated to be a Dr– No medics in the family, much pressure to No medics in the family, much pressure to

succeed from her governmentsucceed from her government– Doesn’t feel there has been enough teachingDoesn’t feel there has been enough teaching– ““sometimes” finds difficulty approaching sometimes” finds difficulty approaching

patientspatients



Problem ListProblem List


Problem ListProblem ListNot fully committed to course, not workingNot fully committed to course, not workingSocial isolationSocial isolation– Language Language – Collaborative learningCollaborative learning– LonelinessLoneliness

Study skillsStudy skills– Limited for theory to simply readingLimited for theory to simply reading– Poor model for learning clinical skills Poor model for learning clinical skills

Never seeks feedbackNever seeks feedbackLimited amounts of practiceLimited amounts of practice


Common problems at Common problems at Barts and the LondonBarts and the London



Negotiated learning contractNegotiated learning contract


Learning ContractLearning Contract1.1. Consider if she wants to be a doctorConsider if she wants to be a doctor

2.2. Consider if she wants to continue training in the UKConsider if she wants to continue training in the UK

If she wishes to remainIf she wishes to remain1.1. Clerk a minimum of 5 patients a weekClerk a minimum of 5 patients a week

2.2. Study a minimum of 2 hours a nightStudy a minimum of 2 hours a night

3.3. Commit to work collaboratively with retake groupCommit to work collaboratively with retake group

4.4. Move into college halls Move into college halls

5.5. Integrate with other students from the medical school Integrate with other students from the medical school



InterventionIntervention


Case Study: InterventionsCase Study: Interventions

Clinical SkillsClinical Skills– Extra clinical skills tuitionExtra clinical skills tuition– Clinical skills study skillsClinical skills study skills– Supportive ward placementSupportive ward placement

Communication SkillsCommunication Skills

Collaborative learningCollaborative learning

PastoralPastoral



OutcomesOutcomes


Case Study: Outcomes ICase Study: Outcomes I

Exam Success/CompetencyExam Success/Competency

Ongoing successOngoing success

PastoralPastoral

Smiling!Smiling!


ScoreScore

AVESCORE

30

20

10

0

Std. Dev = 5.40

Mean = 60.7

N = 233.00


Number of stations passedNumber of stations passed

STATPASS

20.519.518.517.516.515.514.513.512.511.510.59.58.57.5

50

40

30

20

10

0

Std. Dev = 2.57

Mean = 14.7

N = 233.00


Case Study: Outcomes IIICase Study: Outcomes III

50.0 60.0 70.0 80.0

Ai_EMQ%

0

5

10

15

Co

un

t

40.0 50.0 60.0 70.0 80.0

Aii_Data%

0

5

10

15

Co

un

t

55 60 65 70 75 80 85

Aiii%

0

5

10

15

20

Co

un

t

50.0 55.0 60.0 65.0 70.0

b(OSCE)%

0

5

10

15

20

Co

un

t

64%

62%

64%

60%Knowledge test Data

Interpretation test

Skills testContinuous assessment


Finals 2005Finals 2005

OSCE

78.0

76.0

74.0

72.0

70.0

68.0

66.0

64.0

62.0

60.0

58.0

56.0

54.0

52.0

50.0

48.0

46.0

44.0

42.0

35

30

25

20

15

10

5

0

Std. Dev = 6.53

Mean = 65.5

N = 224.00

65%


Questions on Case StudyQuestions on Case Study


ValidationValidation

Student evaluationStudent evaluation

Independent evaluationIndependent evaluation

Exam resultsExam results

Links to curriculum & institutionLinks to curriculum & institution

Publications & presentationsPublications & presentations


Validation: student evaluationValidation: student evaluation


“I felt that it (assessment interview) didn’t put me down, didn’t make me feel thick”

“They (ASP staff) went through all the exams we had done and really allowed us to explore how we do them”

“(In the assessment interview) They are kind and listening and paying attention…I could take the first positive step after that…the interview itself is part of the intervention”

“They (ASP staff) are very friendly, very approachable and very easy to talk to”

Qualitative commentsQualitative comments

“I did go to my senior tutor but no-one was

asking the right questions… not getting

to the root of the problem”


Validation: Formal independent Validation: Formal independent evaluationevaluation

Educational psychologist evaluationEducational psychologist evaluation– Video reviewVideo review– Document reviewDocument review– Focus group and individual student Focus group and individual student

interviewsinterviews– Faculty interviewsFaculty interviews– Educational literature reviewEducational literature review

““The outcomes and processes of the The outcomes and processes of the educational assessment and subsequent educational assessment and subsequent support are highly effective”support are highly effective”


Exam resultsExam resultsAre exam results an appropriate surrogate Are exam results an appropriate surrogate marker for outcomes of an Academic marker for outcomes of an Academic Support Programme?Support Programme?

Students failing OSCEs do well, and Students failing OSCEs do well, and continue to do well after passing through continue to do well after passing through ASP (from bottom 5% to “average”). ASP (from bottom 5% to “average”).


Validation: exam resultsValidation: exam results

AVESCORE

30

20

10

0

Std. Dev = 5.40

Mean = 60.7

N = 233.00

49.1 62.9


Validation: exam resultsValidation: exam results

STATPASS

20.519.518.517.516.515.514.513.512.511.510.59.58.57.5

50

40

30

20

10

0

Std. Dev = 2.57

Mean = 14.7

N = 233.00

10.4 16.2


Continuing successContinuing successOf the 16 students who failed their third year OSCE Of the 16 students who failed their third year OSCE in 2002, all passed through a modified academic in 2002, all passed through a modified academic support programme, support programme, 9 took finals in 2004. Their mean OSCE score at 9 took finals in 2004. Their mean OSCE score at finals was no different from the rest of the cohortfinals was no different from the rest of the cohort

(p=0.52, Scheffe post hoc analysis),(p=0.52, Scheffe post hoc analysis), Compare with the lowest 17 students passing the Compare with the lowest 17 students passing the third year OSCE, third year OSCE, 9 took finals in 2004 and performed significantly 9 took finals in 2004 and performed significantly worse than the rest of their cohort worse than the rest of their cohort

(*p=0.029 Scheffe post hoc analysis from ANOVA).(*p=0.029 Scheffe post hoc analysis from ANOVA).


Validation: Reporting and LiaisonValidation: Reporting and Liaison

Medical Education CommitteeMedical Education Committee

Student Affairs BoardStudent Affairs Board

Senior Tutor systemSenior Tutor system

Queen Mary’s Learning Support UnitQueen Mary’s Learning Support Unit


Validation: Publications & presentationsValidation: Publications & presentationsASME conference: Identifying and supporting medical students with ASME conference: Identifying and supporting medical students with communication difficulties; June 2005, Londoncommunication difficulties; June 2005, London

First International Clinical Skills Conference – 2005 Prato, Italy.First International Clinical Skills Conference – 2005 Prato, Italy.

Educational psychologist input - 2005Educational psychologist input - 2005

CETL - 2005CETL - 2005

Presentation at 11Presentation at 11thth Ottawa international conference on medical Ottawa international conference on medical education, 2004education, 2004

Draper’s award for excellence in teaching & learning 2004Draper’s award for excellence in teaching & learning 2004

Example of good practice at the LTSN-01 National Meeting of Medical Example of good practice at the LTSN-01 National Meeting of Medical Curriculum Co-ordinators 2002Curriculum Co-ordinators 2002

Sayer M. Chaput de Saintonge M. Evans D. Wood D. Support for Sayer M. Chaput de Saintonge M. Evans D. Wood D. Support for students with academic difficulty. Medical Education. 2002 students with academic difficulty. Medical Education. 2002 July;36(7):643 – 650.July;36(7):643 – 650.

ASME: Association for the Study of Medical Education Annual ASME: Association for the Study of Medical Education Annual Scientific Meeting 2000, Dublin.Scientific Meeting 2000, Dublin.


The futureThe futurePublications

Focus on faculty development

Focus on dissemination


SummarySummaryStudents always have, do and always will struggleStudents always have, do and always will struggleThey struggle for a wide variety of reasonsThey struggle for a wide variety of reasonsRemedial programmes should be personalised to Remedial programmes should be personalised to meet individual learning needsmeet individual learning needsSuccessful remedial programmes are resource-Successful remedial programmes are resource-intensive and require institutional support for intensive and require institutional support for successsuccessThere are exciting opportunities for further There are exciting opportunities for further development and dissemination of this programme.development and dissemination of this programme.


QuestionsQuestions



Group 1Group 1

What do you think are the What do you think are the most transferable aspects of most transferable aspects of this Academic Support Model this Academic Support Model to your own institution?to your own institution?

Group 2Group 2

Group 3Group 3

Group 4Group 4


CloseClose

Academic Support NetworkAcademic Support Network

Goodbye and safe Goodbye and safe journey home!journey home!


LTSN-01 WorkshopLTSN-01 Workshop Supporting students who are Supporting students who are

struggling academically:struggling academically:one medical school’s approachone medical school’s approach

Jo BrownJo BrownDason EvansDason Evans

1212