communication skills assessment: the perceptions of medical students at the university of nottingham

11
Communication skills assessment: the perceptions of medical students at the University of Nottingham Charlotte Rees, 1 Charlotte Sheard 2 & Amy McPherson 2 Introduction Despite the wealth of literature surround- ing communication curricula within medical education, there is a lack of in-depth research into medical stu- dents’ perceptions of communication skills assessment. This study aims to address this gap in the research literature. Methods Five focus group discussions were conducted with 32 students, with representatives from each of the 5 years of the medical degree course at Nottingham University. Audiotapes of the discussions were tran- scribed in full and the transcripts were theme analysed independently by 2 analysts. Results Two assessment-related themes emerged from the analysis: namely, students’ perceptions of formative assessment and students’ perceptions of summative assessment. While students seemed to value formative methods of assessing their communication skills, they did not appear to value summative methods like objective structured clinical examinations (OSCEs). Students had mixed views about who should assess their oral communication skills. Some students pre- ferred self-assessment while others preferred peer assessment. Although students appeared to value med- ical educators assessing their communication skills, other students preferred feedback from patients. Al- though summative methods like OSCEs were criticized widely, students suggested that examinations were essential to motivate students’ learning of communica- tion skills. Discussion This study begins to illustrate medical stu- dents’ perceptions of communication skills assessment. However, further research using large-scale surveys is required to validate these findings. Medical educators should provide students with feedback on their com- munication skills wherever possible. This feedback should ideally come from a combination of different assessors. Over-assessment in other subject areas should be minimized to prevent students being dis- couraged from learning communication skills. Keywords Education, medical, undergraduate *stand- ards; *communication; students, medical; attitudes; educational measurement; England. Medical Education 2002;36:868–878 Introduction The General Medical Council (GMC) 1 has stated that, by the end of their undergraduate education, medical students should have acquired and demonstrated their proficiency in communication. In terms of acquiring proficiency in communication, much has been written about the various methods of teaching and learning communication skills 2 and the effectiveness of those methods. 3,4 In addition, several papers have discussed how medical students can best demonstrate their proficiency in communication by talking about the assessment of communication skills. When contemplating communication skills assess- ment, 3 different but related issues must be taken into consideration. The first issue concerns whether the assessment is formative or summative. Some medical educators have focused on formative assessment meth- ods like direct observation of communication skills with feedback, 5 while others have concentrated on summa- tive methods like objective structured clinical examina- tions (OSCEs) 6,7 or written reflective portfolios. 8 The second issue concerns the aspects of communi- cation skills that should be assessed. For example, some medical educators have focused on assessing medical students’ verbal communication skills by direct observation. 6,7,9–11 Other medical educators have con- centrated on assessing students’ knowledge of commu- nication skills theory 6,7,12 and still others have focused 1 Peninsula Medical School, Plymouth, UK 2 Behavioural Sciences Section, Division of Psychiatry, University of Nottingham, UK Correspondence: Dr Charlotte Rees, Peninsula Medical School, ITTC Building, Tamar Science Park, Davy Road, Plymouth PL6 8BX, UK. Tel.: 00 44 1752 764261; Fax: 00 44 1752 764226; E-mail: [email protected] Students and tests 868 Ó Blackwell Science Ltd MEDICAL EDUCATION 2002;36:868–878

Upload: charlotte-rees

Post on 06-Jul-2016

214 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Communication skills assessment: the perceptions of medical students at the University of Nottingham

Communication skills assessment: the perceptions of medicalstudents at the University of Nottingham

Charlotte Rees,1 Charlotte Sheard2 & Amy McPherson2

Introduction Despite the wealth of literature surround-

ing communication curricula within medical education,

there is a lack of in-depth research into medical stu-

dents’ perceptions of communication skills assessment.

This study aims to address this gap in the research

literature.

Methods Five focus group discussions were conducted

with 32 students, with representatives from each of the

5 years of the medical degree course at Nottingham

University. Audiotapes of the discussions were tran-

scribed in full and the transcripts were theme analysed

independently by 2 analysts.

Results Two assessment-related themes emerged from

the analysis: namely, students’ perceptions of formative

assessment and students’ perceptions of summative

assessment. While students seemed to value formative

methods of assessing their communication skills, they

did not appear to value summative methods like

objective structured clinical examinations (OSCEs).

Students had mixed views about who should assess

their oral communication skills. Some students pre-

ferred self-assessment while others preferred peer

assessment. Although students appeared to value med-

ical educators assessing their communication skills,

other students preferred feedback from patients. Al-

though summative methods like OSCEs were criticized

widely, students suggested that examinations were

essential to motivate students’ learning of communica-

tion skills.

Discussion This study begins to illustrate medical stu-

dents’ perceptions of communication skills assessment.

However, further research using large-scale surveys is

required to validate these findings. Medical educators

should provide students with feedback on their com-

munication skills wherever possible. This feedback

should ideally come from a combination of different

assessors. Over-assessment in other subject areas

should be minimized to prevent students being dis-

couraged from learning communication skills.

Keywords Education, medical, undergraduate ⁄ *stand-

ards; *communication; students, medical; attitudes;

educational measurement; England.

Medical Education 2002;36:868–878

Introduction

The General Medical Council (GMC)1 has stated that,

by the end of their undergraduate education, medical

students should have acquired and demonstrated their

proficiency in communication. In terms of acquiring

proficiency in communication, much has been written

about the various methods of teaching and learning

communication skills2 and the effectiveness of those

methods.3,4 In addition, several papers have discussed

how medical students can best demonstrate their

proficiency in communication by talking about the

assessment of communication skills.

When contemplating communication skills assess-

ment, 3 different but related issues must be taken into

consideration. The first issue concerns whether the

assessment is formative or summative. Some medical

educators have focused on formative assessment meth-

ods like direct observation of communication skills with

feedback,5 while others have concentrated on summa-

tive methods like objective structured clinical examina-

tions (OSCEs)6,7 or written reflective portfolios.8

The second issue concerns the aspects of communi-

cation skills that should be assessed. For example, some

medical educators have focused on assessing medical

students’ verbal communication skills by direct

observation.6,7,9–11 Other medical educators have con-

centrated on assessing students’ knowledge of commu-

nication skills theory6,7,12 and still others have focused

1Peninsula Medical School, Plymouth, UK2Behavioural Sciences Section, Division of Psychiatry, University of

Nottingham, UK

Correspondence: Dr Charlotte Rees, Peninsula Medical School, ITTC

Building, Tamar Science Park, Davy Road, Plymouth PL6 8BX, UK.

Tel.: 00 44 1752 764261; Fax: 00 44 1752 764226; E-mail:

[email protected]

Students and tests

868 � Blackwell Science Ltd MEDICAL EDUCATION 2002;36:868–878

Page 2: Communication skills assessment: the perceptions of medical students at the University of Nottingham

on students’ self-assessment and their abilities to reflect

on their communication skills.8,13,14 Humphris and

Kaney6 suggest that the measurement of observable

behaviours has been welcomed by medical educators

due to the unknown association between performance

on written tests and actual communication skills with

patients.

The third issue about communication skills assess-

ment concerns the assessor. In terms of direct obser-

vation of medical students’ communication skills, many

authors discuss the medical educator as the asses-

sor.5,6,9–11 Other authors discuss patients,15 students’

peers16 or the students themselves as assessors.13

Aspegren’s review3 suggested that students’ self-ratings

of their communication skills correlated poorly with

external independent ratings, with both over-rating17

and under-rating18 on the part of the students. In

addition, Cooper and Mira15 found that academic

teachers’ assessments of students’ communication skills

correlated poorly with simulated patients’ assessments,

with lay people giving students higher scores than

academic teachers. The authors concluded that the

communication skills emphasized by academic teachers

did not reflect the skills considered important by

simulated patients.

Although papers have discussed communication

skills assessment, there is a lack of in-depth exploration

into students’ perceptions of communication skills

assessment. In order to address this gap in the research

literature, this study aims to answer the following

research questions:

1 What are students’ views on how communication

skills are assessed?

2 What do students like about communication skills

assessment?

3 What do students dislike about communication skills

assessment?

Methods

Design

This study employed a qualitative design in order to

explore in depth medical students’ perceptions of

communication skills assessment. Focus group meth-

odology19 was used for 3 reasons. Firstly, focus

groups are extremely useful for carrying out explor-

atory research.20 Secondly, the interaction among

group members often produces data unobtainable

using other qualitative methods such as individual

interviews.21 Finally, the interaction between group

members replaces the interaction between participants

and the interviewer, thus minimizing interviewer

bias.21 In the present study, 5 focus group discus-

sions, lasting between 1Æ5 and 2 hours each, were

convened with students from each of the 5 years of

the medical degree course at the University of

Nottingham.

Sampling and recruitment

After receiving ethical approval, all students registered

at the University of Nottingham’s Medical School in

the academic session 2000–01 were invited to parti-

cipate in this study. Three different methods were

used to recruit students to this study. Initially, all stu-

dents were sent an invitation pack by post in October

2000. This pack included a covering letter, an

information sheet, a research consent form and a

Freepost envelope. The same information was also

e-mailed to students. Students were asked to read the

information sheet and, if interested in participating,

encouraged to return the research consent form to

the researchers. However, because this recruitment

method led to a poor response (11 ⁄965 ¼ 1Æ1%),

additional students were recruited using 2 other meth-

ods. The first method involved teaching staff asking

students in seminars whether they would like to

participate in the study. The second method involved

the authors contacting student year representatives to

ask them whether they or any of their peers would

like to participate in the discussions. Having submit-

ted consent forms, respondents were contacted by

phone or e-mail to arrange convenient dates, times

and locations for their focus groups to take

place. Those who gave their consent to participate

Key learning points

While students seem to value formative methods

of assessing their communication skills, they do

not appear to value summative methods such as

OSCEs.

Students have different opinions about who should

assess their oral communication skills.

Despite student criticism of OSCEs, students

suggest that summative assessment is necessary to

motivate students to learn communication skills.

These qualitative data are not intended to be

generalized to a wider population of medical

students; therefore further research using large-

scale surveys is required to validate these study

findings.

Perceptions of communication skills assessment • C Rees et al. 869

� Blackwell Science Ltd MEDICAL EDUCATION 2002;36:868–878

Page 3: Communication skills assessment: the perceptions of medical students at the University of Nottingham

electronically or verbally were asked to sign a consent

form on the day of the focus group discussion.

Materials

Personal details questionnaire

The personal details questionnaire collected demogra-

phic and education-related details from the partici-

pants. Demographic items included age, gender, ethnic

origin, the employment histories of the participants’

parents (to calculate socioeconomic group) and first

languages. Education-related items included year of

study.

Discussion guide

Although the discussions were relatively unstructured,

the discussion guide contained a number of distinct

sections: welcome, introduction, anonymity, ground

rules, warm-up, clarification question, key questions,

concluding question and conclusion.18 The primary

purpose of this guide was to serve as an aid to memory

for the moderators. An adaptation of this discussion

guide was used successfully in a pilot study carried out

by the first author to explore medical students’

attitudes towards communication skills learning.22

Focus group procedure

The moderators welcomed participants and informed

them that the purpose of the discussion was to

determine their perceptions of communication skills

learning. The issue of anonymity was discussed and

participants were asked to give their consent to the

discussion being audiotaped. In order to manage

the group discussions, ground rules were outlined to

the participants. They were informed that only one

person should speak at a time, there were no right or

wrong answers, they did not have to speak in any

particular order and they did not have to agree with

others but needed to state their points of view without

making negative comments about the views or experi-

ences of other group members. The participants were

also informed that they could be interrupted and their

conversation re-directed. The participants were invi-

ted to ask questions before the discussion began

properly.

Participants introduced themselves by saying their

names to the group. Participants were asked to con-

template the words �communication skills� and express

what the words meant to them, thereby drawing them

to the specific topic of inquiry. The participants were

given a non-threatening forum in which they could

discuss issues they felt were important. However, they

were interrupted if they digressed to such an extreme

that they no longer focused on issues relevant to the

study. In these cases, they were brought back to the

topic of inquiry using key questions such as �How are

communication skills assessed?� �What do you like

about communication skills assessment?� and �What

do you dislike about communication skills assessment?�Finally, participants were asked what they thought were

the most important issues concerning communication

skills learning. After taking part in the discussion,

participants were thanked for their time and asked to

complete the demographic questionnaire before they

left the meeting.

Data analysis

All the discussions were audiotaped and transcribed in

full. The transcripts were analysed using a manual cut-

and-paste method of theme analysis,19 which is an

adaptation of naturalistic inquiry23 and the constant

comparative method.24 Analysis followed an iterative

process composed of several stages. The first analytic

stage involved the identification of strong themes

running through the data. The second stage involved

the identification of information units, i.e. �the smallest

amount of information that is informative by itself�.19

The third stage involved the categorization of the infor-

mation units.

Stages one to 3 were carried out independently by 2

analysts (either CS and CR or CS and AM) for each

focus group discussion. The fourth stage involved the

analysts comparing their themes, which allowed for

determination of interrater reliability. Any differences

were then negotiated. This process was conducted

separately for each of the 5 focus group transcripts.

Data were then coded across all 5 focus groups. The

level of agreement between the 2 analysts for each of

the 5 focus group discussions were j ¼ 0.500 (year

1), j ¼ 1.000 (year 2), j ¼ 0.467 (year 3),

j ¼ 1.000 (year 4) and j ¼ 0.588 (year 5). These

levels of agreement were moderate (year 1), perfect

(year 2), moderate (year 3), perfect (year 4) and

moderate (year 5), as indicated by Landis and Koch.25

Finally, corresponding verbatim quotations were used

to support all themes. An adaptation of these analytic

procedures has been used successfully in previous

research.22

Responses to the demographic questionnaire were

analysed using SPSS Version 10Æ0. First, exploratory

data analysis was carried out to determine the distri-

bution of the continuous variable �age�. A median and

interquartile range was established for this non-nor-

mally distributed variable. Secondly, the frequencies

Perceptions of communication skills assessment • C Rees et al.870

� Blackwell Science Ltd MEDICAL EDUCATION 2002;36:868–878

Page 4: Communication skills assessment: the perceptions of medical students at the University of Nottingham

and percentages of categorical variables (e.g. gender)

were determined. Socio-economic class was established

following existing guidelines on standard occupational

classification.26

Validity was established by participant evaluation of

the findings. All 32 medical students were sent a copy

of the results with an evaluation sheet. A total of 17

(53Æ1%) students, comprising representatives from each

of the 5 discussion groups, returned the evaluation

sheet. All of them (100%) stated that the results were a

fair interpretation of the discussions in which they had

participated.

Results

Participant characteristics

Of the 965 students invited to participate in this study,

32 took part, giving a response rate of 3Æ3%. Their ages

ranged from 19 to 27 years (median ¼ 20, interquar-

tile range 19Æ25–22Æ0 years). The majority were white

(n ¼ 28, 87Æ5%), female (n ¼ 22, 68Æ8%) and came

from socio-economic classes I and II (n ¼ 31,

96Æ9%).26 All spoke English as their first language.

Seven (21Æ9%) of the participants were first year

students, 7 (21Æ9%) were second year, 10 (31Æ3%)

were third year, 5 (15Æ6%) were fourth year and 3

(9Æ4%) were fifth year students.

Themes

Two distinct assessment-related themes emerged as a

result of the 5 focus group discussions:

1 students’ perceptions of formative assessment, and

2 students’ perceptions of summative assessment.

Students’ perceptions of formative methods

of assessing communication skills

Students’ communication skills were most commonly

assessed formatively using a method in which a student

would be observed conducting an interview with a

simulated patient. They would then receive immediate

feedback from medical educators and their peers. (All

names attached to the following excerpts have been

changed for the purposes of anonymity.)

Penny: You did it [the interview] on your own in a

room and it was videotaped and then they [peer

group] all watched you and then you had to come

back in and sort of everyone criticized you and said,

�You should have done it like this.�

Clara: Very similar to what we did in the first year.

Penny: But more difficult scenarios.

(Penny and Clara, year 4 students.)

This method of formative assessment seemed to be

valued by students. Not only did they think that the

simulated interviews helped them practise their com-

munication skills, but they also considered the imme-

diate feedback they received helped them develop these

skills:

It’s a true learning experience and you don’t feel

you’re being given a mark out of 10 and I felt that’s

the best assessment for communication skills that

we’ve done.

(Margaret, year 5 student.)

A third year student insisted that all students should

receive feedback on how they could improve their

communication skills:

What we want is how we can improve and that

should be told to everyone who’s teaching these

things, and especially assessing these things, that you

should give feedback.

( Jane, year 3 student.)

However, students had mixed views about who

should assess communication skills formatively.

Self-assessment

Some students suggested that it was important to be

able to assess themselves and identify what they did well

and which aspects of their communication they could

improve. A first year student suggested that self-assess-

ment was the key to formative assessment, largely

because different assessors have different views about

what constitutes good communication:

I think it is quite difficult [to assess communication],

because everyone’s different, and everyone has a

different view of what good communication skills are.

The most important thing is self-assessment.

( Will, year 1 student.)

Although students found it useful and reasonably

straightforward to assess a videotape of themselves

communicating with a patient, they found it difficult to

reflect more globally on their communication skills,

Perceptions of communication skills assessment • C Rees et al. 871

� Blackwell Science Ltd MEDICAL EDUCATION 2002;36:868–878

Page 5: Communication skills assessment: the perceptions of medical students at the University of Nottingham

particularly if their communication skills had not been

captured on video or audiotape:

Trying to reflect on what you’ve done yourself when

you’ve got nothing to reflect… it’s going to be better

for the other years �cause they can video themselves,

they can sit down with a piece of paper and say �I did

that badly, look at me there, that was really bad�, but

this year we can’t do that.

(Claire, year 2 student.)

Peer assessment

Some of the older clinical students valued peer assess-

ment, suggesting that it was useful for them to get

feedback from people who were similar to themselves

and who could truly empathize with their feelings about

interviewing patients:

I valued what they [peers] said as much, if not a bit

more sometimes than what the other person [GP],

because they knew exactly what I was going, you

know, you’re going through the same things, so for

them to say something which is good, you know, it was

quite good, you had to say something positive first or

whatever and then, but then even like the things they

said you could improve on you actually took on board.

(Stuart, year 5 student.)

In addition, some older clinical students thought that

peer assessment offered them opportunities to compare

their communication skills with the skills of other

students:

It was quite interesting to be able to compare how

well you thought you were with the rest of the people

in your group because otherwise you have no sort of

standard apart from the doctors to measure yourself

against, because obviously you’re not going to be as

good as doctors.

(Penny, year 4 student.)

Two second year students stated that peer assess-

ment was a useful learning experience for the assessor

too:

Sandra: I think you pick up probably on things you

might do yourself and see perhaps that you shouldn’t

and also you can learn from them, like you can think,

�Oh, they dealt with that really well, so I could use

that.� Like I wouldn’t really have known how to deal

with that but now I’ve seen them it’s like helped me.

John: They’ve done that and I wouldn’t do that, or

that was bad and it makes you aware.

Sandra: You learn from each other, don’t you?

(Sandra and John, year 2 students.)

While some older students enjoyed receiving feedback

from groups of peers, younger preclinical students

seemed more dissatisfied with group peer observation.

Two first year students criticized group peer observation

because they felt it was unfair to receive feedback in front

of a group and thought that their peers were often too

polite to offer constructive criticism in a group setting:

Kathryn: I think peer assessment in a big group is a bit

mean but if you’re with a partner, no-one else knows

how you’ve done apart from that one person…if you

make a pact with each other that you’ll be brutally

honest.

Anna: People can be too polite, because they’re too

worried about hurting people’s feelings, like �you did

really, really well�, when actually they’re thinking �no

you didn�t’ but they’re not going to say that if it’s a

big group.

(Kathryn and Anna, year 1 students.)

A second year student stated that it was difficult to be

objective about friends’ communication skills:

It’s hard to be objective because they’re your friends

or whatever and they’re going to mark you as well, so

you scratch my back, I’ll scratch yours.

(John, year 2 student.)

Other younger preclinical students felt it was difficult

to give feedback because they did not feel expert

enough to comment on other students’ communication

skills:

It’s very difficult for the ones observing to know, well,

�are they doing this right?� Because they don’t know

themselves what’s right and what’s wrong and what’s

the best way to talk to someone.

(Claire, year 2 student.)

When asked why they thought older students valued

peer assessment more than younger students, fifth year

students thought it was because they were more

mature, had more clinical experience and knew their

peers much better than first and second year students:

Perceptions of communication skills assessment • C Rees et al.872

� Blackwell Science Ltd MEDICAL EDUCATION 2002;36:868–878

Page 6: Communication skills assessment: the perceptions of medical students at the University of Nottingham

Interviewer: A lot of students have said that they’d

rather have feedback from teachers rather than peers.

David: That’s �cause we�re older. Second years and

fourth years are miles apart in terms of what you

know, what to expect.

Stuart: There are 2 things about the fifth year.

You’ve gone through clinicals together to start with,

so I think you know each other better but also you’ve

had to do, you do more communicating with patients

in your first clinical attachment than you’ve done in

your first 3 years…and secondly, you know, you’re

that much older.

(David and Stuart, year 5 students.)

Teacher assessment

Some students seemed to place more value on medical

educators than peers as assessors of their communica-

tion skills, largely because educators were thought to be

better able to pick up on faults than peers. Some

students valued feedback from non-clinical lecturers

such as psychologists:

They can say, �You�re fidgeting all the time or your

body language is aggressive’, which is useful.

(John, year 2 student.)

However, despite witnessing many examples of poor

communication skills from qualified doctors, clinical

students appeared to value feedback from clinical edu-

cators more than non-clinical educators:

You [psychologists] are experts in communication

skills even more so perhaps than GPs, who have the

medical knowledge but might not be good commu-

nicators… My GP videoed me and then we sat and

watched the video and I thought that was really

good…I found that was good because it was a real

doctor who was saying, �Well, you should do this, you

should do that� and you don’t mind criticism from a

doctor.

(Clara, year 4 student.)

Some students received useful feedback from their

clinicians regarding their communication skills:

What he [the GP] did was there was a patient

asking why he was breathless when he laid down, so

he said, �You can explain it. You tell it to me and

then when the patient comes in you explain it� and

he told me what I did right or wrong and that was

really useful.

(Sam, year 2 student.)

However, some students criticized the feedback they

received from their clinicians, either stating that it was

very harsh or that the feedback wasn’t particularly

constructive:

I mean, he [consultant] ripped everyone apart while

you were presenting it and it took like an hour to do a

presentation, 2 people, and you got torn to shreds

but it was worthwhile because you learnt what they

were looking for, what you hadn’t said, but it was

scary.

(Lauren, year 3 student.)

There’s 2 or 3 sentences for feedback, and she [stu-

dent] went and she said �You know, I was wondering

how can we improve on this for next time?� and the

guy [doctor] was just absolutely so blase, and he

basically said something like �Oh, you know, you

can�t expect to get an �A� all the time’ and didn’t tell

her anything about it.

(Jane, year 3 student.)

Another criticism of the feedback from medical edu-

cators was that feedback in GP-led seminars seemed

unnecessarily drawn out:

It got to the stage where you, it was a bit drawn out,

you were trying to carry on analysing the interview

when everything had been said, and you know, trying

to draw out some criticism from somewhere when the

person on the video had done perfectly well.

(Derek, year 4 student.)

Students’ perceptions of summative methods

of assessing communication skills

Students’ verbal communication skills were most com-

monly assessed summatively by communication stations

within OSCEs. Other summative assessments included

written coursework, tapping either students’ knowledge

of communication skills theory or their abilities to

reflect on their communication skills.

The summative assessment of communication skills

did not appear to be valued by the students. Although

one fifth year student did state that the OSCE was a fair

assessment, students mainly criticized OSCEs:

Perceptions of communication skills assessment • C Rees et al. 873

� Blackwell Science Ltd MEDICAL EDUCATION 2002;36:868–878

Page 7: Communication skills assessment: the perceptions of medical students at the University of Nottingham

To come back to the whole OSCE thing, it’s the most

ridiculous assessment I have ever had to go through

in my life.

(Sally, year 3 student.)

OSCEs were criticized because they were thought to

be artificial and because the assessment failed to match

communication skills learning and teaching:

You look at the mark sheet and think �I�ve got to [do]

some empathy’, you know the mark sheets, so you

think of what you’ve got to put in. It’s a bit false.

(Clara, year 4 student.)

It bears no relation to anything we’ve been taught

before and if you pray, if you’re really lucky you get in

the afternoon group, so you can ask the morning

group what they had to do…because it’s honestly,

some of the assessment you didn’t have a hope in hell

of passing unless you’d spoken to somebody who’d

already done it.

(Sally, year 3 student.)

OSCEs were also criticized because there were limi-

ted opportunities for students to receive feedback to

help them improve their communication skills:

You don’t get any feedback because it’s so quick

and you’re moving on to the next one and you get

a brief �Oh, you said this, you missed this out�, you

don’t get �Yeah, that was good, you were talking to

them at the right level or perhaps you could think

about doing this�, because in that situation, it’s all,

you know, time, and you haven’t got time for better

feedback.

(Caroline, year 3 student.)

A fourth year student questioned the appropriateness

of summatively assessing first and second year students’

communication skills when they had received limited

learning opportunities in communication with real pa-

tients. This student felt that it was problematic to fail

first and second year students for their communication

skills:

But how are they going to improve? You’ve failed

your communication skills, �Shit, what am I going to

do?� Like, it’s not going to get people any better

necessarily because they’ve got no opportunity to

practise. Like, you can’t just revise it. You can’t say,

�I�ll go and read my book a bit more and then I’ll be

okay afterwards’. If you fail your whatever, I can’t

remember any of our courses, say your biochemistry,

you can go back to your book and re-learn it. You can

re-do your exam, but if you fail your communication

skills exam you’re telling an 18, 19-year-old that

they’re not capable of being a doctor and that’s not

true and it’s not really fair.

(Penny, year 4 student.)

Finally, OSCEs were also criticized because students

felt that their communicative performance often failed

to match what their communication skills were really

like. A fourth year student suggested that it would be

more appropriate for students to receive marks from an

individual who knew them well and had seen them

communicating on many separate occasions rather than

an individual who simply assessed their communication

skills in a 5-minute OSCE:

The GP gives you your grade that you’ve attained in

that attachment and I think it’s actually one of the best

ways to do it because you spend a month with this

doctor and he knows what you’re like by the end of it

and how willing you are to work and how good your

communication skills are. So, I think it’s better that he

assesses your communication skills rather than maybe

aGPwho’s justmetyou thatdayandseenyouonavideo.

(Anne, year 4 student.)

However, despite students’ criticisms of OSCEs,

students thought that the assessment of their practical

skills was far more appropriate than the assessment of

their knowledge of communication skills theory:

Anna: Cause you could learn the theory, you could

sit there and learn your notes. But it’s whether you

can actually apply it, that’s what’s important…

Tony: Yeah. I think that the theory should be

completely ignored in the assessment.

(Anna and Tony, year 1 students.)

Students had different opinions about who should

assess their communication skills in an OSCE. Some

students felt that patients should assess their commu-

nication skills rather than medical educators:

The particular lecturer just said �Right, you did this,

that and the other, and this and that�…and the

patient’s sat there and I thought, �Well, why don�tyou give the patient the form and let the patient mark

it’ and that’s how I think you should do it.

(Jake, year 3 student.)

Perceptions of communication skills assessment • C Rees et al.874

� Blackwell Science Ltd MEDICAL EDUCATION 2002;36:868–878

Page 8: Communication skills assessment: the perceptions of medical students at the University of Nottingham

A second year student explained that patients’

assessment of medical students’ communication skills

would have more validity:

The examiner could be sitting thinking, �Oh yeah,

they said that and that put them [the patient] at ease�but it might not have put the patient at ease. It’s…the

examiner’s view, �Oh, they said good morning, so

that�s put them nice at ease’. �Good morning?�The guy could be sitting there thinking, �Ask me

what�s wrong, ask me what’s wrong’ and is tense

as anything.

(Claire, year 2 student.)

However, other students criticized patients’ assess-

ments of their communication skills because they were

too generous and failed to give them any constructive

criticism:

In the communication block we had to get patients to

tell us how well we communicated and, you know,

they just gave us a mark, you didn’t get much

feedback on it but it was really silly and a complete

waste of time because I know my communication

skills aren’t 10 out of 10 for like 10 patients and they

were all 10 out of 10…so I don’t think you learn very

much from their feedback.

(Margaret, year 5 student.)

Communication skills learning is dependent on summative

assessment

Some students suggested that the summative assess-

ment of communication skills motivated students to

learn communication skills:

I don’t think people do the interviews for fun, they

do it because they have to do it, you know, it’s an

exam.

(John, year 2 student.)

A first year student suggested that students only learn

communication skills if they know they are being

examined on it:

I think you need to examine it though, in some way,

to make people go and do it. Just a small piece of

coursework, nothing major, you know, like the piece

we did wasn’t that strenuous, but it made you go and

interview the patient, if you’ve got a few bits of work,

people are more likely to go and do it.

(Liam, year 1 student.)

Another student suggested that one way to motivate

students to attend communication skills sessions would

be to award them marks for their attendance:

Making people come to the sessions, like saying, �Part

of your mark will be rewarded on whether or not

you�ve attended all of the sessions’. That’s the only

way to get people to go.

(Sally, year 3 student.)

A fourth year student suggested that over-assessment

in other subject areas, particularly in the first 2 years,

inhibited students’ learning of communication skills:

I think half the problem with our course is that it’s so

focused on learning and everyone has this like focus

towards exams at the end of the course. So, that’s

basically what we do, and we don’t think, �Oh, let�sdo this to become better communicators’…because

like it’s not going to be assessed, so it doesn’t matter.

I think that’s because particularly during the first and

second year we have so many intense exams at the

end of each semester, that’s all that everybody really

focuses on.

(Emily, year 4 student.)

Discussion

To our knowledge, this is the first study to explore

medical students’ perceptions of communication skills

assessment in depth. Two related but different themes

emerged from the analysis of the 5 focus group discus-

sions: students’ perceptions of formative assessment

and their views regarding summative assessment. With

regard to formative assessment, students received

feedback from their peers and medical educators on

their communication skills while interviewing simulated

patients. Students valued this feedback and felt that it

helped them to develop their communication skills, a

finding supporting earlier work27 showing that experi-

ential methods were more effective than instructional

methods in learning communication skills.3 In addition,

it supports GMC28 recommendations that students

should be provided with regular and consistent infor-

mation about their development and progress.

Students had different views about who should assess

their communication skills formatively. Some students

preferred self-assessment, although this has been found

to correlate poorly with external independent ratings in

previous studies.13,17,18 Other students, particularly

those in their clinical years, preferred peer assessment,

either because they felt their peers could empathize

Perceptions of communication skills assessment • C Rees et al. 875

� Blackwell Science Ltd MEDICAL EDUCATION 2002;36:868–878

Page 9: Communication skills assessment: the perceptions of medical students at the University of Nottingham

with their feelings about interviewing patients or

because assessing their peers’ communication skills

was a useful learning experience. On the other hand,

non-clinical students seemed to be more dissatisfied

with peer assessment, either because they felt students

were too polite to criticize or were unable to offer

constructive criticism due to a lack of knowledge and

experience. Some clinical students suggested that diff-

erences in views between themselves and preclinical

students might have resulted from their having more

clinical experience and therefore more experience in

communicating with patients. Aspegren3 suggests that

communication skills training in clinical clerkships may

be more effective than in preclinical courses, which

could influence students’ perceptions of peer assess-

ment. Finally, other students preferred receiving feed-

back from medical educators, as has been indicated in

previous research.29 Interestingly, although some stu-

dents had observed clinicians with poor communication

skills, clinical students seemed to prefer receiving feed-

back from clinical educators rather than from non-

clinical educators. This finding contradicts previous

research29 which found that students rated social

scientists higher than doctors as teachers in small group

communication skills training. This contradictory find-

ing may be due to the different samples used in these 2

studies. For example, Quirk and Letendre’s study29

surveyed first year medical students only and the

present finding was associated with the views of clinical

students. Indeed, while preclinical students may be

satisfied with communication skills teaching from social

scientists, clinical students may prefer communication

skills teaching from qualified doctors.

Unlike formative assessment methods, some students

did not value summative assessment methods for com-

munication skills. In particular, students criticized

OSCEs for being artificial, for failing to represent what

they had learned during the course and for failing to

provide them with opportunities for feedback to

enhance their learning. Nevertheless, the summative

assessment of students’ oral communication skills was

valued more highly than the assessment of their

knowledge of communication skills theory, a view

that is consistent with the educational shift from

written assessments of communication skills to the

assessment of observable behaviours.6 As mentioned

previously, the measurement of observable behaviours

has been welcomed by medical educators because of

the unknown association between performance on

written tests and actual communication skills with

patients.6

Students had different opinions regarding who

should assess their oral communication skills

summatively. On the one hand, some students felt

that patients should mark their communication skills

within OSCEs and other students criticized patients’

assessments because they over-estimated students’

communication skills, as has been found in previous

research.15

Some students suggested that summative assessment

motivates students to learn communication skills.

However, over-assessment in other subject areas, par-

ticularly in the first 2 years, was thought to inhibit

students’ learning of communication skills. This finding

is consistent with the worries expressed by the GMC,1

which stated that students were reluctant to afford time

to explore areas in which they were not examined.

This study has one major limitation that must be

taken into consideration when interpreting the results.

Although all medical students registered at the Univer-

sity of Nottingham were invited to participate in this

study, only 32 students eventually took part. Although

this sample size is satisfactory for a qualitative research

project, the response rate was very low (3Æ3%). There

are 2 possible reasons for this low response rate. Firstly,

students may not have been able or prepared to give up

2 hours of their time to participate in a group discus-

sion. Secondly, students may not have been motivated

to discuss communication skills learning, possibly

because they did not have particularly strong views

concerning such learning. Certainly, the students who

did take part in the focus group discussions appeared to

possess strong views (either very positive or very

negative) regarding communication skills learning.

However, because the response rate was so low, we

cannot be sure that responders were similar to non-

responders. In fact, the demographic characteristics of

our sample were different from those of Nottingham

University medical students as a whole (n ¼ 1016 in

the academic session 2001–02). For example, students

in our sample were more likely to be female (n ¼ 22,

68Æ8%) compared with Nottingham medical students

overall (n ¼ 635, 62Æ5%). Students in our sample

were also more likely to be white (n ¼ 28, 87Æ5%)

compared with Nottingham medical students overall

(n ¼ 729, 73Æ7%). It is possible, therefore, that non-

responders may have possessed different perceptions of

communication skills assessment to responders.

Although qualitative research is not meant to be

generalized to a wider population, the low response

rate in this study indicates that the fullest spectrum of

views may not have been elicited.

Despite this methodological limitation, the data

analysis for this study was reliable (see kappa values)

and the participants thought that the results were a

fair interpretation of the discussion, thus giving

Perceptions of communication skills assessment • C Rees et al.876

� Blackwell Science Ltd MEDICAL EDUCATION 2002;36:868–878

Page 10: Communication skills assessment: the perceptions of medical students at the University of Nottingham

evidence for the validity of the study. The study does

begin to illustrate medical students’ perceptions of

communication skills assessment and the findings have

implications for both further research and educational

practice. Large-scale surveys are needed to quantita-

tively establish medical students’ perceptions of com-

munication skills assessment and how perceptions may

change during the course of medical training. This

quantitative data may then be generalized to a wider

population of medical students. In educational prac-

tice, medical educators should continue to use experi-

ential methods of teaching communication skills rather

than instructional methods. Wherever possible, med-

ical educators should provide students with feedback

on their communication skills, even in summative

assessments like OSCEs and a combination of differ-

ent assessors would probably provide the richest

feedback. Finally, medical educators should reduce

over-assessment within the medical curriculum, so

that students are not discouraged from learning

communication skills.

Contributors

All authors contributed to the data collection, data

transcription, data analysis and writing of this paper.

CR designed and secured funding for the study. CS and

AM recruited participants.

Acknowledgements

We would like to thank all the students who partici-

pated in this study.

Funding

This study was funded by a grant awarded to CR by the

Teaching Enhancement Office at the University of

Nottingham (Ref. 99TL ⁄179).

References

1 General Medical Council. Tomorrow’s Doctors. Recommenda-

tions on Undergraduate Medical Education. London: General

Medical Council; 1993.

2 Hargie O, Dickson D, Boohan M, Hughes K. A survey of

communication skills training in UK Schools of Medicine:

present practices and prospective proposals. Med Educ

1998;32:25–34.

3 Aspegren K. BEME Guide No. 2: Teaching and Learning

Communication Skills in Medicine: A Review with Quality Gra-

ding of Articles. Dundee: Association for Medical Education in

Europe; 1999.

4 Hulsman RL, Ros WJ, Winnubst JA, Bensing JM. Teaching

clinically experienced physicians communication skills. A

review of evaluation studies. Med Educ 1999;33:655–68.

5 McKinley RK, Fraser RC, van der Vleuten C, Hastings AM.

Formative assessment of the consultation performance of

medical students in the setting of general practice using a

modified version of the Leicester Assessment Package. Med

Educ 2000;34:573–9.

6 Humphris GM, Kaney S. The Objective Structured Video

Examination for assessment of communication skills. Med

Educ 2000;34:939–45.

7 Humphris GM, Kaney S. Assessing the development of

communication skills in undergraduate medical students. Med

Educ 2001;35:188–90.

8 Rees CE, Garrud P. Encouraging medical students’ reflective

practice in communication skills learning: an evaluation of a

new course. Presented at the Annual Scientific Meeting of

ASME, Royal College of Surgeons in Ireland, Dublin, 11–13

July; 2001.

9 Ram P, Grol R, Rethans JJ, Schouten B, van der Vleutan C,

Kester A. Assessment of general practitioners by video

observation of communicative and medical performance in

daily practice: issues of validity, reliability and feasibility. Med

Educ 1999;33:447–54.

10 Rowland-Morin PA, Burchard KW, Garb JL, Coe NP.

Influence of effective communication by surgery students on

their oral examination scores. Acad Med 1991;66:169–71.

11 Edwards A, Tzelepis A, Klingbeil C, Melgar T, Speece M,

Schubiner H, Burack R. Fifteen years of a videotape review

programme for internal medicine and medicine-paediatrics

residents. Acad Med 1996;71:744–8.

12 Winefield HR, Chur-Hansen A. Evaluating the outcome of

communication skill teaching for entry-level medical students:

does knowledge of empathy increase? Med Educ

2000;34:90–4.

13 Tamburrino MB, Lynch DJ, Nagel R, Mangen M. Evaluating

empathy in interviewing: comparing self-report with actual

behaviour. Teaching Learning Med 1993;5:217–20.

14 Kidd J, Nestel D. Development of a feedback process for

reflective assignments on communication skills. Presented at

the Annual Scientific Meeting of ASME, Royal College of

Surgeons in Ireland, Dublin, 11–13 July; 2001.

15 Cooper C, Mira M. Who should assess medical students’

communication skills: their academic teachers or their

patients? Med Educ 1998;32:419–21.

16 Hughes IE, Large BJ. Staff and peer-group assessment of oral

communication skills. Studies in Higher Education

1993;18:379–85.

17 Marteau TM, Humphrey C, Matoon G, Kidd J, Lloyd M,

Horder J. Factors influencing the communication skills of

first-year clinical medical students. Med Educ 1991;25:127–

34.

18 Farnhill D, Hayes SC, Todisco J. Interviewing skills: self-

evaluation by medical students. Med Educ 1997;31:122–7.

19 Vaughn S, Schumm JS, Sinagub J. Focus Group Interviews in

Education and Psychology. Thousand Oaks, CA: Sage; 1996;

pp. 106.

Perceptions of communication skills assessment • C Rees et al. 877

� Blackwell Science Ltd MEDICAL EDUCATION 2002;36:868–878

Page 11: Communication skills assessment: the perceptions of medical students at the University of Nottingham

20 Stewart DW, Shamdasani PN. Focus Groups Theory and

Practice. Applied Social Research Methods Series Vol. 20.

Thousand Oaks, CA: Sage; 1990.

21 Morgan DL. Focus Groups as Qualitative Research. Qualitative

Research Methods Series 16. Thousand Oaks, CA: Sage; 1988.

22 Rees CE, Garrud P. Identifying undergraduate medical stu-

dents’ attitudes towards communication skills learning: a pilot

study. Med Teacher 2001;23:400–6.

23 Lincoln YS, Guba EG. Naturalistic Inquiry. California: Sage;

1985.

24 Glaser BG, Strauss AL. The Discovery of Grounded Theory:

Strategies for Qualitative Research. New York: Aldine de

Gruyter; 1967.

25 Landis JR, Koch GG. The measurement of observer agree-

ment for categorical data. Biometrics 1977;33:159–74.

26 Office of Population Censuses and Surveys. Standard Occu-

pational Classification Vol. 3. London: HMSO; 1991.

27 Evans BJ, Stanley RO, Burrows GD, Sweet B. Lectures and

skills workshops as teaching formats in a history-taking skills

course for medical students. Med Educ 1989;23:364–70.

28 General Medical Council. Draft Recommendations on

Undergraduate Medical Education: Consultation Copy July

2001. http://www.gmc-uk.org/med_ed/tomorrowsdoctors/

guidance.htm; visited 3 August 2001.

29 Quirk M, Letendre A. Teaching communication skills to first

year medical students. J Med Educ 1986;61:603–5.

Received 28 August 2001; editorial comments to authors 2 November

2001; accepted for publication 1 March 2002

Perceptions of communication skills assessment • C Rees et al.878

� Blackwell Science Ltd MEDICAL EDUCATION 2002;36:868–878