comparative assessment of student pharmacists’ performance in computer- and paper-based dispensing...
TRANSCRIPT
Comparative assessment of student pharmacists’
performance in computer- and paper-based dispensing
exercises
Monika Sareen and Mike DalyThe School of Pharmacy,
Background
Computer simulations can be used to imitate practical tasks
Widely used in training airline pilots and medical Anaesthetists
However….Can computer simulation mimic common pharmaceutical dispensing tasks ? Can or should they be used as tools for assessment ?
PBEs versus CBEs Paper-based exercises (PBEs) are a common method
of teaching dispensing Large student cohorts mitigate against exposure to
practical dispensing tasks – around 30-40 items in one semester
Computer-based exercises (CBEs) are being developed to support PBEs
However…. Does formative use provide appropriate support ? Would summative use be fair and accurate ?
Methods A series of four prescription tasks were prepared
as PBEs These tasks were converted to a CBE using
WOLF Students sequentially undertook practical or
computer based dispensing tasks Results were marked using a standard pro-forma
already used in the department Students were asked for their comparative
opinions between the two teaching methods
WOLF screenshot
There are a number of compromises :The date is static to save timeOptions are presented as MCQs Manual dexterity is not requiredLabel production is not requiredProduct selection is artificial
Skin preparations
Labels
Results - 1 57 students undertook the two dispensing
exercises (31 male, 26 female)
Pass mark – standard 50%Female students appeared to perform worse in
the CBE than in the (practical) PBE
Evaluation PBE : 16% of male students failed the pass mark of 50%, whereas
only 1 female failed (4%) CBE : The failure rate was 35% of males and 35% of females
There appears to be a gender-related difference :
Females performed better than males on the Paper Based Exercise
but
scored equally poorly as males on the Computer Based Exercise
Does this imply that the use of computerised dispensing simulations could disadvantage
sub-groups of students ?
Preferences Students who preferred the use of the CBE over the PBE performed
better at the CBE No such correlation of preference and performance appeared with the
PBE All students stated their preference for practical paper-based
dispensing over computer simulations In general, students that scored higher electronically performed worse
in the manual exercise, and vice versa
Student performance measured electronically may not automatically match their practical ability – use forsummative assessments requires proper validation
Results - 2
The exercise was repeated the following year with a further cohort of 1st year MPharm students (30males, 24 female).
The results were marked against a standard marking proforma
Females significantly out-performed males in the PBE(P=0.0024)
Results - 3
The exercise was repeated again this year among28 male and 39 female students.
Males :
Females :
Questions / Prompts
Questions may provide ‘answers’which could compromise summative assessments
Summary These results indicate that a dispensing CBE appears to
‘work’ as a teaching tool There does not appear to be equivalence between
students performance between the PBE and the CBE There is currently no intention to use the CBEs
summatively There appears to be a trend, across the studies, of
female students performing worse at CBEs than PBEs Caution may be warranted moving to ‘digital teaching /
assessment’ to ensure that sub-groups of students are not disadvantaged
SummaryThere could be a range of factors involved :
Cultural attitudes to computersAccess to computersFamiliarity with computersLinguistic / comprehension issuesPoor organisation of material on WOLFLack of ‘open book’ access to informationAwareness of being assessed electronically