exploring formativee assessment in leicester medical school

ExploringFormative E-Assessment in Leicester Medical School

Terese Bird PedR Group 4th April 2016

Context• Summative examinations in Medicine• Every semester• Integrated• One Single Best Answer (SBA)• One Short Answer Question (SAQ)

• Previous semesters’ formatives• Document of questions distributed on Blackboard• Face-to-face discussion

Groupwork sessions• Students divided into groups• Workbooks with questions• Answers given the week after or never• Searching for the answer is the point• Facilitated by junior doctors

Problems• NSS feedback scores• Students don’t know where they are• Tutors don’t know where students are

Student study issues• Lecture notes + workbook “answers” = holy canon of notes on iPad• Holdover from coursework• Memorise the form of words for SAQs• Writing notes is good but can’t write it all• Screenshotting is enough = having it in my notes is enough = having the right

answer is enough

• “Revision” happens in the weeks before exams• Not much day by day learning• Not much verbalising or writing of what they know• And if they did, how would we give feedback?

Laurillard’s Conversational Framework

Laurillard’s Conversational Framework (Laurillard, 2008) Dashed line box added by this author to show segment relevant to e-assessment of this study

ExamSoft £30 / student / year

Examsoft formatives GI Unit Autumn 2015

Exam Type When Content Circumstance

Session-by-session exams

Weekly, from Oct through Dec 2015

Material studied in the past week

Exam done in groupwork class sessions, not under exam conditions

Christmas 1Christmas 2New Year 1New Year 2

During Christmas break Dec ember 2015–Jan 2016

All questions from session-by-session exams, put together and randomised

Four separate exams with the same exam content, offered 4 times throughout Christmas break for self-study whenever students wanted.

Instant answers with additional information

Students can log into portal to check progress

How did the formatives change students’ study if at all?• “A mini-revision before the proper revision” – forced earlier study• Uncovered trouble spots which caused them to seek solutions• Back-and-forth communication• Active study

Thinking and learning on higher levels of Bloom’s Taxonomy• If questions were written higher on Bloom’s, students learnt higher on

Bloom’s – constructive alignment • Single Best Answer leads to Evaluation and Analysis• Students reacted with Create to using exams in revision• Created document of info on uncovered trouble spot• Created answers when they tested themselves without the multiple-choice

“It’s not clear that I performed better on summatives because of these exams, as the summatives are integrated, but they helped me to learn GI better.”

“It was a nice way of doing the workbook questions in groupwork”

What next?• Cost of ExamSoft against other costs – Top Hat• ExamSoft is assessment• TopHat is interactive live learning

• Cost of time to create questions• Consider summatives – tech and staff issues