“The End Justifies the Means”

When and how to use Computerised Assessment to Promote LearningPhil Davies School of Computing

What is computerised assessment?

• Easy Solution to increased student numbers … easier for lecturers

• MCQ …. Simple to create questions• Highly Formative• Organising Summative Assessment

a formality• Minimal Costs

•RUBBISH!!!!!!RUBBISH!!!!!!

Multiple Choice Tests OLAL

• Use simplest method– Select one from three

• Provide answer … Formative?• 50 questions took about 10 minutes

– Weakest students flew through

• Reason to slow down / Learn by doing it / Formative not EXPLAINATIVE/2 Passes

• Competition / timing / “BEEP” / Negative

OLAL

Use of the System• Four Tests

– Week 3(5%) Week 6 (10%) Week 9 (15%) Week 12 (20%)

• Improve first pass to second pass (short term???)

• Positive improvement from test 1 to 4

• Weaker students benefited the most

EXAMPLE: what colour is the sky?• Student A

– give me three alternatives

• Student B– Blue

• Student C– sometimes it blue (summer), but

sometimes grey, or even red!!

What makes up the other 50%?• Skills of analysis … REAL SKILLS!!• Synthesis / Presentation / Reasoning• NEED AN ESSAY• REAL WORLD

– GIVEN CLUES?– GIVEN ALTERNATIVES?– ALLOWED TO GUESS?

ESSAYS

• Time consuming to mark• Subjective (Lecturer) .. Opinionated• Should we assess grammar

– Foreign Students– Pre-Requisite– Learning Outcomes

• FEEDBACK … WHAT FEEDBACK???

Computerised Asssessment (by Peers) or (with Plagiarism) SYSTEM

• Computerised Assessment of essays• Peer Assessed• Embedded Web Referencing• Marked Properly• Copious Comments (Destructive

(honest?))

Success / Failure

• Major benefits– style / feedback / objectivity through

subjectivity

• Problems– Wide range of marks / conflicting

comments / why bother marking

• Marks for MARKING (how?)• Long term benefit to ME?

Can it be used for final years?• CRITICAL SKILLS / REFLECTIVE• Self Assessment• Peer Assessment• Reflective Self Assessment

• Marks for Marking

Results and Feedback

• 55% within 5% of Self Assessment Mark and Peer Median Mark

• 80% within 10% of Self and Peer Median Mark

• QUALITY OF MARKING– remove second group from marking

process– take deviation into account

When should I assess?• Learning Outcomes must be assessed• Example … heart operation• Medical Student A Student B• week 6 8/10 2/10• week 12 2/10 8/10

•AT THE END … EXAMS??????– No feedback/ Mitigating Circumstances /

Limited question choice / Testing presentation and grammar skills again

“THE END JUSTIFIES THE MEANS”• Train the students to pass the

assessment of learning outcomes• One essay to meet the learning

outcomes at the end• Train them how to write essays

(peer feedback) … copy styles• Present the correct material (peer

selection)

Essay• Produce an essay that can be used as a

guide for a non-literate computing person in directing them how to build their own computer system. This computer will be used to play modern computer games, and also access the web. It must include choice of hardware / operating system / application software. Proposed costings should be included

Three Stages

• Components required to build a modern day personal computer system

• Operating System Choice• Web access and tools

• FINAL AMALGAMATION

QUALITY?• Student copy content /style / resources• Students are learning• Student work is being marked• Student feedback destructive not

constructive• Final “amalgamated” work marked by

tutor– CONVERGENT NOT DIVERGENT

EDUCATION

CONCLUSIONS• CAA not just for 1st years / not just MCQ• Not easy to set up … changes lecturer

from quantitative marker to qualitative marker

• Explainative CAA replacement for tutorials• CAA can provide Quality Assessment• Can aid “directing assessment” to meet

learning outcomes• Continual assessment may be divergent

… who cares?