an introduction to computer- assisted assessment joanna bull and ian hesketh caa centre teaching and...
TRANSCRIPT
An Introduction to Computer-assisted Assessment
Joanna Bull and Ian HeskethCAA Centre
Teaching and Learning Directorate
Overview
What is CAA?
Why use it?
Objective testsIntegration: assessment
methods & strategies
Hands-on session
Computer-assisted Assessment I
Computer-assisted assessment (CAA) is a common term for the use
of computers inthe assessment of student learning.
Computer-assisted assessment II
Deliver, mark and analyse assignments or examinations (computer or web-
based) Collate and analyse data gathered from optical mark readers (OMR) Record, analyse and report on achievement Collate, analyse and transfer assessment
information through networks
Assessment
Formativeprimarily to provide feedback to students
and promote student learning Summative
contributes to final grade for course/module
Diagnosticto determine ability and understanding
Self-assessmentto allow students to evaluate their learning
Why use CAA?
What are the benefits to using CAA?
Why use CAA ?
Increasing the frequency of assessment
Increasing feedback
Widening the range of assessment methods
Provide objectiveand consistent marking
Decrease marking
Administrative efficiency
Increasing the frequency of assessment
Motivating students to learn
Encouraging students to
practice skills
Broadening the knowledge
assessed
Increasing feedback
Students to motivate students needs to be timely, accurate and
constructive
Staff about student performance about test and question performance
Widen range of assessment methods
Different assessment methods for different skills, abilities and knowledge
Balance of assessment methods
Utilisation of multimedia to create innovative assessments
Provide objective and consistent assessment
Assessment can be unreliable, subjective and inconsistent Automated marking enables consistent, objective assessment for large groups Rigorous evaluation of questions ensures overall consistency and standards.
Decrease marking
Potential time savings:
dependent on level of implementation
must be offset against investment in design
may represent a re-distribution of workload
may demand a cultural shift
Administrative efficiency
Automatic transfer of data to MIS, or
SRS
Examination papers and handling
Speed of results
Objective Tests
Examples of question types
Guidelines for writing items
Assessing skills and abilities
Feedback and scoring
Objective Tests
Require students to provide a response to a question with a pre-determined answer:
select a solution from a set of choices identify an object or position supply brief numeric or text responses
Question Types
Multiple choice Multiple response Assertion-reason Graphical hotspot Text/numerical Matching/ranking Sore finger questions Sequencing Assembly Graphing
Designing questions I
Questions single, definitive statement avoid unnecessary or irrelevant
material use clear language use negatives sparingly put as much of the question in the
stem do not use double negatives
Designing questions II
Distractors ensure there is only one correct answer use plausible distractors avoid clues to the correct answer avoid ‘all of the above’ and ‘none of the
above’ base distractors on common student errors avoid ambiguity
Blooms Taxonomy
knowledge, comprehension, application, analysis, synthesis and evaluation
Example questions
Feedback
Objective tests allow provision of :timely, specific formative feedback to
studentsfast and detailed feedback to staff
Enables a consistent level of feedback to large student groups Can focus and direct learning through
anticipated problem areas Enables curriculum design modification during a course
Scoring
One score per correct
answer
Issue of guessing
Negative marking
Discrimination and facility
Discrimination power of an individual question to
differentiate between weaker and better students
Facility measures the difficulty of a question by dividing the number of correct responses by the total number of responses.
Integration: assessment methods & strategies
Appropriateness and balance
Quality of feedback and student learning
Extending the skills and knowledge assessed
Test structure II
Analyse materialcontentdifficultyassessment typequestion style
Identify areas of low coverage and uneven spread
CAA in business studies
Introduction to financial accounting Intermediate/advanced financial
accounting Principles of finance Production and operations management Professional certificate in management
Advantages of CAA-business studies
respondents
Speed Remote access Anonymity Neutrality Quick feedback Wide course coverage
Disadvantages of CAA-business studies
respondents
Inflation of students' marks
Difficult to set appropriate questions
Steep learning curve/time investment
Fails to develop students' argumentation skills
