Dr. Yuyuan Zhao

Department of Engineering

University of Liverpool

A Scheme for Converting MCQ TestScores to Percentage Marks

Why MCQ?

Easy to set up (?) Easy to Mark Quick to obtain results Suitable for selection processes Suitable for large classes Suitable for assessing knowledge,

aptitude, competence and many skills Especially suitable for fact based

subjects

Why is MCQ not widely used?

Involves guesswork Not always suitable for assessment of certain

skills, e.g.: describing phenomena solving problems writing essays analysing data

performing calculations Not suitable for big problems Lack of databank of questions Raw scores not compatible with conventional

marking scheme

A Good MCQ Test

Assesses most learning outcomes Has 4 choices of answer to each question Has equally feasible choices to a laymen

for each question Has 20 or more questions

the probability of obtaining a mark >40 by pure guesswork in a test consisting of 18 four-choice questions is <1%.

Raw scores are converted to marks that are a true measure of the students’ performance.

The Conversion Scheme

The scheme is developed based on probabilistic analysis

For each multiple-choice question,

A full score is awarded if the choice of answer is correct A full mark is only awarded if the choice is NOT made by

guesswork A partial mark is awarded if a student knows any of the

wrong answers (firm answers) No mark is awarded for pure guesswork

The scheme removes the guesswork element

(YY Zhao, Int. J. Eng. Edu., 21(2005), 1189-1194)

Types of 4-Choice Questions

Number of firm

answers Type

Correct

answer

Wrong

answers

Score Mark Frequency of

appearance

A 1 Any 1 1 f

B 0 3 1 3/4 f3(1-f)

C 0 2 1/2 1/2 3f2(1-f)2

D 0 1 1/3 1/4 3f(1-f)3

E 0 0 1/4 0 (1-f)4

Score vs Mark

5

4920

5120

43725

4125

325333

31333

210050

5

4

3

NffM

ffS

NffM

ffS

NffM

ffS

NSSM

.

.

Each score S corresponds to a mark M.

f – fraction of firm answers

N – number of choices

Conversion Table

No of choices

Score 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

2 0 0 0 0 0 0 0 8 15 23 31 40 48 58 68 79 100 3 0 0 0 3 11 19 26 33 41 48 54 61 68 74 81 89 100 4 0 0 9 17 25 32 39 46 53 59 65 71 76 81 87 92 100 5

Mark

0 9 17 25 33 40 47 53 59 65 71 76 81 85 90 94 100 For a full version of the conversion table see:

Y Zhao, Int. J. Eng. Edu., 21(2005), 1189-1194

Implementation Using EXCEL

Enter conversion table in columns A and B

Enter student names and their raw scores in columns C and D

Enter a formula using VLOOKUP function in column E

Merits of the Scheme

Compatible with standard percentage marking scheme

Independent of class size and historical data

MCQ can be used in conjunction with other assessment methods

Questions with different weightings acceptable

Guesswork allowed but not rewarded

Reliability of Scheme

0

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100

MC

Q M

od

ule

Mar

k

Semester Average

20% within 2 marks 50% within 5 marks 70% within 10 marks

The marks of a module assessed entirely by MCQ tests correlated very well with the average marks of all the modules in that semester for a class of ~170 students

Recommendations

For summative assessment each question should have 4 or 5 choices.

For formative assessment, 2 or 3-choice questions are acceptable.

The number of questions in a MCQ test should be at least 10.

Scores of individual assessment units should be converted and then averaged to give the overall mark.

Summary

A scheme has been developed for converting raw MCQ scores to percentage marks by removing the guesswork element.

The converted marks are a true measure of the students’ performance.

The converted marks are compatible with the conventional marking scheme.

The conversion scheme is independent of class size and historical data.

The conversion scheme is easy to implement in a spreadsheet programme such as MS Excel.

The conversion scheme has been proved to be reliable.