© courseware interactive ltd., 2003 presentation may 22nd 2003 designing effective multiple choice...
TRANSCRIPT
© Courseware Interactive Ltd., 2003
PresentationMay 22nd 2003
Designing Effective Multiple Choice Questions
ILTA,Waterford
Niall Watts
Courseware Interactive Ltd
© Courseware Interactive Ltd., 2003
Evaluation - content & teaching
Retention
Reinforcement
Diagnosis - feedback
Assessment
Engagement - Interaction
Why ask questions?
© Courseware Interactive Ltd., 2003
Interactivity
Students interacting with content
Make them thinkCheck their knowledgeCognitive activity
Not necessarily:FlashMenusDatabases
© Courseware Interactive Ltd., 2003
Types of Question
Choosing - Multiple Choice, Matching, Selecting
Open - Text Entry, Short Answer, Fill in the Blank, Completion
© Courseware Interactive Ltd., 2003
Choosing
Advantages Easy to Mark Usability
Disadvantages Hard to set Guessing answer Recall of factual information
Popular with teachers & designers
© Courseware Interactive Ltd., 2003
Short Answer
Advantages Easy to Set No guessing
DisadvantagesHard to Mark
Less Popular
© Courseware Interactive Ltd., 2003
Multiple Choice
Multiple-choice questions are often criticised for being easy to guess and for emphasising factual information. We contend that well-written multiple-choice questions can do a good job of assessing comprehension, problem solving and other higher order skills.
Alessi & Trollip, Multimedia for Learning 3rd Ed.,2001
© Courseware Interactive Ltd., 2003
Multiple Choice Terminology
Anatomy StemOptionsDistractors (Foils)
© Courseware Interactive Ltd., 2003
Multiple Choice Clarity
Most text in stem
Easy to ReadFocused on questionNo additional or supplementary
informationAvoid scrolling
© Courseware Interactive Ltd., 2003
Multiple Choice Clarity
Clear InstructionsNo ambiguityTell them what is expected
e.g Number of Correct OptionsNo tricks
Just checking the student’s knowledge and understanding of the course
© Courseware Interactive Ltd., 2003
Multiple Choice Clarity
AvoidNegativesConditionals
© Courseware Interactive Ltd., 2003
Multiple Choice Distractors
Hard to setThe secret of good multiple choice question
design is well-written distractorsA distractor must never suggest the correct
answer
© Courseware Interactive Ltd., 2003
Multiple Choice Issues
Plausible distractors
more
Option order
Number of options
Accidental hints
© Courseware Interactive Ltd., 2003
Distractors - Plausibility Sensible
All options should sound plausible No ‘Mickey Mouse’ unless it’s a Disney Quiz
Hint Write question as open Use popular wrong answers as distractors
© Courseware Interactive Ltd., 2003
Testing Comprehension
Avoid repeating lesson text verbatim students may recognise key words students may recall context
Paraphrase for comprehension Use different examples or case studies Ask student to apply a rule or principle
© Courseware Interactive Ltd., 2003
Higher Order Learning
Problem solvingGive several answers to a problem
AnalysisChoose between interpretations Select reasons for choice
© Courseware Interactive Ltd., 2003
Distractors – Accidental Hints
ConditionsGrammatical & Linguistic ConsistencyOption LengthRepeating TextPrecisionIntersecting OptionsOverlapping OptionsAll of the AboveNone of the Above
© Courseware Interactive Ltd., 2003
Distractors – Accidental Hints
Correct answers must always be correct Distractors must always be incorrect Negatives can be confusing Conditionals can obscure the answer or give it away
Exercise
© Courseware Interactive Ltd., 2003
Multiple Choice Distractors
Avoid ambiguityCorrect answers must always be correctDistractors must always be incorrectE.g. Word Processing Question
– Complete the following sentence correctly. A Word processing program is best defined as software used to:
– .Publish newsletters– .Produce documents– .Design graphics– .Send mailshots
– Option 2 is probably ‘the best definition’ but options 1 and 4 could also be
correct. more
© Courseware Interactive Ltd., 2003
Multiple Choice Distractors
Avoid ambiguityE.g. Word Processing Question
– The question could be better written as:
– Complete the following sentence correctly. A Word processing program is best defined as software used to:
– 1. Send e-mail messages– 2. Produce documents– 3. Produce graphics– 4. Produce accounts
more
© Courseware Interactive Ltd., 2003
Multiple Choice Distractors
Avoid ambiguityAvoid Negatives. They can confuse a student.
E.g.– Which of the following is not a fraction?
– 0.5– 1/2
• This could be better written as:– Which of the following is a fraction?
– 0.5– 1/2
more
© Courseware Interactive Ltd., 2003
Multiple Choice Distractors
Avoid ambiguity
Negatives in true/false questions can be particularly confusing. E.g.– London is not the capital of England. True or False.
• In particular avoid negatives in both the stem and the options
more
© Courseware Interactive Ltd., 2003
Multiple Choice Distractors
Avoid Ambiguity - Conditions
Avoid conditions such as ‘often’, ‘sometimes’,’usually’,’always’,’never’ or ‘all’.
Avoid the use of the verb ‘may’. Conditions may make the answer unclear or give it away altogether.
True or False. Metals are usually solids at room temperature.
The word usually should be avoided as it hints that the correct answer is true.
True or False. All Metals are solids at room temperature.
The word all should be avoided as it hints that the correct answer is false.
© Courseware Interactive Ltd., 2003
Multiple Choice Distractors
Grammatical & Linguistic Consistency
Sentence structure reveals answer where language flows from stem to correct answer. Example:Second option below
more
Which option completes the following sentence? In 1492 Christopher Columbus…
is the discoverer of North Americalanded in the New Worldis buried in Seville cathedralwas beaten by the Vikings by 500 years
© Courseware Interactive Ltd., 2003
Multiple Choice Distractors
Length of Options
Where one answer is much longer than the others it may well be correct
more
Which of the following best defines a bird?
A feathered animalA warm-blooded animal with feathersA warm-blooded animal with feathers that lays eggs
© Courseware Interactive Ltd., 2003
Multiple Choice Distractors
Repeating Text Where text in the stem is repeated in one of the options
that option is likely to be correct. Example
more
Why was the Financial Services Authority founded?
To prevent money launderingTo regulate financial servicesTo provide consumer informationTo control pension sales
© Courseware Interactive Ltd., 2003
Multiple Choice Distractors
Precision Where one option is more precise it hints that it may be
correct
more
The height of Ben Nevis is:
1000 metres1343 metres1500 metres4000 metres
© Courseware Interactive Ltd., 2003
Multiple Choice Distractors
Intersecting Options An option covering more than one point is more likely to be
correct. Here, option 2 covers both eating & time.
more
Medication to lower blood pressure should be taken once a day. Should it be taken?
At a mealtimeBefore breakfastIn the morning
© Courseware Interactive Ltd., 2003
Multiple Choice Distractors
Overlapping Options Be careful that you do not choose overlapping distractors. At
first glance they may seem correct but potentially there are multiple correct answers
more
Due to cutbacks the budget was reduced to less than:
£10,000 £8,000 £5,000
Suppose the budget was reduced to £4,500!
© Courseware Interactive Ltd., 2003
Multiple Choice Distractors
All of the Above Avoid ‘all of the above’
If one distractor is wrong it cannot be the answer If two options are correct it must be the answer (1 correct only)
more
© Courseware Interactive Ltd., 2003
Multiple Choice Distractors
None of the Above ‘None of the above’ can be used safely provided
Sometimes it is the correct answer Provided ‘the best answer’ is not required
more
© Courseware Interactive Ltd., 2003
Multiple Choice Distractors
Ambiguity Negatives Conditions Grammatical & Linguistic Consistency Option Length Repeating Text Precision Intersecting Options Overlapping Options All of the Above None of the Above
© Courseware Interactive Ltd., 2003
Multiple Choice Options
How many options?
4 is popularHow many correct?Higher number of options reduces guessing riskThree options may be appropriate e.g.
– At constant temperature with increasing pressure the volume of a fixed mass of gas
– 1. Decreases– 2. Increases
– 3. Stays the Same
© Courseware Interactive Ltd., 2003
Multiple Choice Options
Option Order
3rd position correct is popular1st is unpopular!Position choice:
Correct answer must be randomly positioned Or options in order e.g alphabetical
Position must NOT suggest a correct choice
© Courseware Interactive Ltd., 2003
True/False Questions
Multiple Choice with 2 options 50% chance of correct guess Easy to Mark Hard to Set Can be disguised e.g
select an object on a diagram
© Courseware Interactive Ltd., 2003
CourseBuilder
Used to create learning interactions Extension for Dreamweaver Free Register with Macromedia Download from www.macromedia.com/ Follow instructions on screen
© Courseware Interactive Ltd., 2003
CourseBuilder
Make a multiple choice question
Insert - CourseBuilder Interaction
Multi Choice Radios with Defaults
Red button to edit
© Courseware Interactive Ltd., 2003
CourseBuilder
Customising multiple choice question
Multi Choice Radios with Set Text of Layer
Multi Choice Checkboxes with Layers
© Courseware Interactive Ltd., 2003
ResourcesInstructional Design
IFETS - ifets.ieee.orgITFORUM - University of GeorgiaERICAssessment – Question Mark
Web developmentMacromediaCoursebuilder ListServRapidIntake – www.rapidintake.comJakob Nielsen Usability - www.useit.com
© Courseware Interactive Ltd., 2003
Contact Niall Watts Courseware Interactive Ltd
[email protected] www.courseware.ie (353-1) 295 2484