writing multiple choice tests presentation to first year biology educators’ colloquim
Post on 18-Mar-2016
27 Views
Preview:
DESCRIPTION
TRANSCRIPT
Writing Multiple Choice TestsPresentation to First Year Biology
Educators’ ColloquimProfessor Jeffrey Smith
University of Otago
The Basic ApproachStep One: The test blueprint – what do you
want to test? What is the content and what is the level of thinking you want to assess?
Is the test blueprint tightly related to the course outline/objectives?
Do the students know what you expect of them?
Levels of sophistication in thinking(Bloom’s taxonomy)KnowledgeComprehensionApplicationAnalysisSynthesisEvaluation
The Basic ApproachStep Two: Logistics. How much time is available for the test?What kinds of items will best fit your
purpose?Consider marking in addition to optimal
format from a measurement perspective. (Might a multiple choice format save you a lot of time down the road that could be used in more productive activities than marking essays?)
How many items and of what type? (Rule of thumb is a minute per item for MC, but this is easy to try out.)
The Basic ApproachStep 3: Align your items with your test
blueprint. What kinds of items are you going to use
where?Are you covering all of the content that is
important or just the content where it is easy to write items? (Beware of the tyranny of the readily quantifiable.)
The Basic ApproachStep 4: (Isn’t this fun?) Now you have your
test outlined and items assigned to the various components of the test. So what is next?
Draft the items.
Writing the Multiple Choice ItemWhat do you want to know about the student
with this item? What would a great item do here?
My approach tends to focus a lot on knowledge, comprehension, and application from Bloom’s taxonomy. Especially when writing MC items.
Writing the Multiple Choice ItemWrite the stem first. End it in a question mark.Pose a clear problem.Don’t worry about the choices right now.
Read the item. Think about what you are wanting to measure from your test blueprint. How good a match do you have?
Sample knowledge level item(What does it require?)What is the space between the end of the
axon of one neuron and the beginning of the dendrite of another neuron called?
A. cortex B. corpus callosum C. glial cell D. synapse
Sample knowledge level item 2(What does it require?)Which lobe is associated with higher order
cognitive functioning, such as judgement and problem-solving?
A. frontal lobe B. parietal lobe C. temporal lobe D. occipital lobe
Sample comprehension level item (What does it require?)Which of the following best describes the transmission of information
from one neuron to another? A. The axon of one neuron is attached to the dendrites of a
number of postsynaptic neurons, which lets electrical current be passed amongst neurons.
B. An electrical impulse down the axon of one neuron causes neurotransmitters to be sent across a synaptic gap and received by dendrites of another neuron.
C. The nucleus of the first cell divides, and the information contained in the DNA of that cell passes on to the DNA of the second cell through the synaptic cleft.
D. The axon of one neuron sheds its outermost glial cell, which is received by the postsynaptic neuron through the transmitters contained at the end of the dendrites of the receiving cell.
Sample comprehension level item 2(What does it require?)Which of the following best describes a social
constructivist perspective? A. The process of learning moves from the
external to the internal. B. The process of learning is facilitated by
trying to make sense of the world. C. The process of learning involves taking in
information and effectively processing it. D. The process of learning is not linear, but
occurs sporadically over time.
Sample application level item (What does it require?)Simon has moved from the United States to New
Zealand. He plays softball well but has never played cricket. When invited to take a turn at bat, he tries to adapt the way he would swing with a softball bat to swinging a cricket bat. What would Piaget call this behaviour?
A. Elaboration B. Assimilation C. Rehearsal D. Accommodation
Sample application level item 2(What does it require?)Mrs White is working with her Year 4 children on the properties of
magnets. She asks what a magnet does and Molly answers, “It can pull things.” Mrs White responds, “Let’s test that idea.” She gets out a cotton ball, a pencil, a piece of string, and a bit of clay and tries to pull them with the magnet. When none of them work, she asks what the problem is. Molly says, “That magnet is broken.” Steve says, “Maybe it isn’t a magnet.” What is Mrs White doing with the class?
A. Trying to move the students into a formal operational way of
thinking. B. Trying to set up disequilibrium in the environment of the
students. C. Trying to encourage the students to recall information stored
in long term memory. D. Trying to get the students to adopt a hypothesis-testing
approach to science.
Drafting itemsKnowledge level
What is x called…Which of the following is associated with…Who did x…How many ….
Drafting itemsComprehension level
Which is the best description of… (Or, what does this describe?...)
What is the main advantage of…What kind of ….
(Or, according to theory x, which…)The major difference between x and y is….
Drafting itemsApplication level
Starting with the setting or vignette:Daniel is teaching his son Blake how to ride a bicycle. In the
beginning, he ran alongside Blake and helped him balance the bike. Then, he let go for a few moments to let Blake do the balancing himself. Now, Daniel is only holding on at the very beginning. What is Daniel engaged in?
Mr Chapman wants his chemistry class to learn the periodic table of elements. He wants the students to understand the basics of the molecular makeup of the elements and how they are related to one another. He realises that this is a large task for the students, so he is trying to come up with ways to help them in their learning. Which model of learning would be the most helpful here?
Right answersYour students will be nervous when they take
the test. Make sure your right answers are right and your wrong answers are wrong. Nobody likes “gotcha”!
Seriously, nobody does.Write the right answer first. Make sure it is a
clear and direct response to the question as posed, unambiguous, and correct.
Wrong answersWrong answers determine how difficult an
item is. Toby is using a new computer program to help him learn to read.
The program is set up so that whenever Toby sees a word he cannot pronounce, he can scroll over it with the mouse. The word is then pronounced out loud by the computer. What is this an example of?
A. Mediation A. Meditation B. Transfer B. Trans Alpine C. Web-based inquiry C. Moose doots D. Scaffolding D. Scaffolding
Wrong answersWrong answers need to be:
Clearly wrongReasonable given the contextAbout the same length as right answersWritten as carefully and cautiously as right answersLetter C sometimesAs far away from being right as you want the item to be
difficult
The Basic ApproachStep 5 (Remember the steps?): Review, edit, revise
You are a scholar in the general area of biology, not test construction
I, on the other hand, am a scholar in the area of test construction
I still screw this up all the timeThus, we review, edit, and revise. Get a colleague to read
your test and give you feedback. (It won’t hurt too much.)Get your partner to do so (mine happens to be a scholar in
the area of testing, so that’s kind of a bonus – sometimes)Remember that your students will be nervous and will try
to overthink this stuff.
The Basic ApproachStep 6:Go back to your test blueprint and make sure
your test measures what you want it to.
The Basic ApproachStep 7:Take your test as if you were a student. This
helps for:TimingDetermining difficultyFinal adjustments
That’s It!(If I could have come up with five more steps,
you could become recovering psychometricians!)
Thanks!
top related