rules for writing test items. multiple choice items most popular, most flexible, most widely used...

Rules for Writing Test ItemsRules for Writing Test Items

Multiple Choice Items

• Most popular, most flexible, most widely used• Used to assess:

- knowledge- comprehension of text- simple applications of principles and formulas- recognition of examples of concepts- recognition of reasonable analyses or

evaluations of stimulus materials• Depending on how written, it can require mere recall

or careful thinking• Can assess students’ ability to distinguish between

good and poor examples of concepts, interpretations, inferences, or principles

Multiple Choice Items - Rule #1:

Make sure the stem poses a problem complete enoughthat the student knows what is expected before readingthe answer choices.

IncorrectA topic sentence

CorrectWhat is the function of a topic

sentence?a. always comes first in aparagraph

b. must have supportingsentences

c. is not essential for allwriting

a. to tell what the paragraph isabout

b. to summarize information in the text

c. To give details about the topic


Avoid using qualifying words in the stem like none,every, never, always (signals incorrect response) orusually, some, each, best, generally (signals correctresponse.)

Volcanoes erupt quickly buta. People in their path can never escape.

b. People in their path can always escape.

c. People in their path can sometimes escape.


To assess recall of facts, literal comprehension, or simple applications of algorithms or rules, it is usuallybest to ask questions that begin with which, who, what,where, or when.

For example:• When did the event occur?• Where did the story take place?• Who had the best motive?


To assess thinking, use questions that begin with why,how, or which.

For example:• Why did the main character pick a fight with his father?• How does the addition of a second control variable improve the

investigation?• If the trends continue as they are in the graph, which of the following

are likely to occur in 10 years?[Of course this assumes that the teacher did not teach this to students sothat it is just a recall on the part of students.]


Make certain that grammar in the stem does not giveclues to the answer.

IncorrectThe part of flower that holds

the eggs is called ana. stamenb. pistilc. ovary

CorrectThe part of the flower that holds the

eggs is calleda. a stamenb. a pistilc. an ovary


Answer choices should all be viable, reasonablechoices.

Avoid this typical pattern: a. correct answer b. “almost” correct answer c. incorrect answer d. obviously incorrect answer


Answer choices should be similar in length, grammatical syntax, and conceptual complexity.IncorrectWhat did President Roosevelt doto bring America out of the Depression?

a. Spend federal money on

work programsb. A warc. Subcontracting federal projects to private business

CorrectWhat did President Roosevelt do toBring America out of the Depression?a. He spent federal money on work programs

b. He started a war with Germany and Japanc. He subcontracted federal projects to private business


Make certain there are no clues to the correct responsegiven in the stem.IncorrectTom and his 3 friends went to

the movies. Tickets cost $8.50 times each person. Which expression shows how much money they needed?a. $8.50 x 4 b. $8.50 x 3c. $8.50 + 4d. $8.50 + 3

CorrectTom and his 3 friends went to the

movies. Tickets cost $8.50 per person. Which expression shows how much money they needed?

a. $8.50 x 4 b. $8.50 x 3c. $8.50 + 4d. $8.50 + 3


Avoid answer choices that are designed to trick students into responding incorrectly.

IncorrectThe team who discovered the

double-helical structure of DNA wasa. Watson and Cruck b. Watson and Crack c. Watson and Crick d. Watson and Crock

CorrectWatson and Crick discovered:a. that genes cause cystic fibrosisb. the double-helical structure of

DNAc. that genes can have natural

mutationsd. the function of messenger DNA

Incorrect: The boy had been hiding for so long that he had a ______ look.a. frightened c. hungryb. haunted d. worried


Do not create distractors that are partially correct or overlap with the correct answer.IncorrectFrom the farthest to nearest,

what is the order of the following objects in the night sky?a. galaxies, stars, planets b. galaxies, stars, the moonc. stars, the sun, galaxiesd. planets, stars, the moon

CorrectWhich of the following objects is

closest to Earth?a. a galaxy other than the Milky

Wayb. the sun in Earth’s solar systemc. the closest planet in Earth’s

solar systemd. the Earth’s moon


Do not use the following if students need to locate theBest answer:• “all of the above”• “a and b”• or similar answer choices

[“None of the above” can be used if the question is factual or requiresmath computation.]

Multiple Choice Items

Rule #12 - Randomly use each alternative position as the correct answer.

Rule #13 – Use four to five options to minimizeguessing.

Rule #14 – Avoid negatively state items.

Short Answer/Completion Items

• Are useful in measuring:- knowledge- comprehension and simple analysis of text,

graphics, and other stimulus materials- applications of knowledge, concepts,

principles, and equations- simple explanations

• When writing short answers, teachers must have the expected responses clearly in mind before they write the items

Short Answer Items - Rule #1:

Make certain the question or prompt is simple andClearly indicates what you want to know.

IncorrectWhat is the digestive system?

Correct1. Name 3 organs in the digestive

system.2. Describe 2 functions of the

digestive system.

Short Answer/Completion Items -

Rule #2: Provide sufficient answer space

Rule #3: Make all blanks equal length

Rule #4: Avoid using statements directly from thetextbook

Rule #5: Use at most two blanks for one item.

Rule #6: Use a direct question rather than an incomplete statement

Matching Items

• Are useful in measuring:- whether a student can assess related knowledge

e.g. paired information such as:states and capitalskey events and datesvocabulary words and their definitions

• Are not useful in measuring deep understanding

Matching Items - Rule #1:

Rule #1: Avoid long lists to match since these willconfuse students

Make certain the number of items is appropriate for the ageof the students- For grades below 5 use 3-4 items- For grades 5 and up use 8-10 items- For grades 5 and above, provide more options than there

are blanks so that students can’t use the process of elimination

Matching Items - Rule #2:

Make certain that the concepts you test are verysimilar in type. (Place definitions, examples, or problems on the left and the terms, concepts, orsolutions on the right.)

Incorrect__St. Louis A.The capital of Mo.__Jefferson City B. Major commerce

center in Mo.__Harry Truman C. Author of stories

about Mississippi River

__Mark Twain D.Born in Independence, Mo.

Correct__Capital City of A. St. Louis Mo.__Home of famous B. Jefferson City stockyards__Major commerce C. Kansas City center on Mississippi D. Independence__City from which wagon trains E. Columbia headed west

Matching Items -

Rule #3: List options logically, alphabetically, or numerically

Rule #4: Scatter the correct and incorrect options inthe list

Rule #5: Place all items and options on one page

Rule #6: Describe the basis for matching and the number of times an option may be usedto create a match

True False Items

• Are useful in measuring:- knowledge of concepts or facts- the recognition of the correct application of

principles or rules

• Can be used with graphics and passages• The items should measure important objectives

True-False Items - Rule #1:

Do not combine true and false information in asingle item

IncorrectT/F ___ Houston is one of the

locations of NASA

and is the capital of

Texas

CorrectT/F ___ Houston is one of the

locations of NASA T/F ___ Houston is the capital of

Texas


Make certain that items test factual statements;do not put value judgments into T/F items

IncorrectT/F ___ French should be

spoken by all citizens of Louisiana

CorrectT/F ___ Many of the authors

we read believe that French

should be a state recognized language in Louisiana


Avoid terms that will provide clues to the T/F items like qualifying words such as: none, every, never,always (signals incorrect response) or usually, some,each, best, generally (signals correct response)

IncorrectVolcanoes always erupt so quickly that people in their path can’t escape.

Volcanoes never erupt so quickly that people in their path can’t escape.

CorrectVolcanoes sometimes erupt so quickly that people in their path can’t

escape.

True-False Items -

Rule #4: Do not lift sentences directly from the text. Students with good memories may answer this question without really understanding the concept!

Rule #5: Items should be state positively; never use double negatives.

Rule #6: Use an even mix of true and false statements.

Rule #7: Keep true and false items about the same length.

Performance/Complex Objectives Items

• Are useful in measuring:- demonstrations of performance skills- performance objectives- real world tasks- higher order thinking skills

Performance Items - Rule #1:

The item must measure targeted learning objectivese.g. Students are able to represent mathematical information in a variety of forms including

symbols, tables, graphs, and pictures

IncorrectName and describe three ways to represent data.

CorrectUsing the information below, create a graph that will help the Warrensestimate and compare the costs to rent a van from each rental company.


The question or prompts should be clear and conciseso students know exactly what they are being asked todo.

IncorrectHow were Woodrow Wilson and Harry Truman the same or different?

CorrectWrite an essay comparing the administrations of Woodrow Wilson andHarry Truman in terms of their approaches to war in Europe. Include:

- how their foreign policies were the same or different- how their approaches to American involvement were the same or

differentGive at least two examples from the readings to support your ideas.


Avoid writing performance items that give a series ofquestions or too many directions.

IncorrectRead the excerpt from Hamlet. What is Hamlet thinking about at thispoint? What are his concerns? What does he want to change? What arehis fears?

CorrectRead the excerpt from Hamlet. Write an essay describing how this excerpt relates to one of Hamlet’s conflicts in the play. Be sure to include:

- your ideas about the conflict- the feelings and thoughts Hamlet shows in the excerpt- how his feelings and thoughts relate to the conflict

Performance Items -

Rule #4: Clearly state any criteria that will be used for scoring responses and specify the rubric or

checklist that will be used to judge performance.

Rule #5: Whenever possible, use generic scales or rubrics to evaluate work students are likely to repeat

over time.

Rule #6: Construct items in a form different from that in which the material was originally presented.

Performance Items -

Rule #7: Have students perform tasks such as:- identify assumptions- analyze criteria.- discover relationships among similar topics- project what might have happened if …- use analogies to measure relationships

Essays – some guidelines

• Avoid using optional questions.• Prepare a trial response to the question.• Construct a tentative scoring key in advance.• Score all responses to a single question before

scoring the next question.• Tell students what the rubric is in advance.• Score responses anonymously.

rules for writing test items. multiple choice items most popular, most flexible, most widely used...

Documents

readingthe answer choices

federal projects

federal money

students ability

correct responsegiven

mere recall

recall of facts

incorrect response