Writing Good Multiple Choice Writing Good Multiple Choice Assessment ItemsAssessment Items

Amy CallipareAmy Callipare

Formative Assessment SpecialistFormative Assessment Specialist

Evaluation and ResearchEvaluation and Research

March 5, 2009March 5, 2009

• Relevant Terminology

• Item Writing Guidelines

• Weak items revised to better items

• Checklist for guidelines

Outline

Parts of a Multiple Choice Item

• Stem: the question being asked

• Foils: the answer choices

• Distracters: the incorrect answer choices

• Key: the correct answer

Parts of a Multiple Choice Item

What word best describes the main character?

A. friendly

B. scared

C. angry

*D. proud

Stem

Foils

Key

Distracters

Base questions on the North Carolina Standard Course of Study

• Refer to the goals and objectives for the appropriate

grade level and course.

• Emphasize higher-level thinking skills using the

taxonomy provided by NCDPI.

Base questions on the North Carolina Standard Course of Study

Weak Item:What is the name of the maincharacter?

• NCSCOS Grade 3• Goal 2: The learner will apply

strategies and skills to comprehend text that is read, heard, and viewed.

• Objective 2.04: Identify and interpret elements of fiction and nonfiction and support by referencing the text to determine: the author’s purpose, plot, conflict, sequence, resolution, lesson and/or message, main idea and supporting details, cause and effect, fact and opinion, point of view (author and character), and author’s use of figurative language (e.g., simile, metaphor, imagery).

• Thinking Skill: Remembering

Better:How does the main characterchange from the beginning ofthe selection to the end?

NCSCOS Grade 3 Goal 3: The learner will make

connections through the use of oral language, written language, and media and technology

Objective: 3.02 – Identify and discuss similarities and differences in events, characters, concepts, and ideas within and across selections and support them by referencing the text

Thinking Skill: Analyzing

Each item should measure a single concept, principle, idea, procedure or competency.

Weak Item:Which sentence is writtencorrectly?

A. Mary said, “Lets go outside to play”.

B. That womans umbrella is blowing away in the wind.

C. We are taking a field trip to see the North Carolina Museum of Science.

D. Jane is short Wendy is tallest.

Better:Which sentence shows proper

use of quotation marks.A. Mary said, “Let’s go

outside to play”.B. The woman yelled,

“Catch my umbrella!”C. “Mrs. Jones announced,”

Our class will be taking a field trip on Friday to the museum.

D. Jane told her parents, “I will clean my room after I finish my homework.

Ask questions that measure significant details, not trivial information.

Weak Item:

In the selection, what color jacket was themain character wearing?

Better:

What was the main character’s reason forwearing the red jacket to the party?

Keep vocabulary consistent with grade level of students, unless testing word meaning.

Weak Item:What caused the main character to be malevolenttoward his friends?

Better:What caused the main characterto act in an evil manner towardhis friends?

Acceptable use of above-grade level vocabulary:

According to paragraph 8, what is the meaning of demolish?

When asking the meaning of a word from a selection, refer to the location in the selection to help students located quicker.

When asking the meaning of a word from a selection, the foils should appear as the same tense of the word as it appears in the selection (if it is past tense, foils should be past tense; if it is in noun form, foils should be noun forms).

Use question format.Weak Item: The main charactercould be described as ..

A. friendlyB. scaredC. angryD. proud

Better:What word best describes the main character?

A. friendlyB. scaredC. angryD. proud

Weak Item:Between 1950 and 1965:

A. Interest rates increased.B. Interest rates decreased.C. Interest rates fluctuated greatly.D. Interest rates did not change.

Better:What was the trend in interest ratesbetween 1950 and 1965?

A. increased onlyB. decreased onlyC. increased, then decreasedD. remained unchanged

Avoid a complex item format.

Weak Item:

Which lines are parallel?1. lines P and Q

2. lines R and S

3. lines S and P

A. 1 and 2

B. 2 and 3

C. 1 and 3

D. 1, 2, and 3

Better Item:

P

Q

R S

A. P and Q

B. R and S

C. Q and R

D. S and P

Be brief and clear. Eliminate any extraneous information.

Weak Item:

Mary, Elizabeth, and Jane are sisters. Mary is the oldest ofthe girls. Elizabeth has 12pieces of candy. If Elizabethshares her candy so they allhave the same number ofpieces, how many pieces will each girl have?

Better:

If Elizabeth has 12 pieces ofcandy and wants to sharethem so that her two sistershave the same number ofpieces as she does, howMany pieces of candy willeach girl have?

Keep wording positive. Avoid using negative phrasing in the stem.

Weak Item:All but one of the followingis not an element. Whichone is not?

A. carbon

B. salt

C. sugar

D. plastic

Better:

Which is an element?

A. carbon

B. salt

C. sugar

D. plastic

Common words should be incorporated into the stem instead of the foils.

Weak Item:

Which statement is best

supported by the selection?

A. All the animals were lonely.

B. All the animals enjoyed sleeping in the grass.

C. All the animals wanted to have a caring owner.

D. All the animals must live in the forest.

Better:Which statement about all theanimals is best supported by theselection?

A. They were lonely.B. They enjoyed sleeping in

the grass. C. They wanted to have a

caring owner.D. They must live in the

forest.

Keep foils parallel.

Weak Example:What was the first activity on a

cold wintry day?A. to gather woodB. eating a hearty breakfastC. go for fresh waterD. get a fire started

Better:What was the first activity on a cold wintry day?

A. gathering woodB. eating hearty breakfastC. finding fresh waterD. starting the fire

Weak Item:Which sentence best summarizes theselection?

A The day ends and she begins to sing.

B She explores the pond and avoids danger.

C She watches mist on the pond.

D She enjoys playing in the pond.

Better:Which sentence best summarizethe selection?

A She sings by the pond at the end of the day.

B She explores the pond and avoids danger.

C She watches mist on the pond.

D She enjoys playing in the pond.

Order foils in a logical order.

Numbers/Dates should be listed ascending or descending order Quotations from a reading selection should be listed in the order they

appear in the selection

Weak Item:

If John and Mary have 18candy bars and they wantto split them equally, howmany candy bars shouldeach have?

A. 18B. 6C. 11D. 9

Weak Item:

If John and Mary have 18 candy bars and they want to split them equally, how many candy bars should each have?

A. 6B. 9C. 11D. 18

Be sure there is one and only one correct answer.

Weak Item:How long does an annualPlant generally live?

A. It dies after the first year.

B. It lives for many years.C. It lives for more than

one year.D. It needs to be replanted

each year.

Better:How long does an annual plantgenerally live?

A. three weeksB. two monthsC. one year onlyD. several years

• Foils should be independent and should not overlap

Distracters should be plausible.

Weak Item:

Which person is well

known for flying solo?

A. Clara Barton

B. Easter Bunny

C. Fred Flintstone

D. Amelia Earhart

Better:

Which person is well

known for flying solo?

A. Clara Barton

B. Annie Oakley

C. Eleanor Roosevelt

D. Amelia Earhart

Make sure foils are similar in content and length.

Weak item:What is one major causeof air pollution in many

A water B chemical substances such as

nitrogen oxides, carbon monoxides, and organic compounds that can evaporate and enter the atmosphere

C Sun heating homesD wind powering windmills

Better:What is one major causeof air pollution in many

A water flowing over rocks in the rivers

B chemical substances entering the atmosphere

C homes being heated through solar energy

D wind powering windmills on dairy farms

Avoid clues to the right answer – either within or across items.

1. How does the main characterfeel at the beginning of theselection?

A. scared

B. excited

C. angry

D. proud

2. Why is the main character angry at the beginning of the selection?

Example:

Action:

• Choose between the items• Otherwise items measure

how clever students are at finding the clues rather than how well they have mastered the objective being measured!

Avoid humor, contractions, idioms, clichés, or other language conventions that may be

culturally biased.

Weak Example:

What’s the main idea of the selection?

Weak Example:What lesson does the main character learn by the end of theselection?

A. Get back in the saddle again.

B. One should bite their tongue to prevent trouble.

C. Avoid burning the candle at both ends.

D. Take time to stop and smell the roses.

Better:What lesson does the maincharacter learn by the end of theselection?

A. Keep trying even if you fail at first. B. Think before speaking. C. Limit the activities joined. D. Take time to enjoy nature.

Better:What is the main idea of the selection?

Avoid the use of specific determiners.(always, never, totally, absolutely, all of the above,

none of the above, etc.)

Weak Item:What advice would theauthor of the selectionmost likely give to thereader?

A. Never trust a friend.B. Think before acting.C. Spend time with

family.D. Always do what

parents say.

Better:What advice would the authorof the selection most likely giveto the reader?

A. Be cautious trusting others.

B. Think before acting.C. Spend time with family.D. Parents try to do what is

best for children.

Avoid the use of first and second person.

Weak Item:

Based on the directions of the recipe, when shouldyou add chocolatechips to the mixture?

Better:

Based on the directions of the recipe, when should the chocolate chips be added to the mixture?

List distracters on separate lines and one right after the other.

Weak Item:Who was the first president of the United States?

A. Lincoln B. Jefferson C. Kennedy D. Washington

Better Item:

Who was the first president of the United States?

A. Lincoln

B. Jefferson

C. Kennedy

D. Washington

Use correct grammar!

Weak Item:According to the third paragraph,

what is the meaning of the word

howl?

A. to make noise

B. a dog

C. late at night

D. loudly

• Check spelling, grammar, and punctuation.

• Make sure that grammar does not clue answer or help eliminate possible answers.

Better Item:According to the third paragraph,

what is the meaning of the word

howl?

A. to make a loud noise

B. to follow directions

C. to talk softly

D. to scare away

Which was created in the early 1800’s because ofthe need for a water route to help farmers shiptheir products to market?

A. Gadsden Purchase

B. Louisiana Purchase

C. Mexican Cession

D. Missouri Compromise

Specific Example

Specific Example

What impact did U.S. governmental polices have

on business and industry during the late 1800s?A. It tried to control public and private utilities and

transportation.

B. It tried to limit and regulate industrial and business growth.

C. It regulated industry for the public good.

D. It had little or no influence on business and industry.

In which section of early 19th-century America

was the plantation system an important feature?A. the New England states

B. the Middle Atlantic states

C. the South

D. the West

Specific Example

During the presidency of Andrew Jackson, what

resulted from the spoils system?A. federal laws being nullified by the states

B. elected officials rewarding their supporters with government jobs

C. free males being given the right to vote

D. the end of political corruption in the federal

government

Specific Example

Which statement shows a policy most similar to the foreign policy advice given

by President George Washington in his Farewell Address?

A. Steer clear of permanent alliances with any portion of the foreign

world.

B. The United States will give economic aid to needy countries

anywhere in the world, but will not provide military aid.

C. The United States must prevent the growth of communism.

D. The United States can take over other countries to help them become

more like us.

Specific Example

In addition to Puerto Rico, which territories were

acquired by the United States as part of the peace treaty

that ended the Spanish-American War?

A. Cuba and the Philippines

B. Cuba and Guam

C. the Philippines and Guam

D. the Philippines and Hawaii

Specific Example

What was Alexander Hamilton suggesting when he stated, "Allcommunities divide themselves into the few and the many.The first are the rich and well born, the other the mass of thepeople. . . . The people are turbulent and changing. . . . Givetherefore to the first class a distinct permanent share in thegovernment. They will check the unsteadiness of the second.“ ?

A. The will of the majority should guide public policy.B. Wealthy people are too preoccupied to rule well.C. The common people cannot be trusted to run a stable

government.D. Poorer people must work harder to gain access to

economic and political power.

Specific Example

The Mayflower Compact is important to the concept of a

democratic society because it represents

A. an effort by the colonists to use force to resist the King

B. a clear step toward self-government

C. an early attempt to establish universal suffrage

D. an attempt by the colonists to establish freedom of religion

Specific Example

What did many U.S. congressmen fear aboutFranklin D. Roosevelt’s Quarantine Speech

(1937)?A. It would lead to United States involvement in

World War II.B. It would interfere with U.S. trade relations with

China.C. It would cause the United States to become more

isolationist.D. It would increase immigration problems in the

U.S.

Specific Example

Which statement best explains why Reconstruction

ended?

A. Reconstruction policies were no longer needed when the Southern states rejoined the Union.

B. African Americans prospered financially.

C. Reconstruction was intended to be a short-term event that would end in 10 years.

D. Enforcement of Reconstruction Acts decreased because of political compromise.

Specific Example

References

Burton, S.J., Sudweeks, R. R. , Merrill, P.F., & Wood, B. (1991). How to Prepare Better Multiple-Choice Test Items: Guidelines for University Faculty. Brigham Young University and Testing Services/The Department of Instructional Science.

Department of Pubic Instruction. (2004). Guidelines for Writing Multiple-Choice Test Items [Brochure]. Raleigh, NC: Office of Accountability and Technology Services.

Salkind, N.J. (2006). Tests & Measurement for People Who (Think They) Hate Tests & Measurement. Thousand Oaks, CA: Sage Publications.

Contact Information:

Amy Callipare

Formative Assessment Specialist

Evaluation and Research

acallipare@wcpss.net

431-8245