multiple-choice itemspe.dadeschools.net/eoc/ngsss and assessment development...7/31/2014 4 4...
TRANSCRIPT
7/31/2014 1 7/31/2014 1
Multiple-Choice Items
1
7/31/2014 2 2
Multiple-Choice Item Terminology
Learning Objective:
To learn the terminology used to
describe multiple-choice items.
7/31/2014 3 3
Multiple-Choice Item Terminology
The following slides provide definitions
of the terms commonly used to identify
the various components of multiple-
choice items.
7/31/2014 4 4
Multiple-Choice Item Terminology
Stimulus
Reading material or graphic information that
precede the question posed and provides
information pertinent to answering the
question.
7/31/2014 5 5
Multiple-Choice Item Terminology
Look closely at the following picture.
Stimulus
What was the last step in completing this work of art?
A. Flattening the clay
B. Molding the clay
C. Painting the clay
D. Drying the clay
7/31/2014 6 6
Multiple-Choice Item Terminology
Some multiple-choice items include stimuli and
some do not.
An individual or discrete item may require a
stimulus to provide students with enough context
to answer the question.
A group of items, commonly called a cluster, or a
set may be based on a passage or graphic and
thus each item might not require an additional
stimulus.
7/31/2014 7 7
Multiple-Choice Item Terminology
Stem
The statement that poses the problem to be
solved or the question that must be
answered.
7/31/2014 8 8
Multiple-Choice Item Terminology
Look closely at the following picture.
What was the last step in completing this work of art?
A. Flattening the clay
B. Molding the clay
C. Painting the clay
D. Drying the clay
Stem
Stimulus
7/31/2014 9 9
Multiple-Choice Item Terminology
Every multiple-choice item must include a
stem.
7/31/2014 10 10
Multiple-Choice Item Terminology
Options
The choices provided from which the
student must select an answer.
7/31/2014 11 11
Multiple-Choice Item Terminology
Stimulus
Stem
Options
Look closely at the following picture.
What was the last step in completing this work of art?
A. Flattening the clay
B. Molding the clay
C. Painting the clay
D. Drying the clay
7/31/2014 12
Every multiple-choice item must include
options. However, the number of options
can vary. The Florida Race to the Top
Assessments use 3- and 4-option multiple-
choice items for the Physical Education,
Health, and Visual Arts assessments.
12
Multiple-Choice Item Terminology
7/31/2014 13
Key
The correct answer choice.
13
Multiple-Choice Item Terminology
7/31/2014 14
Look closely at the following picture.
What was the last step in completing this work of art?
A. Flattening the clay
B. Molding the clay
C. Painting the clay
D. Drying the clay
14
Multiple-Choice Item Terminology
Stimulus
Stem
Options
Key
7/31/2014 15
Every multiple-choice item must include one
and only one key.
The key must be correct and defensible as
the one and only correct answer.
15
Multiple-Choice Item Terminology
7/31/2014 16
Distractors
The incorrect answer choices.
16
Multiple-Choice Item Terminology
7/31/2014 17
Look closely at the following picture.
What was the last step in completing this work of art?
A. Flattening the clay
B. Molding the clay
C. Painting the clay
D. Drying the clay
17
Multiple-Choice Item Terminology
Key Distractors
Options
Stem
Stimulus
7/31/2014 18
The distractors should be attractive and
make sense. They should look like possible
answers to the question posed in the stem
and not be so far removed from the question
that they are ludicrous or throw-away
choices.
However, they must be incorrect.
18
Multiple-Choice Item Terminology
7/31/2014 19
Learning Objective:
To learn some of the common flaws to look
for in multiple-choice items.
19
Effective Multiple-Choice Items
7/31/2014 20
The next series of slides provide a list of
guidelines that should be considered
when writing and reviewing effective
multiple-choice items.
20
Effective Multiple-Choice Items
7/31/2014 21
The Item
21
Effective Multiple-Choice Items
7/31/2014 22
The item content should be correct,
accurate, and realistic.
22
Effective Multiple-Choice Items
7/31/2014 23
The facts, dates, documents, people and events presented in an item should be verified as correct.
Dates, spelling of people’s names, spelling of place names, the titles of documents, etc. should be confirmed as accurate.
Data and graphical information provided in an item should be realistic and not contain values that could not be found in the real world.
23
Effective Multiple-Choice Items
7/31/2014 24
The item should be aligned to the strand,
standard, and benchmark it is assessing.
24
Effective Multiple-Choice Items
7/31/2014 25
The following item is a physical education item.
25
Effective Multiple-Choice Items
7/31/2014 26
This item is correctly aligned.
Benchmark: PE.2.C.1.4: Understand the importance of
wearing a life jacket (personal flotation device) when on a
boat or near water.
What should you always wear when on a boat to stay safe?
26
Effective Multiple-Choice Items
A. A helmet
B. A life jacket*
C. A hat
7/31/2014 27
This item is incorrectly aligned. In this case the point being tested is not an inference; it is directly stated in the selection.
PE.2.L.1.4: Describe how opportunities for participation in physical activities change over the seasons.
Which after-school activity will help you stay physically active?
27
Effective Multiple-Choice Items
A. Riding your bike*
B. Doing your homework
C. Watching a movie
7/31/2014 28
The item should be fair and free from
bias.
28
Effective Multiple-Choice Items
7/31/2014 29
The Language Accessibility, Bias and
Sensitivity (LABS) can be found among your
materials.
Please take them out now.
29
Effective Multiple-Choice Items
7/31/2014 30
These guidelines are used to determine if the passages, graphics, and items intended for use on the Race to the Top Assessments contain any unintended bias.
Only passages, graphics, and items that are determined to be free of bias by a Fairness Committee can be used on the Race to the Top Assessments.
30
Effective Multiple-Choice Items
7/31/2014 31
LABS Guidelines
1. Stereotyping
2. Sensitive or Controversial Subjects
3. Advice
4. Dangerous Activities
5. Population Diversity and Ethnocentrism
6. Differential Familiarity
7. Language Accessibility
31
Effective Multiple-Choice Items
7/31/2014 32
This item has a fairness issue. Do you know what LABS Guideline it violates?
Maria and her family eat Mexican food, which can be high in unhealthy fats.
Which item is a healthy food choice for them to eat?
Effective Multiple-Choice Items
A. Soda
B. Tacos
C. Vegetables*
D. Fried plantains
32
7/31/2014 33 33
Effective Multiple-Choice Items
This item is free from bias. Do you know why?
Maria is making a meal for her family.
Which item is a healthy food choice?
A. Soda
B. Cake
C. Vegetables*
D. Potato chips
7/31/2014 34
The item should be written at a reading
level that is at or below the grade level of
the students who will be taking the
assessment.
34
Effective Multiple-Choice Items
7/31/2014 35
Items should not impose an additional
reading burden. If a reading burden is
imposed in a content area that is not
reading, you are not sure if you are testing
the student’s content knowledge or reading
ability.
35
Effective Multiple-Choice Items
7/31/2014 36
Items written at the appropriate grade level
conform to the grammar and vocabulary
demands used in the reading and subject-
area texts that students use in the
classroom.
36
Effective Multiple-Choice Items
7/31/2014 37
Texts written for the subject areas of
physical education, health, and visual arts
can be up to two grade levels lower in
readability than the grade level of the
students who are using these texts. This is
done to insure that the students understand
the concepts presented and are not
hampered by the difficulty of the language in
the texts.
37
Effective Multiple-Choice Items
7/31/2014 38
We are going to look at an item developed for a
third grade assessment.
The typical sentence structure used in third grade
texts is simple with a descriptive phrase included.
The descriptive phrase either comes before or
after the subject-verb-object in the sentence.
The number of words in the sentences in a third-
grade reading text ranges from nine to 15.
38
Effective Multiple-Choice Items
7/31/2014 39
Here is an example of typical third-grade reading material.
Giraffe
A newborn giraffe is the world’s tallest baby. When it gets up on its long legs for the first time, it stands six feet tall. This is taller than most adult humans. Within three days, a baby giraffe is able to run beside its mother. All of the giraffes in the herd take turns watching the young giraffes.
A baby giraffe uses it long neck to reach leaves and fruit at the top of tall trees. Its long neck also helps it to eat grass and bushes on the ground. A young giraffe can grow up to fifteen feet tall and can see for miles.
39
Effective Multiple-Choice Items
7/31/2014 40
This item is not written at the appropriate grade level for a third-grade assessment.
Which image demonstrates the first step in making this collage?
40
Effective Multiple-Choice Items
A. B. C.
7/31/2014 41
This is the same item rewritten at the appropriate grade level for a third-grade assessment.
Which shows the first step in making this collage?
41
Effective Multiple-Choice Items
A. B.* C.
7/31/2014 42
The item should assess significant
information.
42
Effective Multiple-Choice Items
7/31/2014 43
What is significant information and what
is an irrelevant detail is determined by
the standards and benchmarks in a
content area.
43
Effective Multiple-Choice Items
7/31/2014 44
According to the benchmark and indicator, this item assesses significant information.
Strand: Lifetime Fitness
Standard: 2. Develop and implement a personal fitness program to achieve and maintain a health-enhancing level of physical fitness.
Benchmark: PE.3.L.2.7: Engage in appropriate physical activity that results in the development of cardiorespiratory endurance.
Two days each week, Joshua likes to run laps on a track for 30 minutes. What part of Joshua’s physical fitness activity will improve the most?
A. His nutrition
B. His flexibility
C. His muscular strength
D. His aerobic endurance*
44
Effective Multiple-Choice Items
7/31/2014 45
According to the benchmark and indicator, this item assesses a detail that does not have to be assessed.
Strand: Cognitive Abilities
Standard: 1: Identify, analyze, and evaluate movement concepts, mechanical principles, safety considerations, and strategies/tactics regarding movement performance in a variety of physical activities.
Benchmark: PE.3.C.1.7: Explain how appropriate practice improves performance of movement skills.
Which training exercise consists of fast, powerful movements designed to improve performance in sports?
A. Running
B. Pilates
C. Stretching
D. Plyometrics*
45
Effective Multiple-Choice Items
7/31/2014 46
The item should not contain the
passive-voice construction.
46
Effective Multiple-Choice Items
7/31/2014 47
All of us have more difficulty processing text
written in the passive voice. Most student
texts are written in the active voice and not
the passive voice. Therefore, items that
assess a student’s knowledge of a content
area should not introduce a grammar
construction with which students are less
familiar.
47
Effective Multiple-Choice Items
7/31/2014 48
Here is an example of a sentence written
in the passive voice.
A game of chess was played by John and
Mary.
Here is the same sentence written in the
active voice.
John and Mary played a game of chess.
48
Effective Multiple-Choice Items
7/31/2014 49 49
Effective Multiple-Choice Items
This item is written in the passive voice.
The spread of lice can be promoted by performing which action?
A. Putting conditioner in your hair.
B. Washing your hair every day.
C. Using a comb in your hair.
D. Sharing a hairbrush.*
7/31/2014 50 50
Effective Multiple-Choice Items
The item rewritten in the active voice:
Which action promotes the spread of lice?
A. Putting conditioner in your hair
B. Washing your hair every day
C. Using a comb in your hair
D. Sharing a hairbrush*
7/31/2014 51
The item is succinct; it contains only the
needed information.
51
Effective Multiple-Choice Items
7/31/2014 52
This item is too wordy.
A geologist wants to show that layers in Earth’s surface are
composed of different types of rock. Which work of art best
shows the different layers of rock?
52
Effective Multiple-Choice Items
A. B. C. D.
7/31/2014 53
This item is succinct.
A geologist needs to distinguish between rock
layers. Which work of art solves this problem?
53
Effective Multiple-Choice Items
A. B. C.* D.
7/31/2014 54
The Stem
54
Effective Multiple-Choice Item Stems
7/31/2014 55
The item should have a closed stem. In a
closed stem the question posed is a
complete sentence and is punctuated
with a question mark.
55
Effective Multiple-Choice Item Stems
7/31/2014 56
Which activity helps to improve flexibility?
A. Jumping
B. Running
C. Stretching*
56
Effective Multiple-Choice Item Stems
Closed Stem
7/31/2014 57
Most items should not have an open
stem. An open stem is an incomplete
sentence. The sentence is completed by
the options.
57
Effective Multiple-Choice Item Stems
7/31/2014 58
Effective Multiple-Choice Item Stems
Open Stem
An example of a moderate physical activity that a student could do on most days would be
Options
A. Sprinting for 100 yards.
B. Jumping rope for five minutes.
C. Playing a board game with a friend.
D. Walking at a medium pace for 20 minutes.*
58
7/31/2014 59
Open stems are more difficult to process than
closed stems. If the open stem is a short
incomplete sentence, you may be able to hold it in
your working memory while you determine which
of the options complete the sentence.
However, if it is a longer incomplete sentence, you
will have to go back and reread the partial
sentence for each option. This takes extra time
and can get confusing.
59
Effective Multiple-Choice Item Stems
7/31/2014 60
The item should not have a negative
stem. A negative stem is a sentence or
question that asks a student to find the
incorrect answer (key) among the correct
answers (distractors).
60
Effective Multiple-Choice Item Stems
7/31/2014 61 61
Effective Multiple-Choice Item Stems
Which behavior would NOT help prevent the spread of germs?
A. Covering your mouth when you cough
B. Using a tissue to blow your nose
C. Frequently washing your hands
D. Sharing a cup with your friend*
7/31/2014 62
Negative stems are hard to process and the negative word in the stem can easily go undetected. While taking the test, the student is in the mode of selecting the correct answer, not the incorrect answer. If the student does miss the negative word in the stem, he or she might choose the first option because it answers the question correctly and not read the other options.
Negative stems should be avoided unless the point of the assessment is to test the comprehension of negatives.
62
Effective Multiple-Choice Item Stems
7/31/2014 63
Here are some examples of negative words
that should be avoided when writing
multiple-choice items:
not
never
except
none
63
Effective Multiple-Choice Item Stems
7/31/2014 64
Effective Multiple-Choice Item Stems
The stem should have enough
information to direct attention to the
key. We call this type of stem a
“directed stem.”
64
7/31/2014 65
The stem in this item is not directed. You need
to read the options to figure out what the
question is asking.
Which of the following statements about
copying an artist’s work is true?
65
Effective Multiple-Choice Item Stems
7/31/2014 66
You might make a true statement about the definition of copying.
You might make a true statement about getting permission from an artist.
You might make a true statement about how copying can be prevented.
You might make a true statement about the consequences of copying art.
And the list can continue…
66
Effective Multiple-Choice Item Stems
7/31/2014 67
Here is the entire item. What should the stem say?
Which of the following statements about copying an artist’s
work is true?
67
Effective Multiple-Choice Item Stems
A. Copying artwork is allowed when no one knows the
artist.
B. Copying artwork is allowed when you plan to sell the
work.
C. Copying artwork is allowed when the artist gives
permission.*
D. Copying artwork is allowed when the artist will never see
the work.
7/31/2014 68
Effective Multiple-Choice Item Stems
The stem should contain technical
terminology only when testing that
terminology.
68
7/31/2014 69
An item with technical terminology that is not required to assess the skill that is being tested:
Which training principle uses passive exercises that involve passing the active end position, as the external force moves the limbs further than the active contracting of the agonist muscles?
A. Overload*
B. Specificity
C. Reversibility
D. Variance
69
Effective Multiple-Choice Item Stems
7/31/2014 70
An item with the appropriate use of technical terminology.
How does the crossover toe touch enhance physical fitness?
A. Improves flexibility*
B. Strengthens your heart
C. Increases muscle strength
D. Decreases muscle endurance
70
Effective Multiple-Choice Item Stems
7/31/2014 71
The Options
71
Effective Multiple-Choice Item Options
7/31/2014 72
All the options should be approximately
the same length. If they are different in
length, they should be ordered according
to length (shortest to longest or vice
versa) or ordered according to alpha or
numeric order.
72
Effective Multiple-Choice Item Options
7/31/2014 73 73
Effective Multiple-Choice Item Options
This is inappropriate; the key is obvious because it is the longest option.
What is one way that restaurant workers help prevent the spread of disease in a restaurant?
A. They wear matching uniforms.
B. They serve food on plastic plates.
C. They only serve food cooked in a microwave.
D. They wash their hands after using the restroom and
before preparing any food.*
7/31/2014 74 74
Effective Multiple-Choice Item Options
This is appropriate. The options are all the same length.
When you are running, what two body parts work together
to deliver blood and oxygen to different parts of the body?
A. Your lungs and liver
B. Your heart and lungs*
C. Your brain and heart
D. Your liver and spleen
7/31/2014 75 75
Effective Multiple-Choice Item Options
This is appropriate; the options are ordered from shortest to longest.
The soup brand “Hearty’s Tomato Soup” is sold at several stores.
What will be the same everywhere the soup is sold?
A. The price that is charged for one can of soup.
B. The number of soup cans that each store sells.
C. The nutritional information shown on the soup can.*
D. The part of the store where the cans of soup are displayed.
7/31/2014 76
The options should not contain the
words “all”, “always”, “only” or “never”
but may use limiting words like “usually”
or “sometimes.”
76
Effective Multiple-Choice Item Options
7/31/2014 77
Use of limiting terms makes the key more
obvious in this item.
Why is it important to include illustrations in
cookbooks?
77
Effective Multiple-Choice Item Options
A. They make it easier to understand the steps to the recipe.*
B. They provide pretty pictures that readers will always enjoy.
C. They always make the food from the recipes taste better.
D. They always distract readers from the poor writing.
7/31/2014 78
Now the key is not obvious.
Why is it important to include illustrations in
cookbooks?
78
Effective Multiple-Choice Item Options
A. They make it easier to understand the steps to the recipe.*
B. They provide pretty pictures that readers will enjoy.
C. They make the food from the recipes taste better.
D. They distract readers from poor writing.
7/31/2014 79
The options should be grammatically
parallel to one another.
79
Effective Multiple-Choice Item Options
7/31/2014 80
In this item, the options are not grammatically parallel.
How can a pedometer help you achieve your fitness goals?
80
Effective Multiple-Choice Item Options
A. Cardiorespiratory fitness is increased.
B. It causes you to maintain a healthy body composition.
C. It can help you track how far you walk each day.*
D. By recording your eating habits or calorie count.
7/31/2014 81 81
Effective Multiple-Choice Item Options
Here is the item rewritten with parallel options.
How can a pedometer help you achieve your fitness goals?
A. Increase your cardiorespiratory fitness.
B. Maintain a healthy body composition.
C. Track how far you walk each day.*
D. Record your eating habits.
7/31/2014 82
The options should not contain repeated
information.
82
Effective Multiple-Choice Item Options
7/31/2014 83 83
Effective Multiple-Choice Item Options
This item contains the same initial wording in each option.
What is one way that a community can help promote healthy habits in children?
A. A community can have a health fair.*
B. A community can open a movie theater.
C. A community can hold a baking contest.
D. A community can build more restaurants.
7/31/2014 84 84
Effective Multiple-Choice Item Options
Here the duplicate wording is eliminated and the options are more succinct.
What is one way that a community can help promote healthy habits in children?
A. Have a health fair*
B. Open a movie theater
C. Hold a baking contest
D. Build more restaurants
7/31/2014 85
The Key
85
Effective Multiple-Choice Item Keys
7/31/2014 86
The key should answer the question
posed in the stem.
86
Effective Multiple-Choice Item Keys
7/31/2014 87
In this item, the key does not match the question posed
in the stem.
Why was paper chosen to create this work of art?
87
Effective Multiple-Choice Item Keys
A. The crane can flap its wings.
B. The crane is a symbol for wisdom.
C. The work of art is a form of origami.*
D. The artist used one type of material.
7/31/2014 88
Here is the item rewritten so the key answers the
question posed in the stem.
Why was paper chosen to create this work of art?
88
Effective Multiple-Choice Item Keys
A. Paper is white.
B. Paper can be cut.
C. Paper is easy to fold.*
D. Paper can be written on.
7/31/2014 89
The key should be correct and should
be the only correct answer.
89
Effective Multiple-Choice Item Keys
7/31/2014 90
In this item there are multiple correct answers.
Which activity would help you to achieve your long-term goal of increasing muscle strength in your arms?
A. Squats both keys
B. Pull-ups
C. Running
D. Push-ups
90
Effective Multiple-Choice Item Keys
7/31/2014 91
Here is the item rewritten with only one correct answer.
Which activity would help you to achieve your long-term goal of increasing muscle strength in your arms?
A. Squats
B. Pull-ups*
C. Running
D. Stretching
91
Effective Multiple-Choice Item Keys
7/31/2014 92
The key should not contain significant
language from the stem.
92
Effective Multiple-Choice Item Keys
7/31/2014 93 93
Effective Multiple-Choice Item Keys
In this item the key is the only option that uses the wording from the stem.
How can you show respect for other students in your class?
A. Borrow their things without asking for permission
B. Whisper secrets to your neighbor during assignments
C. Let people around you copy answers off of your paper
D. Be quiet when the teacher is talking to show respect for others
7/31/2014 94 94
Effective Multiple-Choice Item Keys
In this item, none of the options uses the wording from the stem and the key doesn’t stand out.
How can you show respect for other students in your class?
A. Borrow their things without asking for permission
B. Whisper secrets to your neighbor during assignments
C. Let people around you copy answers off of your paper
D. Be quiet when the teacher is talking so everyone can hear*
7/31/2014 95
The key should not be the only positively
or negatively worded option.
95
Effective Multiple-Choice Item Keys
7/31/2014 96
In this item the key is the only negatively worded
option.
Allison’s teacher presents this painting as an example of symmetry to her
class.
Why does Allison’s teacher use this painting as an example of symmetry?
96
Effective Multiple-Choice Item Keys
A. Because there is no difference between each side of the painting.
B. Because the painting is made of both warm and cool colors.
C. Because the painting has different types of texture.
D. Because the shapes in the painting are organic.
7/31/2014 97
Here is the item rewritten so the key does not
stand out.
Allison’s teacher presents this painting as an example of symmetry to
her class.
Why does Allison’s teacher use this painting as an example of
symmetry?
97
Effective Multiple-Choice Item Keys
A. Because one side of the painting looks the same as the other side.*
B. Because the painting is made of both warm and cool colors.
C. Because the painting has different types of texture.
D. Because the shapes in the painting are organic.
7/31/2014 98
The Distractors
98
Effective Multiple-Choice Item Distractors
7/31/2014 99
The distractors should be plausible but
incorrect.
99
Effective Multiple-Choice Item Distractors
7/31/2014 100
This item has two obviously wrong distractors, making this a two-option item instead of a four-option item.
Which is an example of muscular endurance?
A. Sleeping for eight hours
B. Running a marathon*
C. Swimming one lap in the pool
D. Watching television while eating candy
100
Effective Multiple-Choice Item Distractors
7/31/2014 101
Here is the item rewritten with only plausible distractors.
Which is an example of muscular endurance?
A. Lifting a heavy box
B. Running a marathon*
C. Swimming one lap in the pool
D. Sprinting to the far side of a field
101
Effective Multiple-Choice Item Distractors
7/31/2014 102
The distractors should represent logical
misunderstandings or common
misconceptions.
102
Effective Multiple-Choice Item Distractors
7/31/2014 103 103
Effective Multiple-Choice Item Distractors
Here is an item with distractors that do not contain misunderstandings or misconceptions.
What should you do to be ready for an important test at school?
A. Eat candy and sweets the day of the test
B. Get adequate rest the night before the test*
C. Play a sports game the night before the test
D. Skip eating breakfast on the morning of the test
7/31/2014 104 104
Effective Multiple-Choice Item Distractors
Here is the same item rewritten with distractors that contain common misconceptions.
What should you do to be ready for an important test at school?
A. Stay up late the night before to study for the test
B. Eat a big meal the day before the test
C. Get adequate rest the night before the test*
D. Wait to study hard the day before the test
7/31/2014 105
The distractors should avoid using
opposing statements.
105
Effective Multiple-Choice Item Distractors
7/31/2014 106
This item contains two options that oppose each other.
What should you do to prevent a person from copying your artwork?
106
Effective Multiple-Choice Item Distractors
A. Sell your artwork
B. Sign your artwork
C. Display your artwork
D. Hide your artwork
7/31/2014 107
Effective Multiple-Choice Item Distractors
Here is the item with the distractors rewritten.
What should you do to prevent a person from copying your artwork?
107
A. Sell your artwork
B. Sign your artwork*
C. Display your artwork
D. Give away your artwork
7/31/2014 108
The distractors should not use the
choices “all of the above” or “none of the
above.”
108
Effective Multiple-Choice Item Distractors
7/31/2014 109
Here is an item with distractors that contain
“all of the above” and “none of the above.”
Fatima and her four friends want to hula hoop
during recess, but they only have three hula
hoops. What way should they solve this problem?
109
Effective Multiple-Choice Item Distractors
A. To break the hula hoops
B. To race for the hula hoops
C. To take turns using the hula hoops*
D. None of the above
7/31/2014 110
Here is the item rewritten.
Fatima and her four friends want to hula hoop
during recess, but they only have three hula
hoops. What way should they solve this problem?
110
Effective Multiple-Choice Item Distractors
A. To break the hula hoops
B. To take turns using the hula hoops*
C. To let the fastest runners use the hula hoops
C. To give the hula hoops to the people who hula
hoop the best
7/31/2014 111 7/31/2014 111
Constructed Response and
Performance Task
What’s the difference?
• Written response vs. performance
• Difference in setup
• Rubrics are very similar
111
7/31/2014 112 7/31/2014 112
Constructed-Response
Items
112
7/31/2014 113 113
Constructed-Response Item
Terminology
Describe a revision you would make to this
photograph using vocabulary learned in
class. Then explain why you would make
this revision.
Stimulus
Prompt
7/31/2014 114 114
Constructed-Response Item
Terminology
Exemplar Response: I would change the camera angle so that the camera looks up at the lion.
It would make the lion look more dangerous.
Item Rubric
2 points The student describes a revision using art vocabulary from the
class and also describes why he or she would make the
revision.
1 point The student describes a revision and why he or she would
make it, but the description does not include vocabulary from
the class.
OR
The student describes a revision using art vocabulary from
class but does not describe why he or she would make the
revision.
0 points The response indicates inadequate or no understanding of the
concept needed to answer the item. The student may have
written on a different topic or written “I don’t know.”
Score
Points
Score-Point
Descriptors
7/31/2014 115 7/31/2014 115
Performance Tasks
115
7/31/2014 116
Note: The teacher may first demonstrate the task so that the student understands the task
116
Performance Task Terminology
Prompt
Materials:
• Inline skates or roller skates
• Six cones
Setup:
• Place the six cones in a straight line. The cones
should be 4 feet apart.
Prompt:
• Say: “Skate in and out of the cones. Then, skate
back through them.”
7/31/2014 117 117
Performance Task Terminology
Item Rubric:
2 points The student successfully skates through the cones both
ways.
1 point The student skates through most of the cones but
misses or knocks over one to three cones.
OR
The student successfully skates through the cones both
ways but falls once or twice.
0 points The student is unable to perform the task, does not
understand how to complete the task, or unsuccessfully
completes the task.
Score
Points
Score-Point
Descriptors
7/31/2014 118
Effective Constructed-Response Items and
Performance Task Items
Writing Effective
Constructed-Response Items
and
Performance Tasks
118
7/31/2014 119
The following slides provide a list of points
to consider when reviewing effective
constructed-response and performance
task items.
Effective Constructed-Response Items and
Performance Task Items
119
7/31/2014 120
The Item or Task
Effective Constructed-Response Items and
Performance Task Items
120
7/31/2014 121
The item or task should be
aligned to the standard,
benchmark, and indicator it is
assessing.
Effective Constructed-Response Items
and Performance Task Items
121
7/31/2014 122
Is this middle school health item correctly aligned? Explain why or why not.
Subject: Health
Standard: 1. Comprehend concepts related to health promotion and disease
prevention to enhance health.
Benchmark: HE.6.C.1.1 Identify the effects of healthy and unhealthy
behaviors on personal health.
Danielle eats large portions of high sugar and high fat foods at every meal.
She also does not exercise regularly.
List two long-term effects of her unhealthy diet and lack of exercise.
Effective Constructed-Response Items and
Performance Task Items
122
7/31/2014 123
The item should be fair and free
from bias.
Effective Constructed-Response Items and
Performance Task Items
123
7/31/2014 124
On the next slide, you will see a middle
school physical education item.
Use the Language Accessibility, Bias and
Sensitivity (LABS) guidelines in your
handouts when reviewing this item.
Effective Constructed-Response Items and
Performance Task Items
124
7/31/2014 125
Which LABS guideline does this item violate?
James enjoys hunting, but his arms quickly tire
from carrying and holding his rifle.
Identify two activities James can do to increase
the muscular strength of his arms.
Effective Constructed-Response Items and
Performance Task Items
125
7/31/2014 126
The item should be written at a
reading level that is at or below
the grade level of the students
who will be taking the
assessment.
Effective Constructed-Response Items and
Performance Task Items
126
7/31/2014 127
Items written at the appropriate grade level conform to the grammar and vocabulary demands of texts used in the grade-level classroom.
On the next slide, you will see a middle school art item.
Effective Constructed-Response Items and
Performance Task Items
127
7/31/2014 128
Is this item written at the appropriate level for a fifth grade assessment? If not, how would you rewrite it?
Explain why this image is exemplary artwork of the new age style, and provide justification for your response.
Effective Constructed-Response Items and
Performance Task Items
128
7/31/2014 129
The item should assess
significant information.
Effective Constructed-Response Items and
Performance Task Items
129
7/31/2014 130
According to the benchmark, does this middle school health item assess significant information? If not, how would you edit this item?
Subject: Health
Standard: 2. Analyze the influence of family, peers, culture, media, technology,
and other factors on health behaviors.
Benchmark: HE.6.C.2.3 Identify health information conveyed to students by the
school and community.
It is good to provide health information in areas used by many people.
What health information is commonly posted at community pools?
A. Stay hydrated
B. Wear sunscreen
C. Cover open wounds
D. Shower before entering*
Effective Constructed-Response Items and
Performance Task Items
130
7/31/2014 131
The item should not contain the
passive voice construction.
Effective Constructed-Response Items and
Performance Task Items
131
7/31/2014 132
How would you rewrite this middle school visual arts item that uses the passive voice?
Write a brief paragraph on why you like to create art and why art is created by other people.
Effective Constructed-Response Items and
Performance Task Items
132
7/31/2014 133
The item is succinct; it contains
only the needed information.
Effective Constructed-Response Items and
Performance Task Items
133
7/31/2014 134
Is this middle school physical education item succinct? If not, how would you rewrite it?
Leo is researching advertising messages about exercise
equipment. The advertising messages appear on
television, in stores, on the radio, and in catalogues. He
sees that many products give true and false claims in
order to make their products sell more quickly.
Identify a piece of exercise equipment and give an
example of one fact about the equipment. Then, give
another example of one false claim about the exercise
equipment that might make someone want to buy it.
Effective Constructed-Response Items and
Performance Task Items
134
7/31/2014 135
The Prompt
Effective Constructed-Response Items and
Performance Task Items
135
7/31/2014 136
The prompt should contain technical
terminology only when testing that
terminology.
Effective Constructed-Response Items and
Performance Task Items
136
7/31/2014 137
How would you revise this middle school health item to eliminate technical language that is not necessary to test the benchmark?
Subject: Health
Standard: 1. Comprehend concepts related to health promotion and disease
prevention to enhance health.
Benchmark: HE.6.C.1.8 Explain how body systems are impacted by
hereditary factors and infectious agents.
Sickle-cell anemia is an autosomal recessive genetic blood disorder.
Explain two ways sickle cell anemia affects the body.
Effective Constructed-Response Items and
Performance Task Items
137
7/31/2014 138
The prompt for a constructed-response
item should require a more complex
response than the stem of a multiple-
choice item.
Effective Constructed-Response Items and
Performance Task Items
138
7/31/2014 139
This visual arts item is really a multiple-choice item.
Describe the first step in creating a drawing.
Effective Constructed-Response Items and
Performance Task Items
139
7/31/2014 140
This visual arts item requires a more complex response.
Explain the importance of organizing and planning an artwork before attempting a final draft.
Effective Constructed-Response Items and
Performance Task Items
140
7/31/2014 141
The prompt should not ask a
question that can be answered by a
simple yes or no.
Effective Constructed-Response Items and
Performance Task Items
141
7/31/2014 142
This middle school Physical Education item can be answered by a yes or a no.
Decide if you should do something that you know is wrong when you are being pressured by your friends.
Effective Constructed-Response Items and
Performance Task Items
142
7/31/2014 143
This version of the middle school Physical Education item requires more than a simple yes or no.
Describe two ways you can respond when
you are being pressured by your friends to do
something you know is wrong.
Effective Constructed-Response Items and
Performance Task Items
143
7/31/2014 144
The prompt should have enough
information to focus attention on the task
that must be accomplished.
Effective Constructed-Response Items and
Performance Task Items
144
7/31/2014 145
This high school visual arts item is too broad. There are many ways to capture motion in photography and the item does not provide students with enough context to discuss any one solution in a sufficiently detailed manner.
Describe two photographic techniques or
tools photographers use to capture motion.
Effective Constructed-Response Items and
Performance Task Items
145
7/31/2014 146
This high school visual arts item focuses students on the task they must accomplish.
A photographer is trying to take a picture of a
waterfall on a bright, sunny day. The water is
moving quickly. The goal is to emphasize the
motion of the water. Describe two
photographic techniques or tools the
photographer can use to achieve this.
Effective Constructed-Response Items and
Performance Task Items
146
7/31/2014 147
The prompt should clearly indicate the
amount and type of information that must
be included in the response for the
student to receive the highest score
point.
Effective Constructed-Response Items and
Performance Task Items
147
7/31/2014 148
This middle school Physical Education item is
vague. It does not clearly communicate the
information required in the response or the
number of points allotted to this task.
Use your knowledge of video games to
describe how they help you improve your
motor skills in your legs and feet.
Effective Constructed-Response Items and
Performance Task Items
148
7/31/2014 149
This version of the middle school Physical
Education item clearly states what information
the student needs to include in the response.
Describe one kind of video game that can
help you improve motor skills in your legs and
feet. Then, explain how it helps you improve
them.
Effective Constructed-Response Items and
Performance Task Items
149
7/31/2014 150
The prompt should not ask for
repetitions of the same task to obtain
additional score points.
Effective Constructed-Response Items and
Performance Task Items
150
7/31/2014 151
This middle school Health item asks student
to repeat the same task a number of times to
gain score points.
Marisa tells Gabby about how much she likes
her soccer team. Gabby decides to join the
same team.
Describe one way this helps Gabby be
healthier, then explain why it helps Gabby be
healthier.
Effective Constructed-Response Items and
Performance Task Items
151
7/31/2014 152
This middle school Health item requires
students to perform different tasks to earn
score points.
Describe one way the Internet can have a
positive effect on a person’s social or
emotional health.
Then, describe one way the Internet can
negatively affect a person’s social or
emotional health.
Effective Constructed-Response Items and
Performance Task Items
152
7/31/2014 153
The prompt should maintain the
independence of each score point and
should try to avoid score point
dependencies.
Effective Constructed-Response Items and
Performance Task Items
153
7/31/2014 154
This middle school Physical Education item has a notable score-point dependency. Tasks should remain as independent as possible.
Gabby is 12 years old and has a healthy body composition. Her resting heart rate is 80 beats per minute. Immediately after riding her bike, her heart rate increases by 40 beats per minute.
Immediately after she exercised, was Gabby’s heart rate in her target heart-rate zone? Explain how you know.
Effective Constructed-Response Items and
Performance Task Items
154
7/31/2014 155
This middle school Physical Education item requires
students to perform different tasks to earn score
points.
Gabby wants to increase her cardiorespiratory
endurance. After riding her bike for 20 minutes,
Gabby’s pulse is still below her target heart-rate zone.
Explain two different actions Gabby could take to
raise her heart rate into her target heart-rate zone
while riding her bike.
Effective Constructed-Response Items and
Performance Task Items
155
7/31/2014 156
The prompt should contain a task that
can be completed in the time allowed.
Effective Constructed-Response Items and
Performance Task Items
156
7/31/2014 157
This middle school Health item prompt is too complex to be completed in five minutes.
Explain a time when you had to solve a conflict
between yourself and another person.
Describe the details of the situation and what you
did to solve it. Then, explain whether or not your
solution worked. Finally, suggest two actions you
could have taken to resolve the conflict in another
way.
Effective Constructed-Response Items and
Performance Task Items
157
7/31/2014 158
This middle school Health item prompt
contains tasks that can be completed in the
time allowed.
Describe two emergency situations that would
require professional medical care.
Effective Constructed-Response Items and
Performance Task Items
158
7/31/2014 159
The Rubric
Effective Constructed-Response Items and
Performance Task Items
159
7/31/2014 160
The rubric should be aligned with the
prompt.
Effective Constructed-Response Items and
Performance Task Items
160
7/31/2014 161
Here is a high school visual arts item. The rubric for this item is not
aligned with the prompt. Describe one connection between the cultures captured in these photographs.
Provide one detail from the photos to support your answer.
Exemplar Response:
When a photographer took these photographs of people dancing, they needed no
knowledge of language or understanding the dancer’s culture. Dance is an art of
motion.
Item Rubric
2 points The student describes the subjects of the photos and how the photographer captured the subjects.
1 point The student describes the photographs as having dancers but not the connection between
the cultures, does not include any details to support the answer.
0 points The response indicates inadequate or no understanding of the concept needed to answer
the item. The student may have written on a different topic or written “I don’t know.”
Effective Constructed-Response Items and
Performance Task Items
161
7/31/2014 162
The rubric for this high school visual arts item is aligned with the prompt.
Describe one connection between the cultures captured in these photographs. Provide one
detail from the photos to support your answer.
Exemplar Response:
Both of these cultures participate in dance. I know this because the photographer captured
the dancing motion of both cultures.
Item Rubric
2 points The student describes the connection between the cultures in the photographs and
includes one detail to support the answer. Possible connections include dance,
costume, or ceremony. 1 point The student describes the photographs as having dancers but not the connection
between the cultures, does not include any details to support the answer.
0 points The response indicates inadequate or no understanding of the concept needed to
answer the item. The student may have written on a different topic or written
“I don’t know.”
Effective Constructed-Response Items and
Performance Task Items
162
7/31/2014 163
The rubric should allow for multiple
response solutions or for multiple ways
to achieve a correct response.
Effective Constructed-Response Items and
Performance Task Items
163
7/31/2014 164
The rubric for this middle school Physical Education
item demonstrates the limited scope of the item. (Would
this item be better as a multiple-choice?)
Two principles of conditioning are Frequency and
Intensity. Name the two other principles of conditioning. Item Rubric
2 points The student names the following principles of conditioning/fitness:
Time
Type
1 point The student names only one of the above principles of conditioning/fitness.
0 points The response indicates inadequate or no understanding of the concept needed to
answer the item. The student may have written on a different topic or written “I don’t
know.”
Effective Constructed-Response Items and
Performance Task Items
164
7/31/2014 165
The rubric for this middle school Physical Education item
accommodates various correct responses.
Item Rubric
2 points The student names two of the following principles of conditioning/fitness:
Frequency
Intensity
Time
Type
1 point The student names only one of the above principles of conditioning/training.
0 points The response indicates inadequate or no understanding of the concept needed to
answer the item. The student may have written on a different topic or written “I don’t
know.”
Name two of the principles of conditioning.
Effective Constructed-Response Items and
Performance Task Items
165
7/31/2014 166
The score point descriptors should
clearly delineate what a student must do
to achieve each score point.
Effective Constructed-Response Items and
Performance Task Items
166
7/31/2014 167
Effective Constructed-Response Items and
Performance Task Items
The score points in this middle school Physical
Education rubric are clearly defined.
Item Rubric
2 points The student successfully explains that poor cardiorespiratory endurance will make the
run-time slower or take longer
AND
The student includes a plausible explanation of why this is so, including but not limited to:
Breathing difficulties, Side cramps, Exhaustion
1 point The student successfully explains that poor cardiorespiratory endurance will make the
run-time slower or take longer
OR
The student includes a plausible explanation of why this is so, including but not limited to:
Breathing difficulties, Side cramps, Exhaustion
0 points The response indicates inadequate or no understanding of the concept needed to answer
the item. The student may have written on a different topic or written “I don’t know.”
Explain how poor cardiorespiratory endurance can impact your time for the mile
run in Physical Education class. Then explain why it has the effect you chose.
Exemplar Response:
Low cardiorespiratory endurance means you will have a slower time. It makes
you have a slower time because you are more likely to get out of breath faster.
167
7/31/2014 168
The score points in this version of the rubric are not clearly
defined. (Is the exemplar below one point or two points based
on the rubric?)
Explain how low cardiorespiratory endurance can impact your time for the
mile run in Physical Education class.
Exemplar Response:
Low cardiorespiratory endurance means you will have a slower time.
Item Rubric
2 points The student successfully describes a consequence of low cardiorespiratory
endurance on running ability.
1 point The student gives an incomplete consequence of low cardiorespiratory endurance
on running ability.
0 points The response indicates inadequate or no understanding of the concept needed to
answer the item. The student may have written on a different topic or written “I don’t
know.”
Effective Constructed-Response Items and
Performance Task Items
168
7/31/2014 169
The rubric should not contain scoring
rules that do not directly correspond to
what is asked in the prompt.
Effective Constructed-Response Items and
Performance Task Items
169
7/31/2014 170
This high school Visual Arts item rubric includes a scoring
rule that does not appear in the prompt.
Item Rubric
2 points The student describes why a subject may be revisited several times and one
revision that could be made during the process.
Reasons why a subject may be revisited may include:
Making a creative change
To review more closely
To correct a technical issue
1 point The student describes why a subject may be revisited several times or describes
one revision that could be made to photographs.
0 points The response indicates inadequate or no understanding of the concept needed to
answer the item. The student may have written on a different topic or written “I don’t
know.”
Explain why you might revisit a subject several times.
Exemplar Response:
I would revisit a subject to make a creative change. The change could
involve a new camera angle.
Effective Constructed-Response Items and
Performance Task Items
170
7/31/2014 171
The scoring rule in this high school Visual Arts rubric is
aligned to the tasks in the prompt.
Explain why you might revisit a subject several times, and describe one
revision you could make during your process.
Exemplar Response:
I would revisit a subject to make a creative change. The change could
involve a new camera angle.
Item Rubric
2 points The student describes why a subject may be revisited several times and one
revision that could be made during the process.
Reasons why a subject may be revisited may include:
Making a creative change
To review more closely
To correct a technical issue
1 point The student describes why a subject may be revisited several times or describes
one revision that could be made to photographs.
0 points The response indicates inadequate or no understanding of the concept needed to
answer the item. The student may have written on a different topic or written “I
don’t know.”
Effective Constructed-Response Items and
Performance Task Items
171
7/31/2014 172
This middle school Physical Education item rubric includes a
scoring rule that does not appear in the prompt.
Prompt:
Say: “Create a sequence of 5 rhythmic movements that incorporate basic
gymnastic poses, such as a walkover.”
Effective Constructed-Response Items and
Performance Task Items
172
Item Rubric
2 points The student successfully creates and completes five rhythmic gymnastics movements
AND
The student uses smooth transitions between movements.
Note: Students will not receive credit for using a walkover because it is simply restating the prompt.
1 point The student partially creates and completes two or three of the gymnastic movements
in the planned sequence.
0 points The response indicates inadequate or no understanding of the concept needed to
answer the item. The student may have written on a different topic or written “I don’t
know.”
7/31/2014 173
The scoring rule in this middle school Physical
Education rubric is aligned to the tasks in the prompt.
Prompt:
Say: “Create a sequence of 5 rhythmic movements that incorporate basic
gymnastic poses, such as a walkover. You may NOT use a walkover as
one of your movements. Be sure to use smooth transitions between
movements.”
Effective Constructed-Response Items and
Performance Task Items
173
Item Rubric
2 points The student successfully creates and completes five rhythmic gymnastics movements
AND
The student uses smooth transitions between movements.
Note: Students will not receive credit for using a walkover because it is simply restating the prompt.
1 point The student partially creates and completes two or three of the gymnastic movements
in the planned sequence.
0 points The response indicates inadequate or no understanding of the concept needed to
answer the item. The student may have written on a different topic or written “I don’t
know.”
7/31/2014 174
The rubric should contain unambiguous
response exemplars.
Effective Constructed-Response Items and
Performance Task Items
174
7/31/2014 175
The rubric of this middle school Health item has
exemplars that do not readily illustrate the type of
response that would earn full credit.
Prompt:
Describe two ways that schools and communities can
help keep students safe when they walk to school.
Exemplar Response:
Safety rules are important to have so that children do
not get hurt when crossing the street. Schools and
communities have different types of safety rules.
Effective Constructed-Response Items and
Performance Task Items
175
7/31/2014 176
This version of the rubric has exemplars that clearly
illustrate full-credit responses.
Prompt:
Describe two ways that schools and communities can
help keep students safe when they walk to school.
Exemplar Response:
They can have speed limits so cars don’t go too fast, and
they can have school crossing guards.
Effective Constructed-Response Items and
Performance Task Items
176