assessment items
TRANSCRIPT
DEVELOPING HIGH QUALITY
ASSESSMENT ITEMS
Region 7 Digital Learning Academy
Dr. LaTricia Townsend
Research Scholar
Educational Evaluation
NCSU Friday Institute
@tricia_townsend
HTTP://REGION7DLA.WIKISPACES.COM/
LOOKING BACK
o Activity-Table Share: Use of Session
1 Information Reflection
How have you used information
gained from Session 1 in your
practice?
How have you shared the
information from Session 1 in your
school or district?
LOOKING FORWARD
Take the poll about how you obtain
assessment items
ITEM DEVELOPMENT PROCESS
Curriculum Standards
ALIGNMENT
Curriculum Standards
Depth and Type of Thinking
ACCESS CURRICULAR DOCUMENTS
Activity- Take a moment to locate curriculum standards materials
Common Core ELA and Math
Essential StandardsNext Generation Science StandardsUnpacking documents completed by
school/districtCurriculum Maps/Pacing GuidesBegin to think about which ones you want
to address today
REVISED BLOOM’S TAXONOMY-KNOWLEDGE
PROCESS DIMENSION
FACTUAL KNOWLEDGE-basic elements students must know to be acquainted with a discipline or solve problems in it
CONCEPTUAL KNOWLEDGE-The interrelationships among the basic elements within a larger structure that enable them to function together
PROCEDURAL KNOWLEDGE-How to do something, methods of inquiry, and criteria for using skills, algorithms, techniques, and methods
METACOGNITIVE KNOWLEDGE-Knowledge of cognition in general as well as awareness and knowledge of one's own cognition
REVISED BLOOM’S TAXONOMY-COGNITIVE
PROCESS DIMENSION
Revised Bloom’s Taxonomy According to Homer
Simpson
WEBB’S DEPTH OF KNOWLEDGE
Level 1: Recall
Recall a fact, information, or procedure
Level 2: Skill/Concept
Use information or conceptual knowledge, two or
more steps
Level 3: Strategic Thinking
Requires reasoning, developing a plan or a
sequence of steps, some complexity
Level 4: Extended Thinking
Requires an investigation, time to think and
process multiple conditions of the problem.
PRACTICE ALIGNMENT
o Activity-Write items (open ended- that align
to your content area and Revised Bloom’s
Taxonomy/Webb’s Depth of Knowledge)
• Write four questions to address each of
the four levels.
• Record your responses using the tool of
your choice for your own records.
• Share your responses on the Google
Spreadsheet
PRACTICE ALIGNMENT
o Activity-Find a partner who teaches/supports in the same grade span in a different district. Share your questions and discuss how you might approach the content differently.
o Reflect #R7DLA
MULTIPLE CHOICE ITEM GUIDELINES
Content General
Format Structure
• Stem
• Foil
Accessibility
• SWD and LEP
Bias/Sensitivity
MULTIPLE CHOICE TERMINOLOGY
Which is the state bird of North
Carolina?
A. bluebird
B. cardinal
C. robin
D. sparrow
Stem
Distractor
Foils
Distractor
Key
Distractor
MULTIPLE CHOICE TERMINOLOGY
Passage
http://www.ncpublicschools.org/docs/accountability/testing/releasedforms/g3elapp.pdf
MULTIPLE CHOICE TERMINOLOGY
Stimulus
http://www.ncpublicschools.org/docs/accountability/testing/releasedforms/g5mathpp.pdf
MULTIPLE CHOICE GUIDELINES-CONTENT
Base items on curriculum standards and
DOK.
Keep the testing vocabulary consistent
with grade level.
Measure a single
concept, principle, procedure, or
competency.
Write items with a mixture of difficulty
levels-Easy, Medium, Hard
MULTIPLE CHOICE GUIDELINES-CONTENT
Choose to assess important information, not trivial material.
POOR ITEM
How many different elements make up the Noble Gases?
A. 3
B. 4
C. 5
D. 6
IMPROVED ITEM
What common property is shared by all the noble gases? A. They are highly reactive and must be stored in air-tight containers.
B. They are inert, and do not easily form chemical compounds.
C. They are metallic in nature, and have basic oxides.
D. When mixed with water, they form a toxic blue-green solution.
MULTIPLE CHOICE GUIDELINES-GENERAL
Test the content, not the test-taking or
reading ability of students-avoid tricky
or misleading items.
Avoid errors of
grammar, abbreviations, punctuation
and spelling.
Do not use contractions.
Do not use first or second person.
Do not use humor.
MULTIPLE CHOICE GUIDELINES-FORMAT
Write clear, concise instructions.
Format foils vertically.
Use space in a balanced manner to
produce an easy to follow format
MULTIPLE CHOICE GUIDELINES-STRUCTURE
Avoid writing complex format multiple-choice
items.
Which of the following are fruits?
1. Tomatoes
2. Tomatillos
3. Habanero peppers
A. 1 & 2
B. 2 & 3
C. 1 & 3
D. 1, 2, & 3
MULTIPLE CHOICE GUIDELINES-STRUCTURE
Do not use fill-in-the blank format. Pose a question.
Be brief . Wordiness in the stem can clue or confuse the student. (“window-dressing”)
POOR ITEM
High temperatures and heavy rainfall characterize a humid climate. People who live in this
kind of climate usually complain of problems dealing with heavy amounts of perspiration.
Even moderately warm days are uncomfortable for most people in this climate. Which
biome is being described?
A. savanna
B. taiga
C. tundra
D. tropical rainforest
IMPROVED ITEM
Which term describes a biome with high temperatures and heavy rainfall?
A. savanna
B. taiga
C. tundra
D. tropical rainforest
MULTIPLE CHOICE GUIDELINES-STEM
Word the stems positively. Do not use any negative phrasing, such as words NOT or EXCEPT.
Do not write stems that ask for or express opinions.
Write the items so that the central idea and any common elements are included in the stem instead of the foils.
Place the interrogative as close to the foils as possible.
Use qualifying words in a stem, when needed, to emphasize the “best answer.” Words such as best, most likely, and most appropriate are bold and italicized.
MULTIPLE CHOICE GUIDELINES- FOIL
DEVELOPMENT
Each item must contain four foils (A,B,C,D)
Order the answer choices in a logical
order, such as ascending or descending value
for numbers, the order in which the words
appear in a chart, chronological order, etc.
Each item written should contain foils that are
independent and not overlapping.
To the extent possible, foils for an item should
be homogeneous in content and length.
MULTIPLE CHOICE GUIDELINES- FOIL
DEVELOPMENT
Do not use the following as foils: all of the above, none of the above, I don’t know.
Word the foils positively, avoid any negative phrasing.
Avoid providing clues to the correct response.
Avoid writing items in which phrases in the stem are repeated in the foils.
MULTIPLE CHOICE GUIDELINES- FOIL
DEVELOPMENT
Avoid absolutes such as ALWAYS, NEVER, TOTALLY, and ABSOLUTELY should not be used when writing items.
Qualifiers such as best, most likely, approximately, etc. should be bold and italic.
Present one and only one best (correct) answer.
Use the common errors made by students as distractors.
MULTIPLE CHOICE GUIDELINES- FOIL
DEVELOPMENT
Use plausible distractors. The best (correct) answer must clearly be the best (correct) answer and the incorrect responses must clearly be inferior to the best (correct) answer. No distractor should be obviously wrong.POOR ITEM
Which example best describes the adhesion
property of water?
A. drops of water on the outside of a glass
B. sugar dissolving in water
C. ice floating in water
D. melting of aluminum
MULTIPLE CHOICE GUIDELINES- FOIL
DEVELOPMENT
Use plausible distractors.IMPROVED ITEM
Which example best describes the adhesion
property of water?
A. drops of water on the outside of a glass
B. sugar dissolving in water
C. ice floating in water
D. water changing from a liquid to a gas
MULTIPLE CHOICE GUIDELINES-UNIVERSAL
DESIGN
Make items accessible and usable by students
of all abilities, including students with
disabilities and students with limited English
proficiency.
Use simple language and sentence structure.
Do not use colloquial language.
MULTIPLE CHOICE GUIDELINES-UNIVERSAL
DESIGN
Limit the use of homonyms, homophones, and homographs
fair1. Adjective. Meaning lovely or beautiful, when describing someone
2. Adjective. Meaning of light complexion, or light colored (blonde) hair
3. Adjective. Meaning reasonable, equitable, free from self-interest
3. Noun. Meaning a gathering of people to sell and barter goods, or to enjoy entertainment, and also a travelling show of rides, stalls and sideshows
SENSITIVITY AND BIAS
Do not use stereotypes based on
gender, race, religion, socio-economic
status, or other factors.
Respect diversity by including people from
all walks of life in use of names and artwork.
Ensure items do not highlight socio-
economic disparities.
Avoid espousing religious and political
beliefs.
REVISING AND WRITING QUESTIONS
o Activity-- Would you change any of the
questions that you posed, initially? Make
any necessary changes.
o Create 3 Multiple Choice Questions-
Record the questions using the same
tool you used before for your personal
use and share your responses on the
Google Spreadsheet
o Reflect #R7DLA
ASSESSING ITEMS
o Activity- Review Questions o Find a trioo Use the item review document to gauge
the quality of each of the itemso What changes would you make to the
items?
Reflect #R7DLA
GOT QUESTIONS…
Source:http://en.clipart-fr.com
DROP THEM ANYWHERE!
SCHOOL NET
o Activity- Tasks in School Net
Access the User Guide
o Log into School Net
o Try these:
o Create an Item in School Net
o Create an Express Test
o Create a Test using items from Item
Central
Reflect #R7DLA
WORK TASKS
Explore online resources
Brainstorm/Develop Culminating Resource
Reflect #R7DLA
ONLINE RESOURCES/SCHOOL NET
Activity- Explore the resources presented. How
could these be used in your instructional delivery or
assessment program? Are there other resources
you would like to share? Share your ideas to
Padlet.
FINAL PRODUCT IDEA BRAINSTORM/WORK
SESSION
Activity- Table Talk-With your team, discuss what
your product for School Net will be at the
culmination of DLA. What is the proposed product?
How do you anticipate incorporating future
information? How will it be disseminated in your
school/district?
Develop Concrete Next Steps
Share your ideas to wiki Padlet.
