textbook assessment items

8/6/2019 Textbook Assessment Items

1/20

Assessment Item Writing Assessment Item WritingWorkshopWorkshopKen RobbinsKen Robbins

FDNFDN--5560 Classroom Assessment5560 Classroom Assessment

ClickClick HEREHERE to return to the Documentationto return to the Documentation


2/20

IntroductionIntroduction

Classroom Assessment activityClassroom Assessment activityThe activity demonstrates how improperlyThe activity demonstrates how improperly

designed assessments leads to improper designed assessments leads to improper student inferencesstudent inferences


3/20

Textbook TestsTextbook Tests

Frisbie, Miranda, and Baker s 1993Frisbie, Miranda, and Baker s 1993research studyresearch study

Textbook publishers assessmentsTextbook publishers assessments90% of assessments at knowledge level90% of assessments at knowledge level

Gateway 2005 research studyGateway 2005 research study Teacher/textbook assessmentsTeacher/textbook assessments

96% at lower three levels of taxonomy96% at lower three levels of taxonomy68% at knowledge level68% at knowledge level


4/20

Blooms Taxonomy of theBlooms Taxonomy of the

Cognitive DomainCognitive DomainBlooms taxonomy of 1956Blooms taxonomy of 1956Updated in 2001 by Anderson et al.Updated in 2001 by Anderson et al. Original study defined six categoriesOriginal study defined six categories

KnowledgeKnowledgeComprehensionComprehension

Application Application Analysis Analysis

SynthesisSynthesisEvaluationEvaluation

NC uses Dimensions of ThinkingNC uses Dimensions of Thinking Adaptation of Marzanos research Adaptation of Marzanos research


5/20

Blooms TaxonomyBlooms Taxonomy

Key WordsKey WordsKnowledge (facts, recall, recognition)Knowledge (facts, recall, recognition) Define, list, select, match, who, what,Define, list, select, match, who, what,

when, wherewhen, where

Comprehension (translate, interpret)Comprehension (translate, interpret) ClassifyClassify

Application (apply knowledge in new Application (apply knowledge in newsituation)situation) Solve, identify resultsSolve, identify results


6/20

Blooms TaxonomyBlooms Taxonomy

Key WordsKey Words Analysis (breaking down into parts) Analysis (breaking down into parts) Relationships, main ideas, relevance,Relationships, main ideas, relevance,

conclusionsconclusionsSynthesis (combine elements intoSynthesis (combine elements intosomething new)something new) Create, develop, formulateCreate, develop, formulateEvaluation (evaluate by a set form of Evaluation (evaluate by a set form of criteria)criteria) Defend, criticize, judgeDefend, criticize, judge


7/20


8/20

MultipleMultiple--Choice ItemChoice Item- -Writing GuidelinesWriting Guidelines

Content ConcernsContent Concerns 1. Every item should reflect specific content and1. Every item should reflect specific content and

a single behavior a single behavior 2. Base each item on important content, no2. Base each item on important content, no

trivial contenttrivial content 3. Use novel material to test for higher 3. Use novel material to test for higher- -learninglearning

4. Keep content of each item independent from4. Keep content of each item independent fromcontent of other items on testcontent of other items on test


9/20


Content ConcernsContent Concerns 5. Avoid over specific or over generalized5. Avoid over specific or over generalized

contentcontent 6. Avoid opinion based items6. Avoid opinion based items 7. Avoid trick items7. Avoid trick items 8. Keep vocabulary simple for student group8. Keep vocabulary simple for student group


10/20


Formatting ConcernsFormatting Concerns 9. Use the question, completion, and best9. Use the question, completion, and best

answer versions of the conventional MC itemanswer versions of the conventional MC item 10. Format the item vertically instead of 10. Format the item vertically instead of

horizontallyhorizontally


11/20


Style ConcernsStyle Concerns 11. Edit and proof items11. Edit and proof items 12. Use correct grammar, punctuation,12. Use correct grammar, punctuation,

capitalization, and spellingcapitalization, and spelling 13. Minimize the amount of reading in each item13. Minimize the amount of reading in each item


12/20


Writing the stemWriting the stem 14. Ensure directions in stem are clear 14. Ensure directions in stem are clear 15. Include the central idea in the stem15. Include the central idea in the stem 16. Avoid window dressing (excess verbiage)16. Avoid window dressing (excess verbiage) 17. Word the stem positively, avoid negatives17. Word the stem positively, avoid negatives

such as NOT or EXCEPT if at all possible. If such as NOT or EXCEPT if at all possible. If

negatives must be used, BOLD/CAPITALIZEnegatives must be used, BOLD/CAPITALIZE


13/20


Writing the choicesWriting the choices 18. Develop effective choices (3 is adequate)18. Develop effective choices (3 is adequate) 19. Ensure only one right answer 19. Ensure only one right answer 20. Vary the location of the right answer 20. Vary the location of the right answer 21. Place choices in logical or numerical order 21. Place choices in logical or numerical order 22. Keep choices independent22. Keep choices independent


14/20


Writing the choicesWriting the choices 23. Keep choices homogeneous in content and23. Keep choices homogeneous in content and

grammar grammar 24. Keep the length of the choices about equal24. Keep the length of the choices about equal 25. None25. None- -of of--thethe--above should be used carefullyabove should be used carefully 26. Avoid All26. Avoid All- -of of--thethe--aboveabove

27. Phrase choices positively, avoid negatives27. Phrase choices positively, avoid negativessuch as NOTsuch as NOT


15/20


Writing the choicesWriting the choices 28. Avoid giving clues to the right answer, such28. Avoid giving clues to the right answer, such

asasa. specific determiners including always, never, ecta. specific determiners including always, never, ectb. choices identical to or resembling the stemb. choices identical to or resembling the stemc. grammatical inconsistencies that cue correctc. grammatical inconsistencies that cue correctchoicechoice

d. conspicuous correct choiced. conspicuous correct choicee. pairs of options that cue correct choicee. pairs of options that cue correct choicef. absurd, ridiculous optionsf. absurd, ridiculous options


16/20


Writing the choicesWriting the choices 29. Make all distractors plausible29. Make all distractors plausible 30. Use typical errors of students to write30. Use typical errors of students to write

distractorsdistractors 31. Use humor if it is compatible with the31. Use humor if it is compatible with the

teacher and the learning environmentteacher and the learning environment


17/20

Final Assessment GuidelinesFinal Assessment Guidelines

To increase knowledge level of students, useTo increase knowledge level of students, useupper levels of Blooms taxonomyupper levels of Blooms taxonomy Analysis Analysis

SynthesisSynthesis EvaluationEvaluationDevelop discriminating itemsDevelop discriminating items Items every misses or answers correctly does notItems every misses or answers correctly does not

discriminate between knowledge levelsdiscriminate between knowledge levels

Review items after each administrationReview items after each administration Ensure items assess learning targetsEnsure items assess learning targets Ensure items provide proper student inferencesEnsure items provide proper student inferences


18/20

ConclusionConclusion

Textbook assessments (NOT GOOD)Textbook assessments (NOT GOOD)Blooms taxonomy (all 6 areas areBlooms taxonomy (all 6 areas are

important, but strive for the top 3)important, but strive for the top 3)ItemItem--writing guidelines (your guidebook for writing guidelines (your guidebook for success in assessment)success in assessment)


19/20

Measure your students learning againstMeasure your students learning againstyour learning targets, not the targets of your learning targets, not the targets of

some textbook publisher some textbook publisher

Questions?Questions?

(Click(Click BackBack to return to the Documentation)to return to the Documentation)


20/20

Reference ListReference List Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R.,Raths, J., & Wittrock, M. C. (2001).Raths, J., & Wittrock, M. C. (2001). A taxonomy for learning, teaching, and assessing: A revision of A taxonomy for learning, teaching, and assessing: A revision of Blooms taxonomy of educational objectivesBlooms taxonomy of educational objectives . New York: Longman.. New York: Longman.Bloom, B. S. (1976).Bloom, B. S. (1976). H uman characteristics and school learning H uman characteristics and school learning . New York: McGraw. New York: McGraw- -Hill.Hill.Bloom, B. S., & et al. (eds) (1956).Bloom, B. S., & et al. (eds) (1956). T axonomy of educational objectivesT axonomy of educational objectives . New York: David McKay. New York: David McKayCompany.Company.Frisbie, D. A., Miranda, D. U., & Baker, K. K. (1993). An evaluation of elementary textbook tests asFrisbie, D. A., Miranda, D. U., & Baker, K. K. (1993). An evaluation of elementary textbook tests asclassroom assessment tools.classroom assessment tools. Applied Measurement in Education, 6(1) Applied Measurement in Education, 6(1) , 21, 21--36.36.Haladyna, T. M., & Downing, S. M. (1989). A taxonomy of multipleHaladyna, T. M., & Downing, S. M. (1989). A taxonomy of multiple- -choice itemchoice item- -writing rules.writing rules. Applied Applied measurement in education, 2(1)measurement in education, 2(1) , 37, 37--50.50.Haladyna, T. M., Downing, S. M. & Rodriquez, M. C. (2002). A review of multipleHaladyna, T. M., Downing, S. M. & Rodriquez, M. C. (2002). A review of multiple- -choice itemchoice item- -writingwritingguidelines for classroom assessment.guidelines for classroom assessment. Applied Measurement in Education, 15(3) Applied Measurement in Education, 15(3) , 309, 309--334.334.Kehoe, J. (1995). Writing multipleKehoe, J. (1995). Writing multiple- -choice test items.choice test items. P ractical Assessment, Research & Evaluation,P ractical Assessment, Research & Evaluation,4(9).4(9).Kehoe, J. (1995). Basic item analysis for multipleKehoe, J. (1995). Basic item analysis for multiple- -choice tests.choice tests. P ractical Assessment, Research &P ractical Assessment, Research &Evaluation, 4(10).Evaluation, 4(10).North Carolina Department of Public Instruction. (1999). Understanding North Carolina tests: ThinkingNorth Carolina Department of Public Instruction. (1999). Understanding North Carolina tests: Thinkingskills level.skills level. Assessment brief Assessment brief , Vol 6, No 4., Vol 6, No 4.Office Port. (n.d.).Office Port. (n.d.). Blooms taxonomy Blooms taxonomy . Retrieved February 28, 2005, from. Retrieved February 28, 2005, fromhttp://www.officeport.com/edu/blooms.htmhttp://www.officeport.com/edu/blooms.htmPopham, W. J. (2005).Popham, W. J. (2005). C lassroom assessment: What teachers need to know. 4th ed.C lassroom assessment: What teachers need to know. 4th ed. Boston:Boston:Pearson Education.Pearson Education.University of Texas Learning Center. (n. d.). Blooms taxonomys model questions and key words.University of Texas Learning Center. (n. d.). Blooms taxonomys model questions and key words.Retrieved February 28, 2005, fromRetrieved February 28, 2005, from http://www.utexas.edu/student/utlc/handouts/1414.htmlhttp://www.utexas.edu/student/utlc/handouts/1414.html

textbook assessment items

Documents