Assessment Items

Download Assessment Items

Post on 14-Jul-2015

551 views

Category:

Education

6 download

Embed Size (px)

TRANSCRIPT

<p>Guidelines for Writing Multiple-Choice Assessment Items</p> <p>Developing High Quality Assessment ItemsRegion 7 Digital Learning Academy1</p> <p>Dr. LaTricia TownsendResearch ScholarEducational EvaluationNCSU Friday Institute</p> <p>@tricia_townsend</p> <p>http://region7dla.wikispaces.com/</p> <p>Looking BackActivity-Table Share: Use of Session 1 Information Reflection How have you used information gained from Session 1 in your practice?How have you shared the information from Session 1 in your school or district? Looking ForwardTake the poll about how you obtain assessment items</p> <p>Item development process</p> <p>Curriculum StandardsCurriculum Standards- formerly Standard Course of Study; Common Core, Essential Standards, Next Generation Stds.</p> <p>We must be clear about what students need to know and be able to do, and we need to be skillful in how we teach it. To be certain that students have learned what we intend for them to learn, we must develop balanced assessment systems that provide data on how students are progressing, systems that also use the classroom assessment process to promote even greater learning. What we teach, how we teach it, how well we assess, and how we use assessment to improve student achievement will all determine the success of our efforts, and ultimately, how we are judged.6Alignment7Access Curricular DocumentsActivity- Take a moment to locate curriculum standards materialsCommon CoreELA and MathEssential StandardsNext Generation Science StandardsUnpacking documents completed by school/districtCurriculum Maps/Pacing GuidesBegin to think about which ones you want to address today</p> <p>Revised Blooms Taxonomy-Knowledge Process DimensionFACTUAL KNOWLEDGE-basic elements students must know to be acquainted with a discipline or solve problems in it</p> <p>CONCEPTUAL KNOWLEDGE-The interrelationships among the basic elements within a larger structure that enable them to function together</p> <p>PROCEDURAL KNOWLEDGE-How to do something, methods of inquiry, and criteria for using skills, algorithms, techniques, and methods</p> <p>METACOGNITIVE KNOWLEDGE-Knowledge of cognition in general as well as awareness and knowledge of one's own cognition</p> <p>FACTUAL-Aar. Knowledge of terminology; Technical vocabulary, music symbolsAb. Knowledge of specific details and elements; Major natural resources, reliable sources of information</p> <p>CONCEPTUAL-Ba. Knowledge of classifications and categories; Periods of geological time, forms of business ownership| Bb. Knowledge of principles and generalizations; Pythagorean theorem, law of supply and demand | BSc. Knowledge of theories, models, and structures; Theory of evolution, structure of Congress</p> <p>PROCEDURAL-Ca. Knowledge of subject-specific skills and algorithmsSkills used in painting with water colors, whole-number division algorithmCb. Knowledge of subject-specific techniques and methodsInterviewing techniques, scientific methodCc. Knowledge of criteria for determining when to use appropriate proceduresCriteria used to determine when to apply a procedure involving Newton's second law, criteria used to judge the feasibility of using a particular method to estimate business costs</p> <p>METACOGNITIVE-Da. Strategic knowledgeKnowledge of outlining as a means of capturing the structure of a unit of subject matter in a text book, knowledge of the use of heuristicsDb. Knowledge about cognitive tasks, including appropriate contextual and conditional knowledgeKnowledge of the types of tests particular teachers administer, knowledge of the cognitive demands of different tasksDc. Self-knowledgeKnowledge that critiquing essays is a personal strength, whereas writing essays is a personal weakness; awareness of one's own knowledge level</p> <p>9</p> <p>Revised Blooms Taxonomy-Cognitive Process DimensionRevised Blooms Taxonomy According to Homer Simpson </p> <p>Thinking Skills!!!</p> <p>We mustremembera concept beforewe canunderstandit.We mustunderstanda concept before we canapplyit.We must be able toapplya concept before weanalyzeit.We must haveanalyzeda concept before we canevaluateit.We must haveremembered, understood, applied, analyzed,andevaluated a concept before we cancreate.10Webbs Depth of KnowledgeLevel 1: RecallRecall a fact, information, or procedureLevel 2: Skill/Concept Use information or conceptual knowledge, two or more stepsLevel 3: Strategic Thinking Requires reasoning, developing a plan or a sequence of steps, some complexity Level 4: Extended ThinkingRequires an investigation, time to think and process multiple conditions of the problem. </p> <p>12Practice AlignmentActivity-Write items (open ended- that align to your content area and Revised Blooms Taxonomy/Webbs Depth of Knowledge)Write four questions to address each of the four levels. Record your responses using the tool of your choice for your own records.Share your responses on the Google Spreadsheet 13Practice AlignmentActivity-Find a partner who teaches/supports in the same grade span in a different district. Share your questions and discuss how you might approach the content differently.</p> <p>Reflect #R7DLA</p> <p>14Multiple Choice Item Guidelines15Multiple Choice TerminologyWhich is the state bird of North Carolina?A.bluebirdB.cardinalC.robinD.sparrow</p> <p>StemDistractorFoilsDistractorKeyDistractorMultiple Choice Terminology</p> <p>Passage</p> <p>http://www.ncpublicschools.org/docs/accountability/testing/releasedforms/g3elapp.pdfMultiple Choice Terminology</p> <p>Stimulus</p> <p>http://www.ncpublicschools.org/docs/accountability/testing/releasedforms/g5mathpp.pdfMultiple Choice Guidelines-ContentBase items on curriculum standards and DOK.Keep the testing vocabulary consistent with grade level.Measure a single concept, principle, procedure, or competency.Write items with a mixture of difficulty levels-Easy, Medium, Hard </p> <p>Multiple Choice Guidelines-ContentChoose to assess important information, not trivial material.</p> <p>POOR ITEMHow many different elements make up the Noble Gases? A. 3 B. 4 C. 5 D. 6</p> <p>IMPROVED ITEMWhat common property is shared by all the noble gases? A. They are highly reactive and must be stored in air-tight containers. B. They are inert, and do not easily form chemical compounds. C. They are metallic in nature, and have basic oxides. D. When mixed with water, they form a toxic blue-green solution. Multiple Choice Guidelines-GeneralTest the content, not the test-taking or reading ability of students-avoid tricky or misleading items.Avoid errors of grammar, abbreviations, punctuation and spelling. Do not use contractions.Do not use first or second person.Do not use humor.Multiple Choice Guidelines-FormatWrite clear, concise instructions.Format foils vertically.Use space in a balanced manner to produce an easy to follow format</p> <p>Multiple Choice Guidelines-StructureAvoid writing complex format multiple-choice items.Which of the following are fruits?1. Tomatoes2. Tomatillos3. Habanero peppersA. 1 &amp; 2B. 2 &amp; 3C. 1 &amp; 3D. 1, 2, &amp; 3Multiple Choice Guidelines-StructureDo not use fill-in-the blank format. Pose a question.Be brief . Wordiness in the stem can clue or confuse the student. (window-dressing)</p> <p>POOR ITEMHigh temperatures and heavy rainfall characterize a humid climate. People who live in this kind of climate usually complain of problems dealing with heavy amounts of perspiration. Even moderately warm days are uncomfortable for most people in this climate. Which biome is being described?A. savannaB. taigaC. tundraD. tropical rainforest</p> <p>IMPROVED ITEMWhich term describes a biome with high temperatures and heavy rainfall?A. savannaB. taigaC. tundraD. tropical rainforest Multiple choice guidelines-StemWord the stems positively. Do not use any negative phrasing, such as words NOT or EXCEPT.Do not write stems that ask for or express opinions.Write the items so that the central idea and any common elements are included in the stem instead of the foils.Place the interrogative as close to the foils as possible.Use qualifying words in a stem, when needed, to emphasize the best answer. Words such as best, most likely, and most appropriate are bold and italicized.</p> <p>Multiple Choice Guidelines- Foil DevelopmentEach item must contain four foils (A,B,C,D)Order the answer choices in a logical order, such as ascending or descending value for numbers, the order in which the words appear in a chart, chronological order, etc.Each item written should contain foils that are independent and not overlapping.To the extent possible, foils for an item should be homogeneous in content and length.</p> <p>Multiple Choice Guidelines- Foil DevelopmentDo not use the following as foils: all of the above, none of the above, I dont know.Word the foils positively, avoid any negative phrasing.Avoid providing clues to the correct response.Avoid writing items in which phrases in the stem are repeated in the foils.Multiple Choice Guidelines- Foil DevelopmentAvoid absolutes such as ALWAYS, NEVER, TOTALLY, and ABSOLUTELY should not be used when writing items.</p> <p>Qualifiers such as best, most likely, approximately, etc. should be bold and italic.</p> <p>Present one and only one best (correct) answer.</p> <p>Use the common errors made by students as distractors.</p> <p>Multiple Choice Guidelines- Foil DevelopmentUse plausible distractors. The best (correct) answer must clearly be the best (correct) answer and the incorrect responses must clearly be inferior to the best (correct) answer. No distractor should be obviously wrong.POOR ITEMWhich example best describes the adhesion property of water?A. drops of water on the outside of a glassB. sugar dissolving in waterC. ice floating in waterD. melting of aluminumMultiple Choice Guidelines- Foil DevelopmentUse plausible distractors. IMPROVED ITEMWhich example best describes the adhesion property of water?A. drops of water on the outside of a glassB. sugar dissolving in waterC. ice floating in waterD. water changing from a liquid to a gas Multiple Choice Guidelines-Universal designMake items accessible and usable by students of all abilities, including students with disabilities and students with limited English proficiency.Use simple language and sentence structure.Do not use colloquial language.</p> <p>The concept of Universal Design originated in architecture with the goal to provide maximum accessibility and usability of buildings, outdoor spaces, and living environments. This concept centered on the belief that our environments should be accessible and usable by everyone regardless of their age, ability, or circumstance31Multiple Choice Guidelines-Universal designLimit the use of homonyms, homophones, and homographsfair1. Adjective. Meaning lovely or beautiful, when describing someone2. Adjective. Meaning of light complexion, or light colored (blonde) hair3. Adjective. Meaning reasonable, equitable, free from self-interest3. Noun. Meaning a gathering of people to sell and barter goods, or to enjoy entertainment, and also a travelling show of rides, stalls and sideshows</p> <p>The concept of Universal Design originated in architecture with the goal to provide maximum accessibility and usability of buildings, outdoor spaces, and living environments. This concept centered on the belief that our environments should be accessible and usable by everyone regardless of their age, ability, or circumstance32Sensitivity and BiasDo not use stereotypes based on gender, race, religion, socio-economic status, or other factors.Respect diversity by including people from all walks of life in use of names and artwork.Ensure items do not highlight socio-economic disparities.Avoid espousing religious and political beliefs.33Revising and writing questionsActivity-- Would you change any of the questions that you posed, initially? Make any necessary changes. </p> <p>Create 3 Multiple Choice Questions- Record the questions using the same tool you used before for your personal use and share your responses on the Google SpreadsheetReflect #R7DLA</p> <p>34Assessing itemsActivity- Review Questions Find a trioUse the item review document to gauge the quality of each of the itemsWhat changes would you make to the items?</p> <p>Reflect #R7DLA</p> <p>Got Questions</p> <p>Source:http://en.clipart-fr.comDrop them anywhere!</p> <p>School NetActivity- Tasks in School NetAccess the User GuideLog into School NetTry these:Create an Item in School NetCreate an Express TestCreate a Test using items from Item Central</p> <p>Reflect #R7DLA</p> <p>Work tasksExplore online resources</p> <p>Brainstorm/Develop Culminating Resource</p> <p>Reflect #R7DLA</p> <p>Online Resources/School Net</p> <p>Activity- Explore the resources presented. How could these be used in your instructional delivery or assessment program? Are there other resources you would like to share? Share your ideas to Padlet.</p> <p>Final Product Idea Brainstorm/Work SessionActivity- Table Talk-With your team, discuss what your product for School Net will be at the culmination of DLA. What is the proposed product? How do you anticipate incorporating future information? How will it be disseminated in your school/district?Develop Concrete Next Steps</p> <p>Share your ideas to wiki Padlet. Each of you came to the session with an idea of how would apply the content to your practice-Improve existing assessments-Begin the process for creating new benchmark tests40http://region7dla.wikispaces.com/</p> <p>41Canyon WrenCornell Lab Of OrnithologyVoices of Song Birds, track 2/112011Natural Sounds29910.42eng - iTunPGAP0eng - iTunNORM 000041F5 00000000 000291D1 00000000 00005534 00000000 00007C2B 00000000 00005463 00000000eng - iTunSMPB 00000000 00000210 00000970 0000000000141500 00000000 00074054 00000000 00000000 00000000 00000000 00000000 00000000eng - Catherpes mexicanusSong; Location callML 120253; 9042Geoffrey A. Keller; L. Irby Davis, William GuionCanyon WrenCornell Lab Of OrnithologyVoices of Song Birds, track 2/112011Natural Sounds29910.42eng - iTunPGAP0eng - iTunNORM 000041F5 00000000 000291D1 00000000 00005534 00000000 00007C2B 00000000 00005463 00000000eng - iTunSMPB 00000000 00000210 00000970 0000000000141500 00000000 00074054 00000000 00000000 00000000 00000000 00000000 00000000eng - Catherpes mexicanusSong; Location callML 120253; 9042Geoffrey A. Keller; L. Irby Davis, William GuionCanyon WrenCornell Lab Of OrnithologyVoices of Song Birds, track 2/112011Natural Sounds29910.42eng - iTunPGAP0eng - iTunNORM 000041F5 00000000 000291D1 00000000 00005534 00000000 00007C2B 00000000 00005463 00000000eng - iTunSMPB 00000000 00000210 00000970 0000000000141500 00000000 00074054 00000000 00000000 00000000 00000000 00000000 00000000eng - Catherpes mexicanusSong; Location callML 120253; 9042Geoffrey A. Keller; L. Irby Davis, William GuionCanyon WrenCornell Lab Of OrnithologyVoices of Song Birds, track 2/112011Natural Sounds29910.42eng - iTunPGAP0eng - iTunNORM 000041F5 00000000 000291D1 00000000 00005534 00000000 00007C2B 00000000 00005463 00000000eng - iTunSMPB 00000000 00000210 00000970 0000000000141500 00000000 00074054 00000000 00000000 00000000 000000...</p>

Recommended

View more >