using naep assessment items instructionally: teaching students to be strategic jeanne foy, naep...
TRANSCRIPT
Using NAEP Assessment Items Instructionally: Teaching Students to Be Strategic
Jeanne Foy, NAEP State Coordinator
Part I: Introduction to Using Assessment Items 2
Organization Part I: Introduction to Using Assessment Items Part II: Why Use Assessment Items for
Instruction? Part III: Web Resources
NAEP Released Items NAEP Assessment Toolbox
Part IV: NAEP Assessment Toolbox Strategies Reading Math
Part V: Alaska Standards Based Assessment Resources
Part I: Introduction to Using Assessment Items 3
Part I
Introduction to Using Assessment Items
Part I: Introduction to Using Assessment Items 4
Using Assessment Items
Wealth of resources available through released National Assessment of Educational Progress (NAEP) questions
Released items can be used instructionally Teach to the standard Teach students strategic reasoning skills
Part I: Introduction to Using Assessment Items 5
What Is the National Assessment of Educational Progress (NAEP)?
The nation’s only nationally representative, continuous assessment of what the nation’ s students know and can do in school
Administered in Alaska as part of NCLB every other year since 2003 for Grade 4 mathematics and reading Grade 8 mathematics and reading
Part I: Introduction to Using Assessment Items 6
Benefits of Using These Resources Alignment with state standards/GLEs Opportunity for students to learn state
standards/GLEs Excellent examples of assessment items Activities that can be used to augment or
supplement what teachers are already doing
Easy activities to “drop in” along with other classroom activities (graded assignments, performance-based projects, etc.)
Part I: Introduction to Using Assessment Items 7
Formative Assessments
Used to gain immediate information on how students are learning and information is used to adapt instruction; students are also aware of their learning process
All activities using NAEP items are intended for formative assessments
Part II: Why Use Assessment Items for Instruction? 8
Part II
Why Use Assessment Items?
Part II: Why Use Assessment Items for Instruction? 9
“By focusing on important standards and using tools such as sample tests, teachers can help their students understand what they need to learn—and what they will be tested on.”
Results Now: How We Can Achieve Unprecedented Improvements in Teaching and Learning. Alexandria, VA: Association for Supervision and Curriculum Development. Schmoker, Mike (2006).
Teach Essential Skills
Part II: Why Use Assessment Items for Instruction? 10
Teach to the Standard “The prevalence of higher-order standards
surprises many educators but is borne out by a recent review of state assessments that found that almost all of the items on these tests—an encouraging trend—are higher-order and inferential in nature (Liben & Liben, 2005). . . .”
Results Now: How We Can Achieve Unprecedented Improvements in Teaching and Learning. Alexandria, VA: Association for Supervision and Curriculum Development. Schmoker, Mike (2006).
Part II: Why Use Assessment Items for Instruction? 11
Model Strategic Thinking “. . . studies suggest that many students of
diverse backgrounds are not receiving the kind of comprehension instruction that would prepare them well on assessments that are increasingly oriented toward higher level thinking with text. It is clear from research that all students need instruction in reading instruction, especially the kind that focuses on the strategies required to answer and generate challenging questions.”
Raphael, T. E., & Au, K. H. (2005). QAR: Enhancing comprehension and test taking across grades and content areas. The Reading Teacher, 59 (3), 206-221.
Part II: Why Use Assessment Items for Instruction? 12
“. . . to many educators and policymakers, NAEP represents the gold standard in testing for its ability to assess both content and critical thinking.”
— Patte Barth, Director of the Center for Public Education
Value of NAEP
Part II: Why Use Assessment Items for Instruction? 13
Items Are Meant for Learning Activities
Ethical test preparation practices do not include “providing students with extended practice on old or parallel forms of the test without guided practice on how to improve”
Classroom Assessment for Student Learning: Doing It Right—Using It Well. Portland, OR: Educational Testing Service. Stiggins, R. J., Arter, J. A., Chappius, J., & Chappius, S. (2006)
Part III: Web Resources 14
Part III
Web Resources NAEP Released Items Aligned to State
Standards NAEP Assessment Toolbox for
Teachers
Part III: Web Resources 15
For NAEP teacher resources, go to state assessment web page: www.eed.state.ak.us/tls/assessment/
Part III: Web Resources 16
Teacher NAEP resources on Alaska NAEP web page: http://www.eed.state.ak.us/tls/assessment/naep.html
Part I: Introduction to Using Assessment Items 17
Bank of NAEP Items Linked to Alaska GLEs
Part III: Web Resources 18
Table of Contents for Each Math and Reading Strand
Table shows type of question
Performance data for Alaska students
Quick description of question
Part III: Web Resources 19
NAEP Questions Linked to Alaska GLEs Posted in Word
Part III: Web Resources 20
What Accompanies NAEP Assessment Items? Multiple-choice and constructed-
response questions that have been field tested
Score guides for constructed-response questions
Tables identifying GLEs assessed by each item
Student exemplars for every score level for constructed-response questions
Student performance data
Part III: Web Resources 21
A note on multiple-choice questions
A common misunderstanding is that multiple-choice questions cannot assess reasoning proficiency. Patterns of reasoning such as comparative reasoning and various types of inference (generalizing, author’s purpose, main idea) can be assessed in selected response format
Classroom Assessment for Student Learning: Doing It Right—Using It Well. Portland, OR: Educational Testing Service. Stiggins, R. J., Arter, J. A., Chappius, J., & Chappius, S. (2006)
Part III: Web Resources 22
NAEP Assessment Toolbox for Teachers
Illustrates how items can be used instructionally
Worksheet format Models state standards Overall objective: short, guided
practice in strategic thinking
Part IV: NAEP Assessment Toolbox Strategies 23
Part IV:
NAEP Assessment Toolbox Strategies
Part IV: NAEP Assessment Toolbox Strategies 24
NAEP Assessment Toolbox Strategies
Reading—focus on reasoning strategies and ability to evaluate/self-assess
Mathematics—students’ pattern of reasoning and errors, communication of mathematics vocabulary and concepts
Part IV: NAEP Assessment Toolbox Strategies 25
NAEP Data Explanation (p. 3*)
*Page numbers in headings refer to Assessment Toolbox pages
Part IV: NAEP Assessment Toolbox Strategies/Reading 26
Improve Students’ Ability to Evaluate Quality of Work Teaching students to be able to assess
the quality of work is key for students to learn how to revise their own work.
Using NAEP Constructed-Response Questions and Scoring Guides To Identify Acceptable and Unacceptable Responses to Questions on a Reading Passage The following worksheets on “Watch Out for
Wombats,” from a grade 4 NAEP reading assessment show how NAEP student samples of work and NAEP scoring guides can be used to help students identify acceptable and unacceptable answers to reading questions.
Part IV: NAEP Assessment Toolbox Strategies/Reading 27
Overview of Questions (p. 9)
Part IV: NAEP Assessment Toolbox Strategies/Reading 28
Reading Passage (p. 11)
Part IV: NAEP Assessment Toolbox Strategies/Reading 29
Student Self-Assessment Worksheet (p. 13)
Part IV: NAEP Assessment Toolbox Strategies/Reading 30
Answer Key (p. 15)
Part IV: NAEP Assessment Toolbox Strategies/Reading 31
Variations (covered in introduction to section on pages 7 & 8)
Discuss only one or two questions at a time or whatever seems appropriate
Give students only unacceptable responses to revise to make them acceptable
Classroom discussion and scoring of responses
Part IV: NAEP Assessment Toolbox Strategies/Reading 32
Guided Practice/Reasoning Strategies Introduction (p. 59)
Released NAEP questions offer teachers many opportunities to use guided practice with students to show how to use reasoning strategies to answer questions over a reading passage.
Part IV: NAEP Assessment Toolbox Strategies/Reading 33
Reasoning Strategies National Reading Panel findings on
Teacher Preparation and Comprehension Strategies Instruction conclude that “reading comprehension can be improved by teaching students to use specific cognitive strategies or to reason strategically when they encounter barriers to comprehension when reading.”
Part IV: NAEP Assessment Toolbox Strategies/Reading 34
Reading Panel Findings
Comprehension strategies include a teacher guiding the reader or modeling for the reader the actions that the reader can take to enhance the comprehension processes used during reading and the reader practicing those strategies with the teacher assisting until the reader achieves a gradual internalization and independent mastery of those processes.
Part IV: NAEP Assessment Toolbox Strategies/Reading 35
Procedures for Modeling Reasoning Strategies (p. 59)
Print the reading passage and questions. Ask students to complete these on their own.
After students answer the questions on their own, share the item map and reasoning strategies worksheets with students.
Use the reasoning strategies worksheets to model how to interpret and answer each question on the item map correctly with the entire class. These worksheets address the following questions: What does the item map show about this
item? How do you know your answer is correct? Why are the other responses wrong or
incomplete?
Part IV: NAEP Assessment Toolbox Strategies/Reading 36
Reading Passage (p. 63)
Part IV: NAEP Assessment Toolbox Strategies/Reading 37
Questions on Passage (p. 63-65)
Part IV: NAEP Assessment Toolbox Strategies/Reading 38
NAEP Item Map (p. 67)
Part IV: NAEP Assessment Toolbox Strategies/Reading 39
Student Worksheet (p. 73)
Students asked to rephrase GLE
National data and Alaska data
Percentage of students who chose each possible answer
Part IV: NAEP Assessment Toolbox Strategies/Reading 40
Questions for Class Discussion (p. 74)
Part IV: NAEP Assessment Toolbox Strategies/Reading 41
Constructed-Response Question Discussion (p. 69 & 70)
Part IV: NAEP Assessment Toolbox Strategies/Reading 42
Scorer’s commentary for NAEP questions reinforce that students must support their opinion with evidence from the text.
Part IV: NAEP Assessment Toolbox Strategies/Reading 43
NAEP Grade Levels
NAEP assessments given at three grade levels: 4, 8, and 12
Assessment Toolbox has grade 4 and grade 8 assessment activities
For use in other grades, teachers can gauge difficulty of questions by performance data
Part IV: NAEP Assessment Toolbox Strategies/Reading 44
Available NAEP Reading Questions NAEP has many constructed-
response questions NAEP uses a variety of fiction and
nonfiction NAEP performance data can serve
as benchmark for comparison on how students are doing, both to other students in Alaska and nationally
Part IV: NAEP Assessment Toolbox Strategies/Reading 45
Discussion Questions
How can assessment items be used for cooperative learning?
How can assessment items be used to involve students in their own learning?
Part IV: NAEP Assessment Toolbox Strategies/Math 46
Examining Math Understanding (p.116)
Examples of how to use NAEP multiple-choice questions for one math strand each at grade 4 and grade 8
Multiple-choice distractors carefully designed; must be plausible indicators of student thinking
Peformance data shows how common “mathematical misconceptions” are
Part IV: NAEP Assessment Toolbox Strategies/Math 47
Grade 8 F&R Question (p. 131)
GLE: F&R-5 translating a written phrase to an algebraic expression
8. If n represents an even number greater than 2, what is the next larger even number? A) n + 1B) 2n + 1C) 2nD) n + 2E) n2
Part IV: NAEP Assessment Toolbox Strategies/Math 48
Mathematical Error Revealed by Each Wrong Response (p. 132)
A: This expression would create an odd number.
B: This expression would create an odd number; also, 2n would not create
the next larger even number.C: This would create an even number, but
not the next larger even number.D: Correct answerE: This would create an even number, but
not the next larger even number.
Part IV: NAEP Assessment Toolbox Strategies/Math 49
Checking for Understanding by Using Response Cards
Low-cost, available materials Technique that can be used for any
content, for open-ended questions or multiple-choice questions
Model strategies to address performance gap for “nonroutine” problems
Part IV: NAEP Assessment Toolbox Strategies/Math 50
“Moment of Contingency” “To gauge the understanding of the whole
class, the teacher needs to get responses from all the students in real time.
One way to do this is to have all students write their answers on individual dry-erase boards, which they hold up at the teacher's request. The teacher can then scan responses for novel solutions as well as misconceptions.”Leahy, S., Lyon, C., Thompson, M., & Wiliam, D. (2005). Classroom
Assessment: Minute by Minute, Day by Day [Electronic version]. Educational Leadership: Assessment to Promote Learning 63 (30, 19-24)
Part IV: NAEP Assessment Toolbox Strategies/Math 51
Responding to Student Understanding
“Another approach is to give each student a set of four cards labeled A, B, C, and D, and ask the question in multiple-choice format. If the question is well designed, the teacher can quickly judge the different levels of understanding in the class. If all students answer correctly, the teacher can move on. If no one answers correctly, the teacher might choose to reteach the concept. If some students answer correctly and some answer incorrectly, the teacher can use that knowledge to engineer a whole-class discussion on the concept or match up the students for peer teaching.”
Part IV: NAEP Assessment Toolbox Strategies/Math 52
Math Achievement Gap “Another pattern evident in NAEP data is that
low-SES and minority students tend to perform worse on nonroutine problems. For example, the vast majority of 4th grade and 8th grade students from all race and class groups correctly answered basic computation problems, such as 238 + 462 = __. In contrast, there were large disparities on computation problems with extraneous information or multiple steps, such as this one:
Carl has 3 empty egg cartons and 34 eggs. If each carton holds 12 eggs, how many more eggs are needed to fill all 3 cartons?”
Part IV: NAEP Assessment Toolbox Strategies/Math 53
Math Performance Gaps “Carl has 3 empty egg cartons and 34
eggs. If each carton holds 12 eggs, how many more eggs are needed to fill all 3 cartons?
More than 53 percent of white, Asian, and nonpoor 4th graders answered this problem correctly, compared with only about one-third of black, Latino, and poor 4th graders.”
Lubienski, S. (2007). What We Can Do About Achievement Disparities [Electronic version]. Educational Leadership: Making Math Count 65 (3, 54-59).
Part IV: NAEP Assessment Toolbox Strategies/Math 54
Use Assessment Items to Build Math Vocabulary
“The meanings of words in math—for example, even, odd, product, and factor—often differ from their use in common language. Many students needing math intervention have weak mathematical vocabularies. It's key that students develop a firm understanding of mathematical concepts before learning new vocabulary, so that they can anchor terminology in their understanding. We should explicitly teach vocabulary in the context of a learning activity and then use it consistently. A math vocabulary chart can help keep both teacher and students focused on the importance of accurately using math terms.”
Burns, M. (2007). Nine Ways to Catch Kids Up [Electronic version]. Educational Leadership: Making Math Count 65 (3, 16-21).
Part IV: NAEP Assessment Toolbox Strategies/Math 55
Math GLEs Can Serve as Vocabulary Chart
Part IV: NAEP Assessment Toolbox Strategies/Math 56
Conclusion
NAEP Assessment Items Illustrate the GLEs for Students Turn abstract idea into learning
activity Clarify expectations One way of students taking
responsibility for own learning
Part I: Introduction to Using Assessment Items 57
NAEP Questions Available in Many Content Areas at http://nces.ed.gov/nationsreportcard/itmrls/
Part V: Alaska SBA Resources 58
Part V
Alaska Standards Based Assessment Resources
Part I: Introduction to Using Assessment Items 59
SBA Practice Tests Assessment Items
Part I: Introduction to Using Assessment Items 60
Scoring Guide is key. For example, look at Grade 6 Reading.
Part V: Alaska SBA Resources 61
Last page of the Scoring Guide has test map showing the GLE assessed by each reading question.
Part I: Introduction to Using Assessment Items 62
Send questions or comments to
Jeanne Foy, Alaska NAEP [email protected]