using naep assessment items instructionally: teaching students to be strategic jeanne foy, naep...

Using NAEP Assessment Items Instructionally: Teaching Students to Be Strategic

Jeanne Foy, NAEP State Coordinator

Part I: Introduction to Using Assessment Items 2

Organization Part I: Introduction to Using Assessment Items Part II: Why Use Assessment Items for

Instruction? Part III: Web Resources

NAEP Released Items NAEP Assessment Toolbox

Part IV: NAEP Assessment Toolbox Strategies Reading Math

Part V: Alaska Standards Based Assessment Resources

Part I: Introduction to Using Assessment Items 3

Part I

Introduction to Using Assessment Items

Part I: Introduction to Using Assessment Items 4

Using Assessment Items

Wealth of resources available through released National Assessment of Educational Progress (NAEP) questions

Released items can be used instructionally Teach to the standard Teach students strategic reasoning skills

Part I: Introduction to Using Assessment Items 5

What Is the National Assessment of Educational Progress (NAEP)?

The nation’s only nationally representative, continuous assessment of what the nation’ s students know and can do in school

Administered in Alaska as part of NCLB every other year since 2003 for Grade 4 mathematics and reading Grade 8 mathematics and reading

Part I: Introduction to Using Assessment Items 6

Benefits of Using These Resources Alignment with state standards/GLEs Opportunity for students to learn state

standards/GLEs Excellent examples of assessment items Activities that can be used to augment or

supplement what teachers are already doing

Easy activities to “drop in” along with other classroom activities (graded assignments, performance-based projects, etc.)

Part I: Introduction to Using Assessment Items 7

Formative Assessments

Used to gain immediate information on how students are learning and information is used to adapt instruction; students are also aware of their learning process

All activities using NAEP items are intended for formative assessments

Part II: Why Use Assessment Items for Instruction? 8

Part II

Why Use Assessment Items?

Part II: Why Use Assessment Items for Instruction? 9

“By focusing on important standards and using tools such as sample tests, teachers can help their students understand what they need to learn—and what they will be tested on.”

Results Now: How We Can Achieve Unprecedented Improvements in Teaching and Learning. Alexandria, VA: Association for Supervision and Curriculum Development. Schmoker, Mike (2006).

Teach Essential Skills

Part II: Why Use Assessment Items for Instruction? 10

Teach to the Standard “The prevalence of higher-order standards

surprises many educators but is borne out by a recent review of state assessments that found that almost all of the items on these tests—an encouraging trend—are higher-order and inferential in nature (Liben & Liben, 2005). . . .”

Results Now: How We Can Achieve Unprecedented Improvements in Teaching and Learning. Alexandria, VA: Association for Supervision and Curriculum Development. Schmoker, Mike (2006).

Part II: Why Use Assessment Items for Instruction? 11

Model Strategic Thinking “. . . studies suggest that many students of

diverse backgrounds are not receiving the kind of comprehension instruction that would prepare them well on assessments that are increasingly oriented toward higher level thinking with text. It is clear from research that all students need instruction in reading instruction, especially the kind that focuses on the strategies required to answer and generate challenging questions.”

Raphael, T. E., & Au, K. H. (2005). QAR: Enhancing comprehension and test taking across grades and content areas. The Reading Teacher, 59 (3), 206-221.

Part II: Why Use Assessment Items for Instruction? 12

“. . . to many educators and policymakers, NAEP represents the gold standard in testing for its ability to assess both content and critical thinking.”

— Patte Barth, Director of the Center for Public Education

Value of NAEP

Part II: Why Use Assessment Items for Instruction? 13

Items Are Meant for Learning Activities

Ethical test preparation practices do not include “providing students with extended practice on old or parallel forms of the test without guided practice on how to improve”

Classroom Assessment for Student Learning: Doing It Right—Using It Well. Portland, OR: Educational Testing Service. Stiggins, R. J., Arter, J. A., Chappius, J., & Chappius, S. (2006)

Part III: Web Resources 14

Part III

Web Resources NAEP Released Items Aligned to State

Standards NAEP Assessment Toolbox for

Teachers

Part III: Web Resources 15

For NAEP teacher resources, go to state assessment web page: www.eed.state.ak.us/tls/assessment/

Part III: Web Resources 16

Teacher NAEP resources on Alaska NAEP web page: http://www.eed.state.ak.us/tls/assessment/naep.html

Part I: Introduction to Using Assessment Items 17

Bank of NAEP Items Linked to Alaska GLEs

Part III: Web Resources 18

Table of Contents for Each Math and Reading Strand

Table shows type of question

Performance data for Alaska students

Quick description of question

Part III: Web Resources 19

NAEP Questions Linked to Alaska GLEs Posted in Word

Part III: Web Resources 20

What Accompanies NAEP Assessment Items? Multiple-choice and constructed-

response questions that have been field tested

Score guides for constructed-response questions

Tables identifying GLEs assessed by each item

Student exemplars for every score level for constructed-response questions

Student performance data

Part III: Web Resources 21

A note on multiple-choice questions

A common misunderstanding is that multiple-choice questions cannot assess reasoning proficiency. Patterns of reasoning such as comparative reasoning and various types of inference (generalizing, author’s purpose, main idea) can be assessed in selected response format

Classroom Assessment for Student Learning: Doing It Right—Using It Well. Portland, OR: Educational Testing Service. Stiggins, R. J., Arter, J. A., Chappius, J., & Chappius, S. (2006)

Part III: Web Resources 22

NAEP Assessment Toolbox for Teachers

Illustrates how items can be used instructionally

Worksheet format Models state standards Overall objective: short, guided

practice in strategic thinking

Part IV: NAEP Assessment Toolbox Strategies 23

Part IV:

NAEP Assessment Toolbox Strategies

Part IV: NAEP Assessment Toolbox Strategies 24

NAEP Assessment Toolbox Strategies

Reading—focus on reasoning strategies and ability to evaluate/self-assess

Mathematics—students’ pattern of reasoning and errors, communication of mathematics vocabulary and concepts

Part IV: NAEP Assessment Toolbox Strategies 25

NAEP Data Explanation (p. 3*)

*Page numbers in headings refer to Assessment Toolbox pages

Part IV: NAEP Assessment Toolbox Strategies/Reading 26

Improve Students’ Ability to Evaluate Quality of Work Teaching students to be able to assess

the quality of work is key for students to learn how to revise their own work.

Using NAEP Constructed-Response Questions and Scoring Guides To Identify Acceptable and Unacceptable Responses to Questions on a Reading Passage The following worksheets on “Watch Out for

Wombats,” from a grade 4 NAEP reading assessment show how NAEP student samples of work and NAEP scoring guides can be used to help students identify acceptable and unacceptable answers to reading questions.

Part IV: NAEP Assessment Toolbox Strategies/Reading 27

Overview of Questions (p. 9)

Part IV: NAEP Assessment Toolbox Strategies/Reading 28

Reading Passage (p. 11)

Part IV: NAEP Assessment Toolbox Strategies/Reading 29

Student Self-Assessment Worksheet (p. 13)

Part IV: NAEP Assessment Toolbox Strategies/Reading 30

Answer Key (p. 15)

Part IV: NAEP Assessment Toolbox Strategies/Reading 31

Variations (covered in introduction to section on pages 7 & 8)

Discuss only one or two questions at a time or whatever seems appropriate

Give students only unacceptable responses to revise to make them acceptable

Classroom discussion and scoring of responses

Part IV: NAEP Assessment Toolbox Strategies/Reading 32

Guided Practice/Reasoning Strategies Introduction (p. 59)

Released NAEP questions offer teachers many opportunities to use guided practice with students to show how to use reasoning strategies to answer questions over a reading passage.

Part IV: NAEP Assessment Toolbox Strategies/Reading 33

Reasoning Strategies National Reading Panel findings on

Teacher Preparation and Comprehension Strategies Instruction conclude that “reading comprehension can be improved by teaching students to use specific cognitive strategies or to reason strategically when they encounter barriers to comprehension when reading.”

Part IV: NAEP Assessment Toolbox Strategies/Reading 34

Reading Panel Findings

Comprehension strategies include a teacher guiding the reader or modeling for the reader the actions that the reader can take to enhance the comprehension processes used during reading and the reader practicing those strategies with the teacher assisting until the reader achieves a gradual internalization and independent mastery of those processes.

Part IV: NAEP Assessment Toolbox Strategies/Reading 35

Procedures for Modeling Reasoning Strategies (p. 59)

Print the reading passage and questions. Ask students to complete these on their own.

After students answer the questions on their own, share the item map and reasoning strategies worksheets with students.

Use the reasoning strategies worksheets to model how to interpret and answer each question on the item map correctly with the entire class. These worksheets address the following questions: What does the item map show about this

item? How do you know your answer is correct? Why are the other responses wrong or

incomplete?

Part IV: NAEP Assessment Toolbox Strategies/Reading 36

Reading Passage (p. 63)

Part IV: NAEP Assessment Toolbox Strategies/Reading 37

Questions on Passage (p. 63-65)

Part IV: NAEP Assessment Toolbox Strategies/Reading 38

NAEP Item Map (p. 67)

Part IV: NAEP Assessment Toolbox Strategies/Reading 39

Student Worksheet (p. 73)

Students asked to rephrase GLE

National data and Alaska data

Percentage of students who chose each possible answer

Part IV: NAEP Assessment Toolbox Strategies/Reading 40

Questions for Class Discussion (p. 74)

Part IV: NAEP Assessment Toolbox Strategies/Reading 41

Constructed-Response Question Discussion (p. 69 & 70)

Part IV: NAEP Assessment Toolbox Strategies/Reading 42

Scorer’s commentary for NAEP questions reinforce that students must support their opinion with evidence from the text.

Part IV: NAEP Assessment Toolbox Strategies/Reading 43

NAEP Grade Levels

NAEP assessments given at three grade levels: 4, 8, and 12

Assessment Toolbox has grade 4 and grade 8 assessment activities

For use in other grades, teachers can gauge difficulty of questions by performance data

Part IV: NAEP Assessment Toolbox Strategies/Reading 44

Available NAEP Reading Questions NAEP has many constructed-

response questions NAEP uses a variety of fiction and

nonfiction NAEP performance data can serve

as benchmark for comparison on how students are doing, both to other students in Alaska and nationally

Part IV: NAEP Assessment Toolbox Strategies/Reading 45

Discussion Questions

How can assessment items be used for cooperative learning?

How can assessment items be used to involve students in their own learning?

Part IV: NAEP Assessment Toolbox Strategies/Math 46

Examining Math Understanding (p.116)

Examples of how to use NAEP multiple-choice questions for one math strand each at grade 4 and grade 8

Multiple-choice distractors carefully designed; must be plausible indicators of student thinking

Peformance data shows how common “mathematical misconceptions” are

Part IV: NAEP Assessment Toolbox Strategies/Math 47

Grade 8 F&R Question (p. 131)

GLE: F&R-5 translating a written phrase to an algebraic expression

8. If n represents an even number greater than 2, what is the next larger even number?  A)  n + 1B)  2n + 1C)  2nD)  n + 2E)  n2

Part IV: NAEP Assessment Toolbox Strategies/Math 48

Mathematical Error Revealed by Each Wrong Response (p. 132)

A: This expression would create an odd number.

B: This expression would create an odd number; also, 2n would not create

the next larger even number.C: This would create an even number, but

not the next larger even number.D: Correct answerE: This would create an even number, but

not the next larger even number.

Part IV: NAEP Assessment Toolbox Strategies/Math 49

Checking for Understanding by Using Response Cards

Low-cost, available materials Technique that can be used for any

content, for open-ended questions or multiple-choice questions

Model strategies to address performance gap for “nonroutine” problems

Part IV: NAEP Assessment Toolbox Strategies/Math 50

“Moment of Contingency” “To gauge the understanding of the whole

class, the teacher needs to get responses from all the students in real time.

One way to do this is to have all students write their answers on individual dry-erase boards, which they hold up at the teacher's request. The teacher can then scan responses for novel solutions as well as misconceptions.”Leahy, S., Lyon, C., Thompson, M., & Wiliam, D. (2005). Classroom

Assessment: Minute by Minute, Day by Day [Electronic version]. Educational Leadership: Assessment to Promote Learning 63 (30, 19-24)

Part IV: NAEP Assessment Toolbox Strategies/Math 51

Responding to Student Understanding

“Another approach is to give each student a set of four cards labeled A, B, C, and D, and ask the question in multiple-choice format. If the question is well designed, the teacher can quickly judge the different levels of understanding in the class. If all students answer correctly, the teacher can move on. If no one answers correctly, the teacher might choose to reteach the concept. If some students answer correctly and some answer incorrectly, the teacher can use that knowledge to engineer a whole-class discussion on the concept or match up the students for peer teaching.”

Part IV: NAEP Assessment Toolbox Strategies/Math 52

Math Achievement Gap “Another pattern evident in NAEP data is that

low-SES and minority students tend to perform worse on nonroutine problems. For example, the vast majority of 4th grade and 8th grade students from all race and class groups correctly answered basic computation problems, such as 238 + 462 = __. In contrast, there were large disparities on computation problems with extraneous information or multiple steps, such as this one:

Carl has 3 empty egg cartons and 34 eggs. If each carton holds 12 eggs, how many more eggs are needed to fill all 3 cartons?”

Part IV: NAEP Assessment Toolbox Strategies/Math 53

Math Performance Gaps “Carl has 3 empty egg cartons and 34

eggs. If each carton holds 12 eggs, how many more eggs are needed to fill all 3 cartons?

More than 53 percent of white, Asian, and nonpoor 4th graders answered this problem correctly, compared with only about one-third of black, Latino, and poor 4th graders.”

Lubienski, S. (2007). What We Can Do About Achievement Disparities [Electronic version]. Educational Leadership: Making Math Count 65 (3, 54-59).

Part IV: NAEP Assessment Toolbox Strategies/Math 54

Use Assessment Items to Build Math Vocabulary

“The meanings of words in math—for example, even, odd, product, and factor—often differ from their use in common language. Many students needing math intervention have weak mathematical vocabularies. It's key that students develop a firm understanding of mathematical concepts before learning new vocabulary, so that they can anchor terminology in their understanding. We should explicitly teach vocabulary in the context of a learning activity and then use it consistently. A math vocabulary chart can help keep both teacher and students focused on the importance of accurately using math terms.”

Burns, M. (2007). Nine Ways to Catch Kids Up [Electronic version]. Educational Leadership: Making Math Count 65 (3, 16-21).

Part IV: NAEP Assessment Toolbox Strategies/Math 55

Math GLEs Can Serve as Vocabulary Chart

Part IV: NAEP Assessment Toolbox Strategies/Math 56

Conclusion

NAEP Assessment Items Illustrate the GLEs for Students Turn abstract idea into learning

activity Clarify expectations One way of students taking

responsibility for own learning

Part I: Introduction to Using Assessment Items 57

NAEP Questions Available in Many Content Areas at http://nces.ed.gov/nationsreportcard/itmrls/

Part V: Alaska SBA Resources 58

Part V

Alaska Standards Based Assessment Resources

Part I: Introduction to Using Assessment Items 59

SBA Practice Tests Assessment Items

Part I: Introduction to Using Assessment Items 60

Scoring Guide is key. For example, look at Grade 6 Reading.

Part V: Alaska SBA Resources 61

Last page of the Scoring Guide has test map showing the GLE assessed by each reading question.

Part I: Introduction to Using Assessment Items 62

Send questions or comments to

Jeanne Foy, Alaska NAEP CoordinatorJeanne.Foy@alaska.gov