TRANSFORMATIVE TRANSFORMATIVE ASSESSMENT IN ACTIONASSESSMENT IN ACTION

W. James PophamUniversity of California, Los Angeles

2011 Teacher Leader InstituteMinnesota ASCD

Brainerd, Minnesota October 5-7, 2011

CUDDLING UP TO THE CURRENT CUDDLING UP TO THE CURRENT CONTEXT IN EDUCATIONCONTEXT IN EDUCATION

• A A Personal Perspective

• A National PerspectiveA National Perspective

THE GHOSTS OF EDUCATION PAST, THE GHOSTS OF EDUCATION PAST, EDUCATION PRESENT, AND EDUCATION PRESENT, AND

EDUCATION FUTURE EDUCATION FUTURE

(With Apologies to Charles Dickens)(With Apologies to Charles Dickens)

• THE PAST: A SLOW BUT RELENTLESS MOVE TOWARD FEDERAL INFLUENCE ON LOCAL CLASSROOM EVENTS

• THE THE PRESENTPRESENT EDUCATIONAL SCENE: EDUCATIONAL SCENE:

3 DELPHI-DEFYING DEVELOPMENTS!3 DELPHI-DEFYING DEVELOPMENTS!

• THE THE FUTURE FUTURE OF U.S. PUBLIC SCHOOLS: OF U.S. PUBLIC SCHOOLS: BOTH SHORT-TERM AND LONG-TERM PERILS BOTH SHORT-TERM AND LONG-TERM PERILS TO BE DODGEDTO BE DODGED

THE COMMON CORE STATE THE COMMON CORE STATE STANDARDS (CCSS)STANDARDS (CCSS)

• Developed by CCSSO and the NGA Developed by CCSSO and the NGA Center Center (Council of Chief State School Officers and (Council of Chief State School Officers and the National Governors Association Center for Best the National Governors Association Center for Best Practices)Practices)

• Fewer, Clearer, Higher?Fewer, Clearer, Higher?

• Although created rapidly and with Although created rapidly and with scant external scrutiny, getting the scant external scrutiny, getting the CCSS so widely adopted is an amazing CCSS so widely adopted is an amazing accomplishment by CCSSO and NGA. accomplishment by CCSSO and NGA.

THE TWO ASSESSMENT CONSORTIATHE TWO ASSESSMENT CONSORTIA

• The Partnership for Assessment of The Partnership for Assessment of Readiness for College and Careers Readiness for College and Careers (PARCC) and the SMARTER Balanced (PARCC) and the SMARTER Balanced Assessment Consortium (SBAC)Assessment Consortium (SBAC)

• Although many similarities exist Although many similarities exist between the two consortia, key between the two consortia, key differences are also present.differences are also present.

Workshop Foci: Four “What-It-Is” Workshop Foci: Four “What-It-Is” Understandings and a Perplexing PuzzleUnderstandings and a Perplexing Puzzle

• What It Is and What It Isn’t• What It Can Do and What It Can’t• Two Cuts of Formative Assessment’s Cake• Why Learning Progressions Must Lurk

*************

Why Aren’t More Minnesota Teachers

Using Formative Assessment?

Formative Assessment:Formative Assessment:What It IsWhat It Is

Formative assessment is a planned process in which assessment-elicited evidence of students’ status is used by teachers to adjust their ongoing instructional procedures or by students to adjust their current learning-tactics.

Formative Assessment:Formative Assessment: What It Isn’tWhat It Isn’t

• It is not a test.

• It is not an interim test (also referred to as a benchmark or periodic test) administered every few months by schools or districts.

• It is not the unplanned, serendipitous use of student cues to adjust teaching.

Formative Assessment:Formative Assessment:What It Can DoWhat It Can Do

In a research review based on 250 empirical studies of classroom assessment that had been drawn from more than 680 published investigations, Paul Black and Dylan Wiliam concluded:

“The research reported here shows conclusively that formative assessment does improve learning.” (Assessment in Education, 1998)

Formative Assessment:Formative Assessment:What It Can DoWhat It Can Do

Two Other Quotes from the Research Review:

• The student gains in learning triggered by formative assessment were “amongst the largest ever reported for educational interventions.”

• “Significant gains can be achieved by many different routes, and initiatives here are not likely to fail through neglect of delicate and subtle features.”

AND, MORE RECENTLY:AND, MORE RECENTLY:

“Five reviews of the research in this area synthesized a total of more than 4,000 research studies undertaken during the last 40 years. The conclusion was clear: When implemented well, formative assessment can effectively double the speed of student learning.” (Wiliam, Educational Leadership, 2007-2008)

AND, EVEN MORE RECENTLY:AND, EVEN MORE RECENTLY:

“There is now a strong body of There is now a strong body of theoretical and empirical work that theoretical and empirical work that suggests that integrating assessment suggests that integrating assessment with instruction may well have with instruction may well have unprecedented power to increase unprecedented power to increase student engagement and to improve student engagement and to improve learning outcomes.” (Wiliam, D., 2011,learning outcomes.” (Wiliam, D., 2011, Studies in Educational Evaluation)Studies in Educational Evaluation)

On October 13, 2009, I received an e-mail from a colleague in Ohio, Dr. Saundra Brennan, who relayed to me an invitation she had received from the Ohio Association of Elementary School Administrators. It was to take part in:

A FORMATIVE ASSESSMENT WORSHIP

BESIDES AN ANALYTICAL AND EMPIRICAL BESIDES AN ANALYTICAL AND EMPIRICAL RATIONALE, HERE’S A SPOT OF RATIONALE, HERE’S A SPOT OF

SPIRITUAL APPROBATION:SPIRITUAL APPROBATION:

Formative Assessment:Formative Assessment: What It Can’t DoWhat It Can’t Do

It cannot raise scores sufficiently on instructionally insensitive accountability tests such as those so widely used these days to satisfy the requirements of the No Child Left Behind Act.

A DEFINITION OF A DEFINITION OF INSTRUCTIONAL SENSITIVITY INSTRUCTIONAL SENSITIVITY

The degree to which students’ The degree to which students’ performances on a test accurately performances on a test accurately reflect the quality of instruction reflect the quality of instruction specifically provided to promote specifically provided to promote students’ mastery of what is being students’ mastery of what is being assessed.assessed.

WHY MIGHT A TEST ITEM BE WHY MIGHT A TEST ITEM BE INSTRUCTIONALLY INSENSITIVE?INSTRUCTIONALLY INSENSITIVE?

• Alignment LeniencyAlignment Leniency

• Excessive EasinessExcessive Easiness

• Excessive DifficultyExcessive Difficulty

• Confusion-Engendering Item FlawsConfusion-Engendering Item Flaws

• Socioeconomic Status (SES) LinksSocioeconomic Status (SES) Links

• Academic Aptitude LinksAcademic Aptitude Links

SOCIOECONOMIC STATUS (SES) LINKSSOCIOECONOMIC STATUS (SES) LINKS

If an item gives a meaningful advantage If an item gives a meaningful advantage to students from higher SES families, to students from higher SES families, then the item will tend to measure what then the item will tend to measure what students bring to school rather than students bring to school rather than how well they are taught once they get how well they are taught once they get there.there.

A 6th-Grade Science Item:

• A plant’s fruit always contains A plant’s fruit always contains seeds. Which of the items seeds. Which of the items below is not a fruit?below is not a fruit?

A.A. orangeorangeB.B. pumpkinpumpkinC.C. appleappleD.D. celerycelery

• In which of the sentences below does the word field mean the same thing as in the sentence above?A. The shortstop knew how to field his

position.B. We prepared the field by plowing it.C. What field do you plan to enter when you

graduate?D. The nurse examined my field of vision.

A 4th-Grade Reading Item:

My father’s field is computer graphics.

ACADEMIC APTITUDE LINKSACADEMIC APTITUDE LINKS

If an item gives a meaningful advantage If an item gives a meaningful advantage to students who possess greater to students who possess greater inheritedinherited quantitative, verbal, or spatial quantitative, verbal, or spatial aptitudes, then the item will tend to aptitudes, then the item will tend to measure what students bring to school measure what students bring to school rather than how well they are taught rather than how well they are taught once they get there.once they get there.

A 3rd-Grade Mathematics Item:A 3rd-Grade Mathematics Item:

• The secret number is inside the circle. It is The secret number is inside the circle. It is also inside the square. It is NOT inside the also inside the square. It is NOT inside the triangle. Which of these is the secret triangle. Which of these is the secret number?number?

A. 2A. 2 B. 3B. 3 C. 5 C. 5 D. 7 D. 7

5 24

76

1

3

A 4A 4thth-Grade Mathematics Item-Grade Mathematics Item:

• Which of the letters below, when Which of the letters below, when folded in half, will have two parts folded in half, will have two parts that match exactly?that match exactly?

FF(A)(A)

ZZ(B)(B)

SS(C)(C)

BB(D)(D)

TWO WAYS OF SLICING UP THETWO WAYS OF SLICING UP THEFORMATIVE-ASSESSMENT PIEFORMATIVE-ASSESSMENT PIE

1. 1. Levels of ImplementationLevels of Implementation, that is, Four , that is, Four Distinct Ways of Conceptualizing the Distinct Ways of Conceptualizing the Formative-Assessment ProcessFormative-Assessment Process

2. 2. Applications of the ProcessApplications of the Process, that is, , that is, Five Different Applications of the Five Different Applications of the Formative-Assessment ProcessFormative-Assessment Process

Formative Assessment:Formative Assessment:Why Why LevelsLevels Can Lead to Lucidity Can Lead to Lucidity

• Because there are two potential sets of players (teachers adjusting instruction and students adjusting learning tactics), it is often confusing to lump the two together.

• Because there are substantially different varieties and levels of implementation, clarity can be compromised by regarding one variant of formative assessment as coterminous with another variant.

Four Levels of Formative Assessment Level 4: Schoolwide Implementation

Level 3: Classroom-Climate Shift

Level 2: Students’ Learning-Tactic Adjustments

Level 1: Teachers’ Instructional Adjustments

Students’ Level 2 Steps

1. Consider adjustment occasions.

2. Consider assessments.

3. Consider adjustment triggers.

4. Adjust learning-tactics?

Teachers’ Level 1 Steps

1. Identify adjustment occasions.

2. Select assessments.

3. Establish adjustment triggers.

4. Make instructional adjustments?

Level 4 Strategies

1. Professional Development

2. Teacher Learning Communities

Level 3 Shifts

1. Learning Expectations

2. Responsibility for learning

3. Role of classroom assessment

Learning Progressions

TheFormative-Assessment

Process

Immediate Instructional Adjustments

Near-Future Instructional Adjustments

Last-Chance Instructional Adjustments

Students’ Learning Tactic

Adjustments

Classroom-ClimateShifts

FIVE APPLICATIONS OF THE PROCESSFIVE APPLICATIONS OF THE PROCESS

• Immediate Instructional Adjustments Immediate Instructional Adjustments (based on assessments (based on assessments oror self-reports) self-reports)

• Near-Future Instructional AdjustmentsNear-Future Instructional Adjustments

• Last-Chance Instructional AdjustmentsLast-Chance Instructional Adjustments

• Students’ Learning-Tactic AdjustmentsStudents’ Learning-Tactic Adjustments

• Classroom Climate ShiftsClassroom Climate Shifts

Formative Assessment:Formative Assessment:Why Learning Progressions Why Learning Progressions MustMust Lurk Lurk

(What a learning progression is:)

A learning progression is a sequenced set of building blocks (that is, subskills and/or bodies of enabling knowledge) it is thought students must master en route to mastering a more remote, target curricular aim.

TWO LEARNING PROGRESSIONS:Both Useful to Educators

Upper-case LEARNING PROGRESSIONS

Lower-case Learning Progressions

Target Curriculum

Aim X

Enabling Knowledge

B

Enabling Knowledge

A

SubskillA

SubskillB

An Illustrative Learning

Progression

A Horizontally Represented Learning Progression

Enabling Knowledge

X

Enabling Knowledge

Y

SubskillZ

TargetCurriculum

Aim Q

Formative Assessment:Formative Assessment:What a Learning Progression is What a Learning Progression is notnot

• A learning progression isn’t unerringly accurate.

• A particular learning progression isn’t going to work for all students.

• A learning progression isn’t necessarily better because it’s more complicated.

THE AFFECTIVE CONSEQUENCESTHE AFFECTIVE CONSEQUENCESOF FORMATIVE ASSESSMENTOF FORMATIVE ASSESSMENT

• The significance of affect in general, The significance of affect in general, and students’ affect specifically related and students’ affect specifically related to formative assessmentto formative assessment

• How the assessment of students’ affect How the assessment of students’ affect will incline teachers to attend to will incline teachers to attend to students’ affective dispositionsstudents’ affective dispositions

• How students’ affect related to How students’ affect related to formative assessment can be measuredformative assessment can be measured

Formative Assessment:Formative Assessment:And Now the PuzzleAnd Now the Puzzle

Given what we know about the success of formative assessment in boosting students’ achievement, why aren’t more Minnesota teachers using the formative-assessment process?

For a no-cost copy of a study guide for using Transformative Assessment with an extended-duration professional learning community, see the following: www.ascd.org/studyguides. Scroll down to the book’s title and you’ll also find info about how to acquire this enchanting, yet insightful volume. The sequel, equally enchanting, contains Reflection Questions for each chapter rather than a study guide.

My e-mail address: wpopham@ucla.edu