early childhood assessment: what early educators need to ... · what early educators need to know...

Buffett Early Childhood Institute

Early Childhood Assessment:What Early EducatorsNeed to Know About Testingand AccountabilitySamuel J. MeiselsBuffett Early Childhood InstituteUniversity of Nebraska

Buffett Early Childhood Institute

Buffett Early Childhood Institute

Buffett Early Childhood Institute

Buffett Early Childhood Institute

Buffett Early Childhood Institute

• Are children learning?• Are public funds being used wisely?

What PolicymakersWant To Know

Buffett Early Childhood Institute

1. Early development isepisodic and uneven

2. Concept of “readiness”is relative

3. Test items often imply teaching

4. Validity of the tests is poor

The Problem

Buffett Early Childhood Institute

Predicting children’s competence from readiness assessmentsOn average, only 25% of variance in early academic/cognitive performance is predicted from preschool or kindergarten cognitive status.

Only 10% or less of the variance in K–2 social/ behavioral measures is predicted by social/behavioral assessments at preschool or kindergarten.

LoParo & Pianta, 2000

Buffett Early Childhood Institute

Predicting children’s competence from readiness assessments

“Instability or change may be the rule rather than the exception during this period.”

LoParo & Pianta, 2000

Buffett Early Childhood Institute

Buffett Early Childhood Institute

Testing andAccountabilityIn Early Childhood

Buffett Early Childhood Institute

Buffett Early Childhood Institute

Buffett Early Childhood Institute

Construct Underrepresentation

The extent to which a test fails to capture important aspects of the construct it is intended to measure.

AERA/APA/NCME, 1999

Buffett Early Childhood Institute

Point to “vase.”

Dunn et al., 1997

Buffett Early Childhood Institute

Point to “horrified.”

Dunn et al., 1997

Buffett Early Childhood Institute

Compared to the Head StartChild Outcomes Framework:

— 33% of the total —and none of the 54 indicators in the other five domains.

The NRS covers only 15 of 46 indicators in language, literacy, and mathematics

Buffett Early Childhood Institute

Construct-Irrelevant Variance

The extent to which test scores are influenced by factors that are irrelevant to the constructs the test is intended to measure.

AERA/APA/NCME, 1999

Buffett Early Childhood Institute

“Point to all the letters that you know and tell me the name of each one. Go slowly and show me which letter you’re naming.”

National Reporting System, 2004

Buffett Early Childhood Institute

“This graph shows how many children on Clark Street have each kind of pet: dogs, cats, or rabbits. Which kind do most of the children have? Which kind of pet is most popular?”

National Reporting System, 2004

Buffett Early Childhood Institute

The NRS Experience• Only 40% of the children wanted to go

with the tester in the spring

• Half the testers reported difficult behaviors persisting from the fall

• One-quarter of the testers said that test procedures were unnatural

Buffett Early Childhood Institute

General Accountability Office

Buffett Early Childhood Institute

Buffett Early Childhood Institute

High stakes testing for accountability doesn’t promote learning in early childhood.

Buffett Early Childhood Institute

ObservationalAssessment

Buffett Early Childhood Institute

Buffett Early Childhood Institute

Standards-based assessments that rely on teachers making systematic observations of children during the course of daily classroom experience.

Observational Assessment

Buffett Early Childhood Institute

The best way to evaluate a child’s performance is to study performance, not something else.

Buffett Early Childhood Institute

in a natural environment and in relationship to standards

Observational assessment consists of:

OBSERVING

RECORDING

EVALUATING

Buffett Early Childhood Institute

Pearson Assessments

Buffett Early Childhood Institute

Developmental Checklist (Kdg.)

Work Sampling System

Buffett Early Childhood Institute

Developmental Guidelines (Kdg.)

Work Sampling System

Buffett Early Childhood Institute

Buffett Early Childhood Institute

Can We TrustTeachers’ Judgments?

Buffett Early Childhood Institute

Principal Study Questions

• Is WSS a valid means of evaluating student achievement and progress?

• What is the impact of WSS on student learning?

Buffett Early Childhood Institute

Sample for the Validity Studies

• K – 3, cross-sectional• N = 345• 17 classrooms, 5 schools• 70% African-American, 80% low

income (free or reduced lunch), 49% male, 8% special needs

Meisels et al., 2001

Buffett Early Childhood Institute

Results of the WSS Validity Study (K - 3)

• WSS correlates very well with a standardized, individually administered psychoeducational battery (the Woodcock Johnson-Revised);

• WSS is a reliable predictor of achievement ratings in kindergarten – Grade 3;

• Data obtained from WSS accurately discriminate between children who are and are not at risk. Meisels et al., 2001; 2008

Buffett Early Childhood Institute

Mean change scores on ITBS for third - fourth graders

Meisels et al., 2003

Buffett Early Childhood Institute

Conclusions

• WSS is a valid and effective assessment of children’s learning.

• Students in WSS classrooms make significant achievement gains.

• Teachers and families are satisfied with the system and become more so as they have more experience with it.

Buffett Early Childhood Institute

The best way to improve a child’s performance is to teach the child, not test the child.

Four Conclusions About Testing and Early LearningBuffett Early Childhood Institute

Buffett Early Childhood Institute

Tests used for high-stakes change what is taught.

1.

Buffett Early Childhood Institute

Accountability should be concerned with what is taught and what is learned, not with rewards and punishments.

2.

Buffett Early Childhood Institute

Standards are best thought of as benchmarks, not thresholds.

3.

Buffett Early Childhood Institute

Understanding the child within context is key to understanding the child.

4.

Buffett Early Childhood Institute

Measure whatwe treasureBuffett Early Childhood Institute

Buffett Early Childhood Institute

Start early. Start well.

Samuel J. Meisels, Ed.D.Buffett Early Childhood InstituteUniversity of Nebraskasmeisels@nebraska.edu www.buffettinstitute.nebraska.edu

Buffett Early Childhood Institute