VT-Developmental Reading Assessment: An example of large-

scale assessment during the primary grades

• Brief summary of the assessment itself: context, development, nature of the assessment, results obtained, uses of results

• Strategies used to increase the validity and consistency of scoring of the assessment

• Continuing Challenges and Hindsight

• Susan Carey Biggam, VT Department of Education, 1999

Context and Background

• Needs for information about early reading success

• Early reading initiative

• Title 1: need for accountability in the lower grades

• Recent adoption of Framework/standards

• Educational context

• Task Force : 1996• Criteria used, added• Explore DRA through

Feasibility Study• Advisory Board• Development of

Comprehension Scoring Guides: UVM team/NE Regional Lab

• Significant changes

Unique Aspects of the VT DRA

• Individually administered, standards based assessment of early reading

• Designed to provide accountability information and to influence instruction

• Continually shaped by research

• Includes processes for quality control

What has been learned so far from the VT DRA?

• % of students at five levels of proficiency -- two years (but not comparable at highest two levels because of changes in scoring criteria)

• beginning disaggregation

• increased focus by teachers on comprehension and fluency

• enhanced communication across grades, common expectations

• decreasing concerns about time, subjectivity

Strategies Used to Increase the Validity and Consistency of Scoring of the

Assessment

Involvement of educators: task force, advisory council, questionnaires, meetings…

Learning from results: refinement in scoring guides-- from audiotapes/transcripts, results from scoring, feedback from scorers; teacher judgement study in process

Rescoring Institute:consistency of scoring and administration, confidential feedback to teachers

Professional Development (led by pairs of trainers)-- for teams of primary-level educators: Modeling, guided practice, calibration. “New Folks” training plus annual update sessions

Communications with schl. administrators, public

Lessons Learned / Hindsight

• Feasibility studies help. Buy time if possible.• For program-level assessment, encourage

participation/ownership by as many program participants as possible (not just grade X tchrs.)

• Standards help clarify the focus of the assessment : what is assessed VS what is not.

• Teachers welcome efforts to strengthen consistency… fairness is a shared goal.

• It’s hard work, and it’s important to build in sources of relief: food, humor, etc.

Continuing Challenges…. Issues

• Balancing the need for stability of the data with the need for continued improvements to strengthen the assessment’s validity

• Walking the fine line between “state control” and “technical assistance”

• Helping schools/districts use other data sources for early reading as well, and keep in mind which standards are tapped by the VT DRA

• Perseverance -- both on the part of those managing the assessment and those administering it

Resources Used

• Developmental Reading Assessment. by Joetta Beaver, Upper Arlington Elem. Schools. Published by Celebration Press (imprint of Pearson Learning)

• Technical assistance from Lipson, Mekkelsen & Mosenthal, research team from University of VT. Supported by funds from the Northeast and Islands Regional Lab at Brown University

• Advice and technical assistance from a variety of resource people-- within VT and from other states

Vermont-Developmental Reading Assessment

An example of large-scale assessment during the primary grades

Susan Carey Biggam

VT Department of Education

September 16, 1999

CRESST conference, UCLA