Assessing Student Achievement with PARCC The New Jersey Leadership Summit

Jeffrey Nellhaus

Director of Policy, Research and Design

PARCC, Inc.

April 3, 2014

• The Consortium

• Goals and Priorities

• Assessment System Design

• Reporting Schedule and Reporting Metrics

• Technology

• Field Test

• Practice Tests

• Rollout

Presentation Topics

2

The PARCC Consortium

• Led by member state Chief State School Officers with significant support from SEA and LEA staff

• PARCC, Inc., Project

Management Partner • Consultants • Contractors

3

Develop tests

worth taking

Texts worth reading

Essays worth writing

Problems worth solving

Accessible to all students

Goals …

4

Report results worth using

Valid Reliable Timely

Actionable

Determine whether students are college- and career- ready or “on track”

Provide tools to assess student learning and support instruction during the

school year

Generate data, including growth metrics, for accountability uses

Report comparable data across schools,

districts and member states

Priorities

Use technology to create efficiencies

and engage students in the testing process

5

Design of the Assessment System

6

Formative Tools For use during the school year

• Grades 2-8

• Reading, Writing, Math

• Computer adaptive

• Designed to pinpoint students’ learning needs

• Links to interventions/enrichments

Diagnostic Assessments Mid-Year/Interim Assessments

• Grades 3-11

• ELA/Literacy and Math

• Computer- and paper-based

• Built from released PBA tasks

• Can be used for assessment at individual, classroom, school levels

K-1 Tools Speaking & Listening Tools • Grades K-1 • Reading and math • Checklists, running records,

performance tasks • Links to interventions/enrichments

• Grades 3-12 • Performance-based activities • Spontaneous oral response to oral

prompt; share findings of research in an oral presentation

7

Summative Assessments Measure and report achievement and growth

ELA/Literacy

Writing essays drawing evidence from sources, including multi-media, some comprehension

Math

Solving multi-step problems that require reasoning and address real world situations

ELA/Literacy

Demonstrating comprehension of literary and informational texts

Math

Demonstrating understanding of concepts and procedures and carrying out short applications

Performance-Based Component (PBA)

End-of-Year Component (EOY)

Overall Score = Combination of PBA + EOY 8

• Other Resources to Support Instruction

– Annual release of test items and benchmarked student work

– Guidance for using formative tools and interpreting results of summative assessments

• Test Design

– End-of-course math tests: Alg. I, Geo., and Alg. II

– ELA/L tests in each of the grades 9, 10, and 11

– Tests for high schools with block schedules

• Accessibility

– Computer-delivered accommodations for SWDs & ELs

– Translations of math assessments for ELs

Additional Features

9

PARCC’s Accessibility Features

10

Accessibility Features for All

Students

Accessibility Features

Identified in advance

Accommodations

Selected features

highlighter answer masking

magnification line reader …

color contrast

answer masking, math text-to-speech …

ELA text-to-speech

scribe extended time …

• The goal is to return results of the summative assessments prior to the end of the school year

• Standard setting will occur in

Summer 2015, with year 1 results

reported in September

• Reporting schedule for year 2 is

under discussion

Reporting Schedule

Claim Sub-Claim Performance Level

Scale Score Sub-Score

ELA/L X X

Reading X

Reading Literary Text

X

Reading Informational Text

X

Vocabulary X

Writing X

Written Expression X

Knowledge of Conventions

X

English Language Arts & Literacy Claims and Reporting Metrics

12

Claim Sub-Claim Performance Level

Scale Score Sub-Score

Math X X

Major Content X

Additional & Supporting Content

X

Expressing Mathematical Reasoning

X

Modeling and Applications

X

Mathematics Claims and Reporting Metrics

13

• PARCC will report results of summative assessments using 5 performance levels

• Allows for finer classifications of student performance and supports reporting of improvement and growth

• Performance level descriptors at www.parcconline.org/plds

• Standard setting event will occur in summer 2015

• K-12 and HE educators will serve on standard-setting panels

Performance Levels

14

Technology Number and Types of Devices

• Rule of thumb: In a school with 3 tested grades, at least one computer for every 2 students in the largest tested grade

• Desktop, laptop, netbook, and thin client/VDI computers (Windows, Mac, Chromebook), and tablets

• All devices must meet the established hardware, operating system, and networking specifications http://www.parcconline.org/technology

15

• Technology purchases should consider instructional needs, not only assessment

• Proctor caching allows schools with limited bandwidth to administer PARCC’s computer-based assessments by reducing the amount of data being transmitted over the internet during testing

• Caching involves downloading the test content to a local server prior to testing, then distributing the test to students through the school’s internal network.

• Caching will be available to all PARCC schools as part of the Technology Platform (TestNav 8) and only requires 5 kbps per simultaneous test-taker.

Proctor Caching: A Solution for Schools with Limited Bandwidth

16

• Not designed to yield individual student, school, or district results

• Also an opportunity for a number of important research studies

Field Tests – Testing the Test

The field tests are designed to evaluate the quality of the test questions, delivery platform and administration protocols.

17

Study Brief Description Participating States

1. Mode Comparability Can paper- and computer-based assessments be reported on the same scale?

ALL

2. Device Comparability Are assessment results of tablet and desktop/laptop administrations comparable?

MA, AR, CO

3. Quality of Items and Tasks Do the items measure what was intended to be measured? Do any items show bias? Was human scoring reliable?

ALL

4. Text-to-Speech Validity Does the text-to-speech accommodation provide desired differential boost to those who need it?

MD

5. High School Math Comparability Can traditional and integrated EOC assessments be reported on the same scale?

ALL

6. Quality of Test Administration

Do test administrators understand administration protocols? Do students understand test directions?

Surveys in ALL Site observations in MD, TN, OH, NM, IL

7. Feasibility of International Benchmarking

What points on PARCC scale correspond to benchmarks from PISA, PIRLS and TIMSS?

ALL

8. Psychometric Studies Can assessment results be put on a vertical scale? What is the best way to combine results from the PBA and EOY?

ALL

Field Test Research Studies

18

Schools in 14 States & DC are Participating in the Field Tests

19

• Over 1 million students are participating

• Nearly 16,000 schools

• In most cases, 1-3 grades per school, 2 classrooms per grade

• Most student take one component in one content area

• https://pearsonstsprod.service-now.com/parcc/

• With the exception of a few glitches, the delivery platform is functioning well

• Most students are able to use the platform, especially those who practiced first with sample questions and tutorials

• Schools that conducted a “dress rehearsal” have had fewer issues

• Social media is a factor: on the one hand, immediate feedback; on the other, threat to security

• Some directions need to be clarified: responding to essay questions, closing out of test sessions

20

Field Tests Already Providing Great Feedback

PARCC will get additional feedback from test

administrator and student questionnaires

– Practice tests, sample items and navigation tutorial now available

http://practice.parcc.testnav.com/#

Practice Tests and Tutorial Helping Students Get Ready

21

Next Steps For PARCC

22

SEPTEMBER

States launch PARCC

SUMMER

Model Content Frameworks

Released

AUGUST Item

Prototypes Released

APRIL Test

Blueprints released

We are here!

WINTER/SPRING

Field Test/Practice Test Online

SPRING

First Administration

of New Tests

SUMMER

Establishment of Cut Scores

FALL

Release of Diagnostic and

Formative Assessments

2010 2011

2012 2013

2014 2015

FALL

Use of Cut Scores for IHE

Placement

2016

Next year

APRIL PARCC

Becomes Independent Non-Profit

SUMMER

PARCC Reports Research From

Field Test Results

• NJ has made great strides transitioning to the CCSS over past several years

• Stay the course. Getting kids ready for college and careers is the right thing.

• PARCC states are committed to developing a high quality assessment system

• We invite your feedback and will strive to improve the system each year

• It’s about tests worth taking, reports results worth using!

Final Comments

23

Thank You!

jnellhaus@parcconline.org

Please visit PARCC online at: www.parcconline.org

Or follow us on Twitter at:

www.twitter.com/PARCCPlace

24