caveon webinar series: weathering the perfect test security storm - february 2015

Jennifer Miller, Caveon Test Security

Dennis Maynes, Caveon Test Security

Michelle Croft, ACT

Weathering the Perfect Test Security Storm in Educational Assessments

February 25, 2015

Caveon Webinar Series presents:

Agenda for Today

• Elements of the Perfect Storm

(Jennifer Miller)

• Fallout from the storm

(Michelle Croft)

• Weathering the storm

(Dennis Maynes)

Introduction

The Perfect Storm

• Test scores are tied to school funding.• Performance evaluations are tied to test

scores.• The responsibility of test security is placed in

the hands of those who are motivated to cheat.

• Security threats and vulnerabilities are increasing.

NCLB (2001)

“Adequate yearly progress”

Federal funding goes to struggling

schools, but…..

…with funding come sanctions

NCLB & Funding

• Each state sets timeline and definition for AYP.

• Schools held accountable for performance of all students, not just the average performance.

• Schools that do not meet AYP for two years in a row are subject to immediate interventions.

– Technical assistance

– Severe corrective actions

Adequate Yearly Progress

ESEA Flexibility Waiver

• Announced in 2011 to allow flexibility from some specific NCLB requirements.

• State-developed plans where turnaround interventions are implemented at “priority” schools.

• Requires student growth data as part of educator evaluations.

Parks Middle School in Atlanta

“school in need of improvement” 2001-2005

In 2006, “Unless fifty-eight per cent of students passed the math portion of the test and sixty-seven per cent passed in language arts, the state could shut down the school.”

State Assessments and School Funding

The New Yorker, 21 July 2014

Governor’s investigation in Georgia found cheating in 44 APS schools, with almost 200 educators potentially involved.

“Investigators found ‘organized and systemicmisconduct’ …and said administrators created a‘culture of fear, intimidation and retaliation.’The administrators used data as an ‘abusive andcruel weapon’ to coax employees into crossingethical lines”

Los Angeles Times, 6 September 2014

State Assessments and School Funding

Performance Evaluations and Test Scores

States are moving to performance-based compensation and using test scores to evaluate teachers

Administrators(Superintendents, etc.)

Teachers

• 2012 El Paso Independent School District.

• Lorenzo Garcia, Superintendent.

• Discouraged and/or prevented low-performing students from taking the TAKS.

• 381 students 170 students

• Garcia received almost $60K in bonuses.

New York Times, 13 October 2012

Use of Test Scores for Personal Gain

Test Security in the Hands of Teachers

"If the IRS notified you that your tax return had irregularities,and they wanted you to recheck it yourself, everybody willwrite back and say, 'I checked my taxes, and they're fine.‘”

– Greg Cizek

Test Security in the Hands of Teachers

Teachers feeling the pressure:

Survey of Educators in Michigan

• 29% felt pressure to cheat on standardized tests.

• 34% felt pressure to help students.• 21% knew other educators who helped

students.• 8% admitted to changing answer sheets.

“Survey: Nearly 30% of Michigan Teachers Report Pressure to Cheat,” Detroit Free Press, July 27, 2011

Threats are Increasing

• Cheating technology is becoming more sophisticated.

• Threats and CBT.• Threats associated with

consortia membership (PARCC and SBAC).

• Threats with Social Media.

thegreenhead.com

• Harm to Students

• Tarnished Reputations

• Lost Credibility

• Negative Media Attention

•$$$$$

Fallout from the Storm

Harm to Students“As a graduating senior from a tarnished schoolsystem, I have to worry every time I apply to acollege…good grades isn’t considered a reward initself anymore because those grades come from aschool system of perceived cheaters…it’s no joke forstudents who have to overcome the perception of amorally bankrupt school system.”


Jenn Steckl APS School Districtyouthradio.org27 October 2014

Negative Media Attention

• APS Scandal, Atlanta Journal Constitution, 2008.

• Cheating in 44 APS schools.• Prosecution of 12 educators under RICO• Trial ongoing…


Pa Schools, The Notebook, 2009• Anomalies in 89 Schools - Cheating?

• As school starts, no action against Philly principals where cheating is suspected, suspected, 8/24/2012.

• Two District administrators surrender licenses in cheating scandal, 4/3/2013.• District to release report on cheating investigation at 19 schools, 7/22/2013.• 138 Philadelphia educators implicated in cheating probes, 2/5/2014.• SRC to vote on terminating three principals related to cheating investigation,

1/16/2014.• Two former principals charged in state cheating investigation, 9/25/2014.• Former principal now eighth educator charged in cheating probe, 1/7/2015.


thenotebook.org


District of Columbia

• USA Today article (2011).• Entire year of items released.• Solicited bids for external

investigators to increase transparency.

• Budget

• Strategy

• Implement a Comprehensive Program

Weathering the Storm

• Strategy

• Create Mission and Vision Statements

• Assess Threats and Vulnerabilities

• Review and Allocate Resources

• Establish a Communications Plan


• Implement a Comprehensive Program

• Protection

• Detection

• Response

• Improvement



Comprehensive Program

Protect

Detect

Respond

Improve

“Never before have so many had so muchreason to cheat. Students’ scores are now usedto determine whether teachers and principalsare good or bad, whether teachers should get abonus or be fired, whether a school is a successor failure…”

New York Times, 31 July 2011

Audience Questions

25

Thank you!

- Follow Caveon on twitter @caveon

- Check out our blog…www.caveon.com/blog

- LinkedIn Group – “Caveon Test Security”

Jennifer Miller

Data Forensics Coordinator,

Caveon [email protected]

Michelle Croft

Principal Research Associate,

[email protected]

Dennis Maynes

Chief Scientist, Caveon [email protected]