caveon webinar - weathering the perfect test security storm may 2015 edited
TRANSCRIPT
www.caveon.com 1
Caveon Webinar Series
Weathering the Perfect Test Security Storm in Educational Assessments
Jennifer Miller, Caveon Test Security
Dennis Maynes, Caveon Test Security
Steve Addicott, Caveon Test Security
May 27, 2015
www.caveon.com 2
Agenda for Today
• Elements of the Perfect StormJennifer Miller
• Fallout from the Storm Steve Addicott
• Weathering the StormDennis Maynes
www.caveon.com 3
Introduction
www.caveon.com 4
The Perfect Storm
• Test scores are tied to school funding.• Performance evaluations are tied to
test scores.• The responsibility of test security is
placed in the hands of those who are motivated to cheat.
• Security threats and vulnerabilities are increasing.
www.caveon.com 5
NCLB & Funding
NCLB (2001) “Adequate yearly progress”
Federal funding goes to struggling schools, but…..
…with funding come sanctions
www.caveon.com 6
Adequate Yearly Progress
• Each state sets timeline and definition for AYP.
• Schools held accountable for performance of all students, not just the average performance.
• Schools that do not meet AYP for two years in a row are subject to immediate interventions.– Technical assistance– Severe corrective actions
www.caveon.com 7
ESEA Flexibility Waiver
• Announced in 2011 to allow flexibility from some specific NCLB requirements.
• State-developed plans where turnaround interventions are implemented at “priority” schools.
• Requires student growth data as part of educator evaluations.
www.caveon.com 8
State Assessment and School FundingParks Middle School in Atlanta “school in need of improvement” 2001-2005
In 2006, “Unless fifty-eight per cent of students passed the math portion of the test and sixty-seven per cent passed in language arts, the state could shut down the school.”
The New Yorker, 21 July 2014
Governor’s investigation in Georgia found cheating in 44 APS schools, with almost 200 educators potentially involved.
www.caveon.com 9
State Assessment and School Funding“Investigators found ‘organized and systemic misconduct’ …and said administrators created a ‘culture of fear, intimidation and retaliation.’
The administrators used data as an ‘abusive and cruel weapon’ to coax employees into crossing ethical lines.”Los Angeles Times, 6 September 2014
www.caveon.com 10
States are moving to performance-based compensation and using test scores to evaluate teachers
Performance Evaluations and Test Scores
Administrators(Superintendents,
etc.)
Teachers
www.caveon.com 11
Use of Test Scores for Personal Gain• 2012 El Paso Independent School
District.
• Lorenzo Garcia, Superintendent.
• Discouraged and/or prevented low-performing students from taking the TAKS.
• 381 students 170 students.
• Garcia received almost $60K in bonuses.
New York Times, 13 October 2012
www.caveon.com 12
Test Security in the Hands of Teachers"If the IRS notified you that your tax return had irregularities, and they wanted you to recheck it yourself, everybody will write back and say, 'I checked my taxes, and they're fine.‘”
– Greg Cizek
www.caveon.com 13
Teachers feeling the pressure:
Survey of Educators in Michigan• 29% felt pressure to cheat on
standardized tests.• 34% felt pressure to help students.• 21% knew other educators who
helped students.• 8% admitted to changing answer
sheets.
Test Security in the Hands of Teachers
“Survey: Nearly 30% of Michigan Teachers Report Pressure to Cheat,” Detroit Free Press, July 27, 2011
www.caveon.com 14
Threats are Increasing• Cheating technology is becoming
more sophisticated.• Threats and CBT.• Threats associated with
consortia membership (PARCC and SBAC).
• Threats with Social Media.
thegreenhead.com
www.caveon.com 15
Fallout from the Storm
• Harm to Students
• Tarnished Reputations– Lost Credibility– Negative Media
Attention
•$$$$$
www.caveon.com 16
Fallout from the Storm
Harm to Students“As a graduating senior from a tarnished school system, I have to worry every time I apply to a college…good grades isn’t considered a reward in itself anymore because those grades come from a school system of perceived cheaters…it’s no joke for students who have to overcome the perception of a morally bankrupt school system.”
Jenn Steckl APS School Districtyouthradio.org27 October 2014
www.caveon.com 17
Negative Media Attention
• APS Scandal, Atlanta Journal Constitution, 2008.
• Cheating in 44 APS schools.
• Prosecution of 12 educators under RICO.
• Trial ongoing…
Fallout from the Storm
www.caveon.com 18
PA Schools, The Notebook, 2009• Anomalies in 89 Schools - Cheating?
– As school starts, no action against Philly principals where cheating is suspected, 8/24/2012.
– Two District administrators surrender licenses in cheating scandal, 4/3/2013.
– District to release report on cheating investigation at 19 schools, 7/22/2013.
– 138 Philadelphia educators implicated in cheating probes, 2/5/2014.
– SRC to vote on terminating three principals related to cheating investigation, 1/16/2014.
– Two former principals charged in state cheating investigation, 9/25/2014.
– Former principal now eighth educator charged in cheating probe, 1/7/2015.
Fallout from the Storm
thenotebook.org
www.caveon.com 19
District of Columbia
• USA Today article (2011).
• Entire year of items released.
• Solicited bids for external investigators to increase transparency.
Fallout from the Storm
www.caveon.com 20
Weathering the Storm
• Budget/Resource Allocation
• Strategy
• Implement a Comprehensive Program
www.caveon.com 21
Strategy
• Create Mission and Vision Statements
• Assess Threats and Vulnerabilities
• Review and Allocate Resources
• Establish a Communications Plan
Weathering the Storm
www.caveon.com 22
Implement a Comprehensive Program
• Protection
• Detection
• Response
• Improvement
Weathering the Storm
www.caveon.com 23
Weathering the Storm
Comprehensive Program
Protect
Detect
Respond
Improve
www.caveon.com 24
“Never before have so many had so much reason to cheat. Students’ scores are now used to determine whether teachers and principals are good or bad, whether teachers should get a bonus or be fired, whether a school is a success or failure…”
New York Times, 31 July 2011
www.caveon.com 25
Weathering the Perfect Storm
www.caveon.com 26
Thank You!
Jennifer Miller Data Forensics Coordinator, Caveon [email protected]
Dennis MaynesChief Scientist, Caveon [email protected]
Steve AddicottVice President, Caveon [email protected]
Follow Caveon on twitter @caveonCheck out our blog www.caveon.com/blogLinkedIn Group “Caveon Test Security”