response to the blue ribbon commission’s recommendations of the 2009 criterion referenced...

Response to the Blue Ribbon Commission’s Recommendations of the 2009 Criterion Referenced

Competency Test (CRCT) Erasure Investigation

Dr. Beverly L. HallSuperintendent

Atlanta Public Schools

August 16, 2010

Blue Ribbon Commission ResponseAgenda

▪Purpose and Overview▪Recommendations and APS Approach▪Next Steps

2

Purpose

Atlanta Public Schools has begun the work to implement each of the recommendations in the report presented by

the Blue Ribbon Commission. We are committed to ensuring that 100 percent of the recommendations are

addressed and fully implemented.

3

Overview

▪Work that has already started–Caveon security measures–Accelerated Academic Recovery Program–Investigation of employees referred by the BRC–Reassigned 12 principals

▪Twenty-nine recommendations outlined in the BRC report–Plan to address each recommendation–Plan to track and ensure implementation

4

Background

▪Blue Ribbon Commission’s report included 29 recommendations:– Security (in addition to Caveon recommendations provided in April)– Process– Leadership– Comparison

▪Recommendation Owners– Each recommendation has been assigned to a division and individual

owner– Each owner is a member of my leadership team

▪Action Plans– Each recommendation owner has submitted initial action plans

outlining how each recommendation will be achieved

5

Security Recommendations

▪Security recommendations encompass:–Developing a comprehensive test security plan that

includes best-in-class test administration procedures–Continuing to research the unusually high number of

erasures–Reinforcing and communicating existing policies and any

new policies or changes

6


7

▪BRC Security Recommendation 1–Implement a well-defined program for follow-up and

continued monitoring of CRCT test security

▪Administration’s Response–A comprehensive APS security manual is being developed–Manual will include annual security goals and input from

the Georgia Department of Education and/or the Governor’s Office of Student Achievement–APS Internal Auditor will develop a plan to conduct

regular internal audits for each CRCT


8

▪BRC Security Recommendation 2–Research and identify causes of unusually high number of

erasures with the district and implement best practices

▪Administration’s Response–Requested erasure data from GOSA–Conduct statistical analysis of erasure data received–Identify and implement best practices in test preparation–Analyze test results extensively


9

▪BRC Security Recommendation 3–Reinforce APS' testing policies and procedures, and

ensure that all appropriate training takes place for employees involved in the administration of the CRCT

▪Administration’s Response–All security measures and annual enhancements will be

posted to the district website–School-based and central office staff will be trained

annually–All APS staff involved with testing will have clearly

defined security duties and responsibilities


10

▪BRC Security Recommendation 4–Communicate a policy that emphasizes ethics and

accountability for adherence to the policies and training requirements -- as well as the ramifications for violating such policies and requirements

▪Administration’s Response–Ethics training will be provided to employees district-

wide–Testing workshops will reinforce and review ethics

policies–Employees will be required to annually acknowledge

receipt and understanding of district ethics policies


11

▪BRC Security Recommendation 5–Formally document any policy or schedule changes

related to the CRCT test materials

▪Administration’s Response–Formal documentation was implemented for schedule

changes during the Spring 2010 CRCT–This process will be formalized as part of the Security

plan


12

▪BRC Security Recommendation 6–Establish a test security plan and ongoing budget for test

security

▪Administration’s Response–A comprehensive test security plan in development–Cost of implementation part of annual budget


13

▪BRC Security Recommendation 7–Implement annual compliance and ethics certifications

and approvals

▪Administration’s Response–Ethics training/certification program will be provided

annually to employees district-wide


14

▪BRC Security Recommendation 8–Establish ongoing external monitoring and reporting for

standardized testing at all APS schools

▪Administration’s Response–The district will continue to utilize a three-tiered

monitoring process–State-level monitoring will be requested

Process Recommendations

▪The process recommendations involve:–Follow up to ensure immediate assistance is given to

students impacted by testing irregularities–Developing a plan to track implementation of all of the

recommendations–Following up on the results of the investigation–Enhancing existing protocols for future CRCT

administrations and new audit routines

15


16

▪BRC Process Recommendation 1–Commit publicly to aggressively follow-up on the results

of the investigation by establishing a well-defined plan to consider the findings and recommendations in the BRC report

▪Administration’s Response–Public acceptance of the BRC report–Provide a plan of action to address the findings of the

BRC report


17

▪BRC Process Recommendation 2–Develop a plan that includes a project leader, clear

objectives, work plan, schedule and deliverables

▪Administration’s Response–A project leader has been appointed–An APS owner has been assigned to each

recommendation–Initial action plans have been created and appropriate

monitoring and status reporting will be scheduled


18

▪BRC Process Recommendation 3–Conduct further investigation of the referred individuals

within the Identified Schools and take whatever administrative, legal or professional action APS considers appropriate

▪Administration’s Response–The BRC report and all relevant documents are being

reviewed–Each matter is being referred to appropriate counsel or

another government agency–Investigations will be conducted and disciplinary actions

will be processed as appropriate


19

▪BRC Process Recommendation 4–Conduct an analysis of individual student performance of

those students APS determines may have been impacted by testing irregularities to determine if additional academic assistance and support are needed. If support is warranted, immediate action should be taken to mobilize appropriate assistance for the students for as long as necessary

▪Administration’s Response–Analysis of individual student performance has been

completed–12 week accelerated academic recovery plans are being

implemented–Regular communications with parents


20

▪BRC Process Recommendation 5–Develop protocols to pro-actively conduct erasure

analysis after future CRCT administrations

▪Administration’s Response–Erasure data has been requested from GOSA and will be

requested each year–Conduct statistical analysis of erasure data received


21

▪BRC Process Recommendation 6–Develop regular internal audit routines for each

administration of the CRCT focused on: test security protocols; test administration; and detailed APS erasure analysis

▪Administration’s Response–APS Internal Auditor will develop a plan to conduct

regular audits for each CRCT test administration


22

▪BRC Process Recommendation 7– Document any testing irregularities identified and specific plans for

follow-up and resolution, including referrals to the Georgia Professional Standards Commission and/or Fulton County District Attorney's Office, as appropriate

▪Administration’s Response– Strictly adhere to all of the Georgia Department of Education’s

testing and assessment processes for reporting testing irregularities– Establish a reporting timetable to facilitate communication among

relevant implementing departments– Identify resources for collection, review, entry and reporting of

data

Leadership Recommendations

▪Leadership recommendations involve–Driving culture change–Re-assessing current incentive, performance and

evaluation systems–Utilizing new forms of review and feedback–Publishing and addressing any additional findings related

to the 2009 testing period–Reinforcing and communicating existing, supporting

policies

23


24

▪BRC Leadership Recommendation 1–Drive culture change throughout the district to create an

environment that effectively balances attention to measurable student outcomes with attention to positive, ethical behavior

▪Administration’s Response–All educators will be required and held accountable for

providing effective instructional environments while complying with all ethics standards–Encourage employees to be transparent in internal and

external communications


25

▪BRC Leadership Recommendation 2–Proactively re-assess incentive systems and performance

evaluation processes for all district employees with a focus on achieving this appropriate balance

▪Administration’s Response–The current incentive programs and performance

evaluation processes will be reviewed as APS develops future compensation approaches


26

▪BRC Leadership Recommendation 3–Establish annual goals and performance plans that

include a significant emphasis on leadership, integrity and ethics

▪Administration’s Response–Ethics training/certification program will be provided

annually to employees district-wide


27

▪BRC Leadership Recommendation 4–Incorporate self-assessment into the annual plan

evaluation processes for leadership

▪Administration’s Response–A self-assessment process will be developed and

implemented within the existing evaluation process for all instructional leaders


28

▪BRC Leadership Recommendation 5–Incorporate surveys of teachers and support staff into

annual performance planning for principals to provide qualitative evidence on principal performance

▪Administration’s Response–A standardized survey instrument will be used to assess

school climate district-wide–School climate survey results will be utilized as an artifact

to provide feedback to principals in the evaluation process


29

▪BRC Leadership Recommendation 6–Provide observations of principals, which may include

shadowing or other efforts, for assessments of leadership ability compared to the critical skills determined for the position. Incorporate data on observed behaviors into the annual assessment process for leaders to assess their competency relative to those skills deemed to be critical for each position

▪Administration’s Response– Coordinate site visits or instructional audits to assess the use of

instructional best practices and procedures– Feedback from site visits and instructional audits will be provided

to principals and used to inform the principal evaluation process


30

▪BRC Leadership Recommendation 7–Provide opportunities for regular, anonymous peer and

upward feedback as well as processes for following through and managing this information

▪Administration’s Response–School Climate Survey referenced in Recommendation 5–Research being conducted on other feedback

instruments to provide this information


31

▪BRC Leadership Recommendation 8–Conduct principal peer reviews and incorporate this

information into annual evaluation processes

▪Administration’s Response–A peer review process will be standardized for district

principals


32

▪BRC Leadership Recommendation 9–Provide the ABE additional recommendations on plans to

monitor the principals and other leaders of the schools being referred by the BRC

▪Administration’s Response–An experienced educational specialist has been

appointed to support and monitor principals during implementation of accelerated academic recovery plans


33

▪BRC Leadership Recommendation 10– Reinforce and publicize externally and internally APS' policies that

protect APS employees who report misconduct or violations from reprisals, retaliation, and other hostile workplace actions

▪Administration’s Response–An external company has been selected to provide a

confidential hot line for employees to report misconduct–A plan will be developed and implemented to facilitate

on-going communication with employees regarding school district policies in the areas of whistle blower protection, ethics, and conflict of interest


34

▪BRC Leadership Recommendation 11–Publish additional findings made with respect to events

that may have occurred during the 2009 testing period

▪Administration’s Response–A communication plan has been developed–The district website will be utilized for regular updates to

the CRCT portal


35

▪BRC Leadership Recommendation 12– Given that the findings relate to events that may have occurred

during the 2009 CRCT and that personnel changes may have occurred since that time period, APS must provide appropriate support for the implementation of recommendations by the current leadership of schools

▪Administration’s Response– An educational specialist will support and monitor the

implementation of recommendations– Leadership and staff will be provided necessary support, training

and resources to implement the recommendations district-wide

CRCT Comparison Recommendations

36

▪BRC CRCT Comparison recommendations encompass:–Additional recommendations focused on the findings

related to the comparison of the 2009 and 2010 CRCT results


37

▪BRC CRCT Comparison Recommendation 1– Given the high number of declines in Math (2010 CRCT), the BRC

recommends that APS conduct a broad assessment of math outcomes to further determine the root causes of the declines

▪Administration’s Response– The use of state instructional frameworks will be standardized

district-wide and the monitoring process will be updated and aligned to ensure the correct use of frameworks– Existing mathematics coaching structures will be revised to ensure

job embedded support for all schools with mathematics performance declines– The professional development strategy for improving mathematics

will be reviewed


38

▪BRC CRCT Comparison Recommendation 2– The BRC recommends that the District conduct a further in-depth

assessment in the 33 schools that demonstrated a significant decline in 2010 CRCT test results

▪Administration’s Response– Based upon spring CRCT scores, the district will conduct further in-

depth analysis to determine root causes– The district will provide support, progress monitoring and targeted

intervention strategies for the selected schools to improve instructional quality

Next Steps

▪Committed to implementing 100% of recommendations

▪ Collaborative effort across several divisions and departments

▪ Ensure full implementation, we will utilize:–APS Project Management Methodology–Balanced Scorecard (BSC) for reporting progress▫ Semi-annual report▫Annual report▫Quarterly reports to ABE

39

▪Our highest responsibility is to provide the best possible education for our students in every classroom and in every school

▪We will continue to follow up on the BRC investigative report while staying focused on our main purpose – greater student achievement

Next Steps

40

response to the blue ribbon commission’s recommendations of the 2009 criterion referenced...

Documents

brc security recommendation

caveon recommendations

overview recommendations

caveon security measures

annual security goals

crct slide

defined security duties

recommendation plan