nycsa public comments on proposed e&r methodology

8/11/2019 NYCSA Public Comments on Proposed E&R Methodology

1/8

May 29, 2012

SUNY Charter Schools Institute/

State Education Department

Attn: Webinar Feedback

Submitted via email to:[email protected]

Thank you for the opportunity to comment on the proposed methodology for setting enrollment

and retention requirements for New York charter schools. The creation of the target setting

methodology and its proposed implementation process is an extremely complex task. The

materials associated with the statistical developments and possible outcomes clearly reflect an

intense and sustained effort by SED and CSI staff members to be fair and transparent while

meeting all legal obligations.

NYCSA appreciates the release of the technical materials and data files which help to present a

clear picture of how the targets will impact schools. Some of the most thoughtful and

appreciated components of the proposed methodology include the decisions to use grade bands,

compensate for students attending NYCs District 75, and incorporate standard error bands into

the effective targets. Despite the attempt to always use statistically valid formulas, there were

several important decisions that reveal the willingness of SED and CSI to make judgment calls

grounded in either reality (by using whatever data is readily available) or common sense. For

example, the decision to evaluate any school with one high school grade against city-wide targets

reflects an adjustment made for practical data purposes and city-wide choice, while using

discretion with the many small subgroups reflects the willingness of the authorizers to use

discretion when the data isnt statistically sound.

With this in mind, NYCSA respectfully requests consideration of the following questions and

comments:

Three Year L ook-Backis Not Adequately Defined

Based on a careful reading of the FAQ document, the technical memo, and an understanding of

how data is collected in SIRS, it appears that a schools classification records for current year
mailto:[email protected]:[email protected]:[email protected]:[email protected]


2/8

2

data, and classification records for an additional two years prior, will comprise the three year

look back. Because it is referred to as a three year look-back, we are concerned it confuses

schools regarding which specific sets of data are involved, if it actually only looks back two

years prior.

The Three Year Look-Back Should be Extended

Any student who was ever classified as an English language learner or has an IEP should be

counted toward a schools enrollment, regardless of the number of years that passed since the

declassification. Each student should count from entry until they move on from the school.

Many charter schools sense that this isnt enough to compensate for the justifiable concern that

their targets will be set based upon school systems that more aggressively classify students, and

are often slow to get their students to proficiency and de-classified. Many charter schools

attempt to declassify students as quickly as possible. Other charter schools use robust Response

to Intervention methods and targeted Academic Intervention Services to avoid classifications

altogether. The changes to the Charter Schools Act were not intended to negatively contribute areason for charter schools to stop warranted declassifications. And in light of the over-

classification of many students in poverty, the execution of these targets should be thoughtful

about avoiding unintentional disincentives.

Phase-in Approach to Implementation

Using a phase-in approach by applying the target analysis only to new cohorts of students

between now and a schools first renewal should be considered. Student enrollment

demographics for current cohorts cannot and should not be changed by the charter schools

(lottery regulations require schools to readmit current students without the lottery), but they can

change recruitment and enrollment outreach and policies and procedures for entering cohorts,which would result in the most fair comparison. Understanding that such a decision may prove

logistically impossible, such factors ought to be considered as part of the good faith portion of

renewal decisions.

No Proposed Enrol lment Target Methodology for Subgroups with Fewer than Ten Students

Many charter schools are impacted by this issue and no information has been shared regarding

this process. If no statistical methodology will be used, and it is solely based on subjective

discretion, then the question follows why the authorizers have chosen to implement two different

approachesone statistical method for larger schools, and one for smaller (or newer) schools

with fewer than ten students in one of the three subgroups. Early data estimates suggest this

affects at least seventy currently open schools across the state in at least one subgroup. Since a

large number of schools are impacted by this issue, this discussion is relevant prior to making a

final decision about how to combine targets if the approach is adopted.


3/8

3

No Proposed Retenti on Target Methodology for Subgroups with Fewer than Thir ty Students

Authorizer discretion will be used to determine retention outcomes for subgroups with fewer

than 30 students, but no further explanation has been provided. Early data estimates suggest this

affects at least one subgroup for approximately 97 percent of charter schools. We also noticed

there are more than 30 students in at least one subgroup for an estimated 50 percent of charter

schools, but the empirical analysis does not provide a target, effective target, or percentages,

which likely is a result of the number of students receiving a Type 2 discharge.

Schools Should Be Able to Request a Target Adjustment

It has been noted that the authorizers are still working to determine if and when target

adjustments will be made during the charter term. Schools should be able to request an

adjustment when there have been major changes or enrollment shifts in their district or at their

school.

The Source of Student-Level Data in SIRS is UnclearThe materials provided lack a clear and consistent description of exactly which data have been

used to drive the empirical analysis. Are enrollment numbers those taken on BEDS day, as noted

in the Briefing Memorandum? The date October 1, 2011 was not BEDS day for the past school

year. If it isnt BEDS data, are the numbers based on an average of enrollment on multiple days,

and if so, how many? It is important for schools to understand when the numbers are gathered

and which specific data points are used. Until it is clear what data are used, it will be difficult for

the schools to verify. Since this data are ultimately tied to renewal decisions, data transparency

is paramount.

Data Concerns

The spreadsheets containing the Empirical Analysis has many inconsistencies. While it is clearly

stated that data files are for illustrative purposes and that the data has not been verified, certain

problemssuch as the grade span servedare basic enough that a comment on this point seems

warranted. The following issues may need clean-up:

Grade Bands

Some charter schools have enrollment grade bands with gapsschools that are

enrolling either up or down, or sometimes in both directions, and serve K-2 and 5-10 for

example. The methodology does not seem to correct for such intricacies. It appears any

school with a gap will be compared to the averages for all grades in the range,

regardless of whether students are being served by each, which would be an inaccurate

comparison.

The grade bands for a number of charter schools in the column October 2011 Grade

Spanare incorrect. Given the methodological approach to compute targets based on a

charter schools specific grades, we hope all will be confirmed prior to roll-out.


4/8


5/8

5

o Family Life Academy Charter School;

o Girls Preparatory Charter School of the Bronx;

o Girls Preparatory Charter School of New York;

o Green Tech High Charter School;

o Harlem Success Academy Charter School 3;

o

Harlem Village Academy Leadership Charter School;

o Harlem Success Academy Charter School;


o Harlem Link Charter School;

o Harlem Childrens Zone Promise Academy Charter School;

o Harlem Childrens Zone Promise Academy II Charter School;


o Hebrew Language Academy;

o Hyde Leadership Charter School;

o

Hyde Leadership Charter SchoolBrooklyn;o Inwood Academy for Leadership Charter School;

o Innovate Manhattan Charter School;

o John W LaVelle Preparatory Charter School;

o Kings Collegiate Charter School;

o King Center Charter School;

o KIPP Infinity Charter School;

o KIPP Academy Charter School;

o KIPP AMP Charter School;

o KIPP STAR Charter School;

o

LaCima Charter School;o Leadership Preparatory Bedford Stuyvesant Charter School;

o Leadership Preparatory Ocean Hill Charter School;

o Lefferts Gardens Charter School;

o Metropolitan Lighthouse Charter School;

o Mott Haven Academy Charter School;

o New World Preparatory Charter School;

o Ocean Hill Collegiate Charter School;

o Summit Academy Charter School;

o Syracuse Academy of Science Charter School (this is an example of a school

with grade gaps: in 2011 it serves K-2 and 5-12, but not grades 3 and 4);

o Rochester Academy Charter School;

o True North Rochester Prep Charter School;

o Williamsburg Collegiate Charter School;


6/8

6

Enrollment Inconsistencies

The column Total Enrolled on Oct 1, 2011 appears to have data for some charter

schools that is quite different from what has been reported in preliminary BEDS data

from October 2011. In some instances the data is also drastically different from what

would be expected when extrapolating from confirmed and final 2010 BEDS data, evenafter accounting for new grades and normal enrollment fluctuations.

Missing Schools:

The following NYCSA member charter schools were not included in the Empirical

Analysis for Enrollment Rates data file (if they are missing in the Empirical Analysis

database, were they also excluded from the calculation of the targets?):

o Achievement First Apollo Charter School;

o

KIPP Tech Valley Charter School;o Harlem Success Academy Charter School 5;

o Success Academy Bedford Stuyvesant Charter School 1;

o New York Center for Autism Charter School;

o Renaissance Charter High School for Innovation;

o Enterprise Charter School;

o Health Sciences Charter School;

o Eugenio Maria de Hostos Charter School;

o Urban Choice Charter School;

o Discovery Charter School.

The following issues were noted in the Empirical Analysis for Retention Rates data file:

o New York Center for Autism Charter School is missing;

o Ross Global Charter School is now closed, but is included in the file.

We did not complete a comprehensive review, but did notice the following district

schools are missing from the Empirical Analysis for Enrollment data file (again

suggesting a further review is warranted):

o Martin Road Elementary School in Lackawanna;

o PS 59 Dr Charles Drew Science Magnet in Buffalo;

o Hempstead Early Childhood Center in Hempstead;

o Franklin School in Hempstead;

o Fulton School in Hempstead;

o Jackson Main Elementary School in Hempstead;


7/8

7

o Barak Obama Elementary School in Hempstead;

o Jackson Annex School in Hempstead.

Miscellaneous Data Issues

Child Development Center of the Hamptons is located in the Wainscott district. It is a

small district, but still should serve as the district of location for the development and

implementation of these enrollment and retention targets. To combine its targets with the

East Hampton district is to singularly treat that charter school differently. Or, if the

decision has been made to compare the school to the district where most students reside

(rather than the district of location for the school) then the question follows whether other

schoolssuch as Charter School for Applied Technologies or Global Concepts Charter

Schoolshould also be considered in a similar fashion.

There is no discussion of whether School 84 in Buffalo should be (or already has been)

excluded from the district-wide calculations in the same way District 75 was excluded inNew York City. As it appears to serve a similar function as District 75, we suggest that it

should be excluded from the calculations as well.

There have been concerns that some New York City district schools are inaccurately

listed as having 100 percent free- and reduced-price lunch populations as a result of

participation in the Universal School Meals program, but in many cases the actual rate is

lower. The use of these data drive up the target for charter schools.

Combination Targets that Factor i n Standard Error Bands

The two options presented in the technical materials reflect an either-or scenario: either threetargets will be set using the effective target and the lower end of the error band, or a combination

target based on a target without error bands. The authorizers should consider combining both

methods.

Similar to their establishment of effective targets for small subgroups, authorizers should use

discretionin the creation of metrics that impact charter renewal. The banded approach

maximizes the statistical benefit of the doubtfor charter schools while the combination target

ensures that no at-risk category receives preference over the other. Moreover, the need for

judgment decisions for more than a third of the currently open charter schoolsenrollment targets

in at least one subgroup, and nearly all of the open schools for at least one retention target, means

that the authorizers have already accepted the concept of using multiple measures

simultaneously.

This approach may not be traditional, but so far the methodology and decision-making process

suggest that the emphasis has been to establish a methodology that is fair and clear. We


8/8

8

appreciate that such a metric may not be perfectly valid for a statistical standpoint, but we

suggest that was not the primary aim of the statute. Creating a combined metric will flag schools

that are truly struggling with their enrollment efforts and will better focus renewal analysis.

nycsa public comments on proposed e&r methodology

Documents