pennsylvania charter schools: what’s working and what’s not christopher nelson, ph.d. the...

Pennsylvania Charter Schools: What’s Working and What’s Not

Christopher Nelson, Ph.D.The Evaluation CenterWestern Michigan Universitychristopher.nelson@wmich.edu

http://www.evaluation.wmich.edu

The Evaluation Center Western Michigan University

Purposes of the Talk Identify key strengths and

weaknesses Identify key policy issues Generate discussion of options for

addressing weaknesses and shoring up strengths

Outline Background and data sources Evaluative criteria Key findings Questions and discussion

Background and Data Sources

Background on the Evaluation Act 22 mandates evaluation after 5 years Contract awarded through competitive

bidding process Current report is culmination of 4 years of

field research and secondary data analysis One of the longest sustained charter school

evaluations Evaluation report focuses mainly on

aggregate trends and patterns

Data Sources Site visits to nearly all charter

schools Interviews Focus groups Observations

Data Sources (cont’d) Analysis of secondary data

PSSA Demographic information Financial data

Charter school annual reports In-depth information on school missions

and practices

Data Sources (cont’d) Documentation provided by schools

Mission statements Internal evaluations

Interviews with district officials Interviews with other stakeholder

groups

Evaluative Criteria

Factual Claims +

Criteria of Merit & Worth

Evaluative Judgments

Evaluative Criteria

Evaluative judgments combine facts and values

Thus, data alone cannot fully address questions of merit and worth

Evaluative Criteria . . .According to Act 22

“Improve pupil learning . . .”

“Provide expanded choices . . .”

“Increase learning opportunities for all pupils . . .”

“Encourage . . . different and innovative teaching practices . . .”

“Create new professional opportunities for teachers . . .”

“Hold charter schools accountable . . .”

Key Findings

Student Achievement

Key FindingsStudent Achievement

Student achievement as an evaluative criterion

Key issues Aggregate trends Consistency of the charter school effect Correlates of success

Charter schools All PA public schools

Charter school students have typically scored well below the average PA student

But score levels say more about student composition than school effectiveness

150 point difference

Charter school students have scored only slightly below those in demog. similar noncharter schools

But matching on demographics might mask less tangible differences (e.g., “interventionist” parents)

Charter schools Demographically & geographicallysimilar schools

36 point difference

Average charter school student has gained 15 points per year, net of changes in student composition

1998 1999 2000 2001 2002

1997 cohort

1998 cohort

1999 cohort

2000 cohort

Zero indicates parity with comparison schools

Key FindingsStudent Achievement Yet, there is considerable variation

among schools – some positive

Year1999 2000 2001 2002

among schools – others negative

iltere

Year1998 1999 2000 2001 2002

among schools

Year1998 1999 2000 2001 2002

Key FindingsStudent Achievement – the Bottom Line

Achievement appears to be a source of modest strength

But considerable school-to-school variation

Not yet clear what the drivers of success are

Key Findings

Choice and Satisfaction

Key FindingsChoice and Satisfaction

Consumer satisfaction as an evaluative criterion

Key issues Satisfaction with charter schools Number of charter school options Are charter school options any different than those

in noncharter public schools? – the question of “innovation”

Parents, students, and teachers are generally satisfied

However, unable to make comparisons with noncharter schools

I am satisfied with the instruction offered

4% 5%14%

Stronglydisagree

Disagree Neitheragree ordisagree

Agree Stronglyagree

The choice “halo” effect?

Typical charter school has a self-reported waiting list of 28% of current enrollment

Yet, schools report turnover rates of 0 - 40% per year Dissatisfaction, counseling out, or

realization of poor fit?

Key FindingsChoice and the “Supply” of Options

The number of charter school options has grown considerably

1997-98 1998-99 1999-2000 2000-01 2001-02

10,000

15,000

20,000

25,000

30,000

ber of Students

SchoolsStudents

Key FindingsChoice and the “Supply” of Options

Yet, those options remain concentrated in a few regions of the Commonwealth

Only 18 of 67 counties (27%) have charter schools

Of those, charter enrollment exceeds 1% in only 10

County2001/02

Enrollment(% total)

Philadelphia 7.9

Clinton 4.4

Beaver 4.2

Bucks 3.8

Chester 2.5

Charter EnrollmentTop 5 Counties

Key FindingsChoice and Innovation

Many charter schools offer new choices to particular education markets

Non-traditional grade groupings (67%)

Unique mission foci Non-traditional use of

time and calendar Smaller schools &

classes Cyber schools

Non-traditional grade groupings (67%)

Unique mission foci Non-traditional use of

time and calendar Smaller schools &

classes Cyber schools

Key FindingsChoice and Innovation

However, there is little evidence that charter schools are creating wholly new educational approaches

Barriers to innovation Slack time Slack resources Challenges of start-up Consumer preferences Problems with CAB

definition

Key FindingsChoice and Innovation – the Bottom Line

Parents, students, and teachers appear to be quite satisfied with their charter schools

Number of charter school options has grown considerably but remains geographically concentrated

Many charter schools offer new options in particular markets, but few appear to be producing wholly new educational approaches

Key Findings

Equity and Access

Key FindingsEquity and Access

School choice . . . Engine of desegregation? Sorting machine?

Key issues Types of communities served Types of students selected (self-

selected?) from these communities

Key FindingsEquity and Access – School Location

Charter schools tend to be in areas with larger schools, lower test scores, and higher concentrations of low income and nonwhite students.

Variable Nonchartering Chartering

Median total district enrollment (2001-02) 1,749 3,446

Median per-pupil expenditure (2000-01) $8,319 $8,304

Median aggregate PSSA score (2001-02) 1341 1221

Median % low income students (2001-02) 20.3 63.9

Median % nonwhite students (2001-02) 4.9 67.0

Key FindingsEquity and Access – (Self?) Selection

Declining relative percentage over time

Considerable school-by-school variation

Charter-Host Demographica Differences

1997-98 1998-99 1999-00 2000-01 2002-02

)%Nonwhite

More nonwhite students

Slightly more low income students

Relatively stable over time

Charter-Host Differences

1997-98 1998-99 1999-00 2000-01 2002-02

)%Nonwhite

Fewer students with IEPs

IEP students in charter schools tend to have less severe disabilities

Charter-Host Demographica Differences

1997-98 1998-99 1999-00 2000-01 2002-02

)%Nonw hite

Key FindingsEquity and Access – The Bottom Line

On the whole, charter schools have target communities of need

In the aggregate, little systematic evidence of cream-skimming

Possible exception of special education Outlier schools might raise some red flags Negative trend on race Some anecdotal evidence of cream-skimming

Key FindingsEquity and Access – The Bottom Line

But, findings limited by limitations in using enrollment data

Family self-selection vs. school selection

Key Policy Issues

Achievement

Key Policy Issues & OptionsAchievement

What are the correlates of success?

How can we reduce variability on the low end?

What is the potential for success?

Study correlates of success

Facilitate identification and dissemination of best practices

Key Policy Issues & OptionsAccountability as Possible Correlate

Accountability lies at the heart of the charter concept

PA has done better than most states in reporting and technical assistance

Yet, there is still room for improvement

Continue improvements to charter school annual report format

Make information public to facilitate market accountability

Continue providing technical assistance in developing clear, measurable goals

Consider assessment of LEA capacity for oversight (wills, skills, & bills) Consider the impact of “just-in-time”

oversight by many LEAs Consider targeted state oversight

based on past performance & compliance

Key Policy Issues

Choice and Equity

Key Policy Issues & OptionsChoice & Equity

Is choice good in itself or as tool for increasing achievement?

Is innovation reasonable for charter schools?

What counts as “innovation”?

Possible trade-off between choice and equity

Slippery relationship between satisfaction and achievement

Slippery Relationship Between Achievement & Satisfaction

r = -0.08 (p = 0.64)

rage a

(filter

Mean satisfaction level (parents, teachers, and students)3 3.5 4 4.5

Key Policy Issues

Cross-Cutting Issues

Key Policy Issues & OptionsCross-Cutting Issues

Balancing achievement, choice, and equity – the challenges of multi-attribute evaluation systems

Cost-effectiveness How do charter schools stack up against other

investments in educational improvement? Actual vs. potential charter school

performance Addressing questions of fact and value

Final Thought “By providing important data and

raising key policy issues, this report has sought to make a sound contribution to the debate over the Commonwealth’s charter schools”

pennsylvania charter schools: what’s working and what’s not christopher nelson, ph.d. the...

Documents

fine arts - wmich.edu

annual report - wmich.edu

travel agent login - wmich.edu

forum speakers discuss medical school - wmich.edu

charter school turnaround: proving what’s possible seth ·...

open optical flow paper - wmich.edu

graduate training handbook - wmich.edu

counseling psychology - wmich.edu

virtual 3mt handbook 2020 - wmich.edu

lecture 27 low speed psp large tunnel - wmich.edu

chapter i introduction to the study - wmich.edu

sherine o. obare, phd - wmich.edu

what’s in a name?? randi mcmillen vance charter school:...

flvneritt lrtews - wmich.edu

radiological controls program - wmich.edu

10/22/19 - wmich.edu

1. 2. 3. - wmich.edu

hisp 1240 2 - wmich.edu

ppt chapter 01 - wmich.edu

radiological controls program purpose - wmich.edu