Effects of Poverty, Funding Structure and Scale on Public School System Performance

John MackenzieFREC/CANR, University of Delaware

May, 2010

BACKGROUND:

Coleman Report (1966): finds link between money and school performance to be tentative at best. (Basic problem: lack of data!)

Eric Hanushek (1981,1986,1997): “Throwing Money at Public Schools” meta-analyses of studies relating funding to school performance. (Methodological error in counting studies as datapoints:7 heads in 10 coin tosses does not prove a coin is biased, but combining 100 trials of 10 tosses and getting 7 or more heads in 60% of the trials does.)

Jay Greene (Manhattan Institute): Real per-pupil spending “almost doubled” from 1972 to 2002, while NAEP scores did not improve much at all.(So what? Real per-capita disposable incomes “almost doubled” too. Compare rising costs of college!)

Public schools today are…more inclusive of minorities, immigrants, etc.;offer a broad array of non-traditional services;serve an expanded population of poor children;deliver more remedial & special education.

Education is typically a “luxury” good: as incomes rise, households invest larger proportions of their incomes in education. Education confers status; may be a positional good.US income inequality continues to increase, and the relative economic return to a HS diploma is falling.

If education lifts families from poverty for multiple generations, is residual poverty becoming more intractable? Does the rising real cost of public schools simply reflect the rising marginal cost of eliminating poverty?

School Spending/Pupil vs. Median Household Income, by State

y = 0.1242x + 2793.5R2 = 0.2252

US: $8,287 per pupil$44,231 median income

Elasticity = +0.66

$4,000

$6,000

$8,000

$10,000

$12,000

$14,000

$30,000 $35,000 $40,000 $45,000 $50,000 $55,000 $60,000

Median Household Income, 2003

K-1

2 P

ublic

Sch

ool S

pend

ing/

Pup

il, 2

004

Is Public Education a “Luxury” Good?

SAT05 vs. Total 2004 Per-Pupil Spending, by State

950

1000

1050

1100

1150

1200

1250

$4,000 $5,000 $6,000 $7,000 $8,000 $9,000 $10,000 $11,000 $12,000 $13,000 $14,000

A favorite neo-con hobbyhorse: “Throwing Money at Schools”

NAÏVE UNIVARIATE MODELS [t-statistics in brackets]

SAT02 = 1187.16 – 0.0156*TXPP02 {25.70] [-2.59] N=50; R-square=0.123

SAT03 = 1185.38 – 0.0143*TXPP03 [26.06] [-2.51] N=50; R-square=0.116

SAT04 = 1181.65 – 0.0131*TXPP04 {26.85] [-2.49] N=50; R-square=0.114

SAT05 = 1196.88 – 0.0117*TXPP05 [28.45] [-2.87] N=50; R-square=0.146

More waste and bloat in public education?School spending rises and SAT scores fall!

They suredo, Bob!

Those kids get dumber every

year, Jill!

Is my hair OK?

State mean SAT scores depend on test participation rates

SAT04 = 993.23 - 59.574Ln(%Participation)

R2 = 0.8435

950

1000

1050

1100

1150

1200

1250

0% 20% 40% 60% 80% 100%

Percent of seniors taking the SAT in 2004

20

04

SA

T v

erb

al+

ma

th

REGRESSION MODELS CONTROLLING FOR PARTICIPATION

SAT02 = 1062.5 + 0.0135*TXPP02 – 244.58*Partic02 [43.29] [3.46] [-12.82]

N=50; R-square=0.806

SAT03 = 1074.5 + 0.0113*TXPP03 – 234.60*Partic03 [43.29] [3.46] [-12.82]

N=50; R-square=0.809

SAT04 = 1079.4 + 0.0104*TXPP04 - 230.50*Partic04 [48.65] [3.37] [-13.00]

N=50; R-square=0.807

SAT05 = 1095.9 + 0.0070*TXPP05 - 221.17*Partic05 [46.60] [2.63] [-11.47]

N=50; R-square=0.775

A textbook example of an omitted variable problem:

ECONOMETRIC DIGRESSION

A formal treatment of the participation bias problem:

When only high-performing students take the SAT the mean score is biased upward. Expanding participation to include more typical students educes the bias.

Low SAT participation boosts the mean scores of low-performing states above the mean scores of high-performing states.

2005 ACT Participation vs. 2005 SAT Participation, by State

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

2005 SAT Participation vs. Total 2004 Per-Pupil Spending, by State

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

$4,000 $6,000 $8,000 $10,000 $12,000 $14,000 $16,000

FORMAL HECKMAN MODEL:PROBIT: NORMSINV(Partic%05) = ExpPP04, ACTPart%05

Regression Statistics

Multiple R 0.9153

R Square 0.8378

Adjusted R Square 0.8309

Standard Error 0.3970Observations 50

ANOVA

df SS MS F

Regression 2 38.245 19.122 121.357

Residual 47 7.406 0.158Total 49 45.651

Coefficients Standard Error t Stat P-value

Intercept -0.093569 0.374798 -0.249651 0.803945

ExpPP04 0.000107 0.000038 2.790748 0.007576ACTPart% -2.531849 0.211647 -11.962623 0.000000

PROBIT Residuals

State Predicted Residuals Lambda PredPart% SAT05PCTAlabama -1.34361 0.06205 0.17768 9.0% 10%

Alaska 0.32769 -0.27753 1.01753 62.8% 52%Arizona 0.06969 -0.50960 0.84277 52.8% 33%Arkansas -1.29832 -0.25645 0.19021 9.7% 6%California 0.37901 -0.37901 1.05379 64.8% 50%Colorado -1.80895 1.16560 0.08052 3.5% 26%Connecticut 0.80478 0.27554 1.37111 79.0% 86%Delaware 0.89688 -0.25354 1.44319 81.5% 74%Florida -0.40756 0.79288 0.55781 34.2% 65%Georgia -0.00240 0.67689 0.79636 49.9% 75%Hawaii 0.41217 -0.13285 1.07747 66.0% 61%Idaho -0.91858 0.11216 0.31873 17.9% 21%Illinois -1.67618 0.39463 0.10272 4.7% 10%Indiana 0.25853 0.15393 0.96943 60.2% 66%Iowa -0.95007 -0.69479 0.30646 17.1% 5%Kansas -1.21531 -0.12544 0.21470 11.2% 9%Kentucky -1.28251 0.10753 0.19472 10.0% 12%Louisiana -1.47621 0.07114 0.14428 7.0% 8%Maine 0.67091 0.00358 1.26839 74.9% 75%Maryland 0.58593 -0.03254 1.20453 72.1% 71%

Lambda = NORMDIST(PredY,0,1,FALSE)/(1-NORMDIST(PredY,0,1,TRUE))

= f(X)/[1-F(X)] where f and F are standard normal density and distribution

2005 SAT %Participation, by State -- Actual vs. Predicted from Probit

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

1.60

1.80

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Predicted & Actual Participation vs. LAMBDA

Heckman 2nd Stage: SAT05 = ExpPP04, Lambda

Regression Statistics

Multiple R 0.8222

R Square 0.6760

Adjusted R Square 0.6622

Standard Error 38.9339

Observations 50

ANOVA

df SS MS F

Regression 2 148622.07 74311.03 49.02

Residual 47 71244.91 1515.85

Total 49 219866.98

Coefficients Std Error t Stat P-value

Intercept 1054.6446 30.8222 34.2170 0.0000

ExpPP04 0.0148 0.0045 3.3095 0.0018

LAMBDA -145.1194 15.9954 -9.0726 0.0000

Incorporate LAMBDA as participation bias correction instrument:

Or construct the bias correction instrument directly from the observed participation rates:

Fast Heckman: SAT05 = ExpPP04, L*L* = (NORMDIST(NORMSINV(%Partic05),0,1,FALSE)/(1-%Partic05)

Regression StatisticsMultiple R 0.86749R Square 0.75254Adjusted R Square 0.74201Standard Error 34.02397Observations 50

ANOVA

df SS MS FRegression 2 165458.35 82729.17 71.46Residual 47 54408.63 1157.63Total 49 219866.98

Coefficients Std Error t Stat P-valueIntercept 1064.9402 26.1859 40.6684 0.0000ExpPP04 0.0128 0.0037 3.4751 0.0011Heckman -132.3417 11.9657 -11.0601 0.0000

Fast McFadden: SAT05 = ExpPP04, MU where MU = -%Part05*ln(%Part05)/ln(1-%part05) - ln(1-%Partic05)

Regression StatisticsMultiple R 0.85742R Square 0.73517Adjusted R Square 0.72390Standard Error 35.19792Observations 50

ANOVA

df SS MS FRegression 2 161639.0 80819.5 65.2Residual 47 58228.0 1238.9Total 49 219867.0

Coefficients Std Error t Stat P-valueIntercept 1060.261 27.343 38.777 0.0000ExpPP04 0.01335 0.00386 3.45532 0.0012

MU -74.0986 7.0262 -10.5460 0.0000

Or use the equivalent logit-based instrument (McFadden):

Fast McFadden vs. Fast Heckman

y = 1.7858x - 0.0017

R2 = 0.9985

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

0.00 0.20 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00

“50 experiments in public school system finance and structure”

Data Sources:

US Census Bureau, Public Elementary–Secondary Education Finance Data (annual State-level tables) http://www.census.gov/govs/school

US Dept. of Education, 2003, 3005, 2007 and 2009National Assessments of Education Progress (NAEP)http://nces.ed.gov/nationsreportcard/statecomparisons/

NAEP09 vs. Total 2007 Per-Pupil Spending, by State

y = 0.0035x + 967.43

R2 = 0.1343

940

960

980

1000

1020

1040

1060

1080

$6,000 $8,000 $10,000 $12,000 $14,000 $16,000 $18,000 $20,000

NAEP09 vs. 2007 FEDERAL Funding per Pupil, by State

y = -0.0312x + 1038.3

R2 = 0.1488

940

960

980

1000

1020

1040

1060

1080

$500 $700 $900 $1,100 $1,300 $1,500 $1,700 $1,900 $2,100 $2,300

AK

LA

NAEP09 vs. 2007 STATE Funding Per Pupil, by State

y = -5E-05x + 1007.4

R2 = 0.00002

940

960

980

1000

1020

1040

1060

1080

$2,000 $4,000 $6,000 $8,000 $10,000 $12,000 $14,000 $16,000

HI

VT

NAEP09 vs. 2007 LOCAL Funding per Pupil, by State

y = 0.0062x + 978.52

R2 = 0.2697

940

960

980

1000

1020

1040

1060

1080

$0 $2,000 $4,000 $6,000 $8,000 $10,000 $12,000

NJCT

NY

MA

PA

NAEP09 vs. 2007 FEDERAL, STATE and LOCAL Per-Pupil Funding, by State

Regression StatisticsMultiple R 0.62032R Square 0.38480Adjusted R Square 0.34468Standard Error 20.55471Observations 50

ANOVAdf SS MS F

Regression 3 12156.15 4052.05 9.59Residual 46 19434.82 422.50Total 49 31590.97

Coefficients Standard Error t Stat P-valueIntercept 992.89010 15.28438 64.96109 0.00000FED07PP -0.02583 0.00964 -2.67844 0.01022STATE07PP 0.00211 0.00137 1.54618 0.12891LOCAL07PP 0.00599 0.00144 4.16168 0.00014

NAEP09 vs. Percent of Students Eligible for FRPL, by State

y = -249.32x + 1112

R2 = 0.8076

920

940

960

980

1000

1020

1040

1060

1080

20% 30% 40% 50% 60% 70% 80%

2009 NAEP

4th grade

MATH

2009 NAEP

8th grade

MATH

2009 NAEP

4th grade

READING

2009 NAEP

8th grade

READING

NAEP vs. Log2(Avg. District Size), by State

y = -7.123x + 1090.6

R2 = 0.2322

940

960

980

1000

1020

1040

1060

1080

6 8 10 12 14 16 18

Targeted Federal dollars

(Title I, etc.)

State funding replacing

property tax

Less reliance on property

taxes in larger districts

Less reliance on local funding in higher-poverty

states

Larger average district size inhigher-poverty

states

NAEP08 vs. Per-Pupil Funding by Source and Log2 of Avg. District Size

Regression StatisticsMultiple R 0.7959R Square 0.6335Adjusted R Square 0.5918Standard Error 16.2225Observations 50

ANOVAdf SS MS F

Regression 5 20011.50 4002.30 15.21Residual 44 11579.47 263.17Total 49 31590.97

Coefficients Standard Error t Stat P-valueIntercept 1089.5879 21.7298 50.1425 0.0000FED07PP -0.0318 0.0077 -4.1269 0.0002STATE07PP 0.0028 0.0011 2.5162 0.0156PROPTX07PP 0.0046 0.0015 3.1430 0.0030OTHRLOC07PP 0.0059 0.0013 4.6607 0.0000L2AVDISTSZ -7.7268 1.4171 -5.4526 0.0000

NAEP08 vs. Federal, State, Property Tax and Other Local Funding, Percent Poverty and log2 of District Size

Regression StatisticsMultiple R 0.9315R Square 0.8676Adjusted R Square 0.8491Standard Error 9.8625Observations 50

ANOVAdf SS MS F

Regression 6 27408.45 4568.07 46.96Residual 43 4182.52 97.27Total 49 31590.97

Coefficients Standard Error t Stat P-valueIntercept 1127.24317 13.89842 81.10587 0.00000FED07PP -0.00488 0.00561 -0.87019 0.38903STATE07PP 0.00061 0.00071 0.85619 0.39664PROPTX07PP 0.00129 0.00096 1.34205 0.18662OTHRLOC07PP 0.00241 0.00087 2.77650 0.00811PCTPOV -197.56627 22.65538 -8.72050 0.00000L2AVDISTSZ -3.74497 0.97503 -3.84086 0.00040

of Avg. District Size

Similar results are obtained with 2007, 2005 and 2003 NAEP scores analyzed against contemporaneous or lagged funding, average district size, poverty, etc.(These are large, slowly-evolving systems)

Poverty is the strongest predictor of NAEP performance, but is collinear with federal funding (+), local funding (-) and average district size (+).

Structural Variables: Federal funding is mainly targeted to low-performing populations—Title I and special needs. State funding does not drive NAEP performance much at all.Local funding drives NAEP performance most efficiently. Average district size is negatively correlated with NAEP performance, suggesting scale diseconomies.

Implications of funding and average district size:

HI, FL*, MD*, NV, UT, NC, LA: large average district size, generally more state funding, lower NAEP scores

VT*, MT, ND, ME, SD, NE, OK, NH: many small districts, generally more local funding, higher NAEP scores

Likely sources of public dissatisfaction with schools:Displacement of local funding with state funding, and consequent loss of local control.Legacy of desegregation, Serrano, consolidation, bureaucratization, political intervention and accumulation of regulation, union protection of teacher seniority, etc.

These results suggest the efficiency of public education can be improved by restoring local autonomy of school systems.

Why would local dollars drive school system performance more efficiently than state dollars?Local funding implies stronger local governance and accountability.