assessing the impact of north carolina's early college high

Working Paper: Assessing the Impact of North Carolina’s Early College High Schools on College Preparedness

Luke Miller1 & Matthew Corritore2

Updated 11 December 2012.

Center on Education Policy and Workforce Competitiveness University of Virginia

PO Box 400879 Charlottesville, VA 22904

CEPWC working papers are available for comment and discussion only. They have not been peer-reviewed. Do not cite or quote without author permission.

This research is funded by a research grant (NSF Award 0723412) from the National Science Foundation to The

Urban Institute and completed when both authors were on staff. The opinions expressed are those of the authors and do not necessarily represent views of the National Science Foundation. The authors would like to thank our research

collaborators, Julie Edmunds and Nina Arshavsky at The SERVE Center at the University of North Carolina – Greensboro and Ross Milton and Joel Mittleman of the Urban Institute, for their valuable assistance and their

insightful and valuable questions and comments. We would also like to thank Tom Dee, Jane Hannaway, Austin Nichols, Randy Reback, Kim Rueben, Becky Smerdon, Doug Wissoker, and Zeyu Xu for helpful comments. All

errors are our own.

CEPWC Working Paper Series No. 7. January 2013.Available at http://curry.virginia.edu/research/centers/cepwc/publications.

Curry School of Education | Frank Batten School of Leadership and Public Policy | University of Virginia

Early College High Schools are small, innovative public schools that offer students the oppor-tunity to earn both a high school diploma and two years of college credit in four or five years free-of-charge. The model has been replicated more than 230 times in 28 states and the District of Columbia. We study 33 Early Colleges in North Carolina to measure their effect on student progression through the mathematics and science pipelines each of which consist of three college-preparatory courses. The pipeline concept stresses the importance of when a student takes a course, not just if she does. Our analysis of student-level longitudinal data comparing Early College students to carefully matched non-ECHS students within the same district finds positive effects on mathematics course-taking and performance but nil to negative effect in science. Early Colleges are also found to narrow differences in successful mathematics pipeline progression between student subgroups defined by parental education and 8th grade math-ematics achievement.

Center on Education Policy and Workforce Competitiveness

1University of Virginia405 Emmet Street South

PO Box 400277Charlottesville, VA 22904

[email protected]

2American Institutes for Research

Early College High Schools and College Preparedness

1 CEPWC Working Paper Series No. 7. January 2013.

Available at http://curry.virginia.edu/research/centers/cepwc/publications. Curry School of Education | Frank Batten School of Leadership and Public Policy | University of Virginia

ASSESSING THE IMPACT OF NORTH CAROLINA’S EARLY COLLEGE HIGH SCHOOLS ON COLLEGE

PREPAREDNESS By Luke Miller, University of Virginia, & Matthew Corritore, American Institutes for Research

Introduction

Labor market returns to education are substantial. Compared to high school graduates,

median weekly earnings in 2011 for non-institutionalized full-time workers aged 25 year or older

without a high school diploma were 29 percent lower, workers with an Associate degree were 20

percent higher, and workers with a Bachelor’s degree were 65 percent higher (Bureau of Labor

Statistics, 2012). The implications for economic inequality by race and ethnicity are quite concerning

what with educational attainment much higher among white workers than non-white workers (U.S.

Census Bureau, 2012). Among individuals aged 25 years or older in 2011, whites relative to blacks

were 36 percent more likely to have earned at least an Associate’s degree and 41 percent more likely to

have earned a Bachelor’s degree. The differences are even more pronounced between white and

Hispanics: 53 and 59 percent, respectively. Significant inequalities in educational attainment across

race and ethnicity cause inequalities in employment and income. The importance of post-secondary

education and training to personal and societal well-being is forecasted to grow. One estimate

forecasts 63 percent of the newly created jobs by 2018 will require at least some college education

(Carnevale, Smith, & Strohl, 2010). Failure to equip workers with the necessary skills will depress

future economic expansion and exacerbate the growing inequities between segments of the

population (Sherman & Stone, 2010).

America’s K-20 educational system plays a prominent role in ensuring the alignment of

workers’ skills to the skills needed for 21st Century jobs and delivering on the promise of equal

opportunity for economic stability for all (Partnership for 21st Century Skills, 2008). Post-secondary

educational success is closely linked to high school success and course-taking (Adelman, 2006; Long,




Conger, & Iatarola, 2012); yet, many individuals do not successfully transition between high school

and college. Conditional on graduating from high schools, only 42 percent earned at least an

Associate’s degree and less than a third (30 percent) held a Bachelor’s degree with rates much lower

among blacks and Hispanics than whites (U.S. Census Bureau, 2012). High schools are under

significant pressure from diverse stakeholders to implement policies and programs to assist students,

particularly those from underrepresented groups, transition successfully to post-secondary education.

The Early College High School (ECHS) model, with more than 230 replications in 28 states

and the District of Columbia, re-envisions the high school educational experience to be one in which

students earn both a high school diploma and a two-year college degree (or two-year’s worth of

transferrable college credit) within four or five years. This goes beyond most college transition

programs such as Advanced Placement or International Baccalaureate which tend to provide students

the opportunity to earn far fewer college credits than does the early college model. Typically located

on a college campus, the rigorous blended academic program offered by Early Colleges enable

students a tuition-free entrée into the partnered post-secondary institution. Early College schools’

mission to target underrepresented student groups such as first-generation college-goers, racial and

ethnic minorities, and low-income students can help improve their access to current and future skilled

jobs and the economic security they provide.

Providing public school students with two years of college credits free-of-charge is not a trivial

expense. Substantial public and private funds has supported the replication of the ECHS model. The

dollars do not represent a long-term funding stream. Rigorous evidence of the model’s effectiveness

will be helpful in efforts to convince district and state officials to allocate public dollars.

Whereas many previous studies of college transition programs have focused on post-

secondary outcomes, the current study shifts the focus to its prerequisite – success in high school. We




employ the concept of a pipeline to examine the effect of Early Colleges on students’ successful

progression through a sequence of college-preparatory mathematics and science courses. Each

pipeline imposes a timeline for course completion in recognition that, for a student to graduate well-

prepared for post-secondary education and careers, it is important when, not just if, a course is taken.

In order to demonstrate mastery in four mathematics and science courses by the end of high school

(an increasingly common expectation), students must take pipeline courses in the very first year of

high school and continue taking pipeline courses in each of the later grades to avoid doubling up on

coursework. The pipeline concept allows us to assess when students are at greatest risk of falling out

of the pipeline and to measure Early Colleges’ impact on reducing that risk. Thirty-three Early

Colleges in North Carolina comprise our treatment sample.

The remainder of this paper is organized into seven sections. After reviewing the literature on

college transition programs in general and the ECHS model in particular, we describe North

Carolina’s initiative to open 75 replications of the ECHS model throughout the state. Next, we

summarize the longitudinal data analyzed and the pipeline measures constructed and then the quasi-

experimental methodology used. In the following section, we present our findings on Early Colleges’

impact on student course-taking and performance in both pipelines. The penultimate section includes

a discussion of policy implications, and the final section concludes.

College Transition Programs

The ECHS model is a college transition program in which students enroll in both high school

and college courses and earn both high school and college credit for some courses. In 2006, all but

eight U.S. states had some type of concurrent enrollment policy, and some local districts operated

their own programs in the states that did not (Karp, Calcagno, Hughes, Jeong, & Bailey, 2007;

Western Interstate Commission for Higher Education, 2006). An estimated 10 to 30 percent of high




school juniors and seniors take at least one college course in states with enduring, no-cost concurrent

enrollment programs (Hoffman, 2005). The origins of these programs can be traced back to the 1930s

and 40s when Leonard Koos pitched his “6-4-4” plan to remove the barrier between secondary and

post-secondary coursework by organizing grades seven through ten in a junior high school and grades

eleven through fourteen in a junior college (Kisker, 2006). Most modern programs allow high school

juniors and seniors to take college courses at no cost to the student. These programs are theorized to

benefits students and their families by reducing the time students need to earn a college degree, saving

families money through the receipt of free, transferable college credits, and exposing high school

students to a collegiate environment (Hoffman, 2005).

However, unlike Early Colleges, most concurrent enrollment programs are not designed for or

have struggled to serve students from at-risk or underserved populations (Conger, Long, & Iatarola,

2009; Hoffman, Vargas, & Santos, 2008; Klopfenstein, 2004). Most states require students to have

high grades and standardized test scores to participate (Hoffman, 2005), and schools serving the most

disadvantaged, lowest achieving students are less likely to offer concurrent enrollment programs

(Iatarola, Conger, & Long, 2011; Waits, Setzer, & Lewis, 2005). These programs can also be

challenging to sustain due to the cost of providing free college credits, and the difficulty of

maintaining partnerships between high schools and colleges which are institutionally separate by

design (Hoffman, Vargas, & Santos, 2009).

Research on the effectiveness of college transition programs at increasing student achievement

is sparse with the extant studies often lacking methodological rigor, sometimes not controlling for

basic student covariates. One review of the research on these programs found prior work on program

effectiveness largely inadequate, due in great part to the unavailability of relevant data (Learner &

Brand, 2006). Another review of 45 articles and reports on these programs found only 21 attempted




to quantify the effects of the program, while others were qualitative, descriptive, or only focused on

participants’ perceptions. The 21 studies that in some way quantified effects varied widely in

methodological rigor; only a few attempted to control for preexisting student characteristics, prior

student achievement, or students’ motivations to participate in the program (Bailey & Karp, 2003). A

pair of recent studies in Florida using data similar to ours provides stronger evidence of the effect of

course-taking on secondary and post-secondary outcomes. Advanced course-taking increases 10th

grade test scores and raises high school graduation and college enrollment rates (Long et al., 2012),

and differences in math course-taking explains roughly a third of the gap in readiness for college-level

math between white, non-poor students and black, Hispanic, and poor students (Long, Iatarola, &

Conger, 2009).

Among the correlational studies controlling for student demographic and academic

characteristics, a number find positive correlations between concurrent enrollment participation and

post-secondary student outcomes. Participants are more likely than non-participants to enroll in

college (Karp et al., 2007; Swanson, 2008) and more likely to pursue a bachelor’s than associate’s

degree (Karp et al., 2007). Conditional on matriculating in a post-secondary program, participants are

more likely to return for a second year of coursework (Eimers & Mullen, 2003; Michalowski, 2007;

Swanson, 2008), have higher first year GPAs (Michalowski, 2007), have higher second year GPAs

(Karp et al., 2007; Kentucky Council on Postsecondary Education, 2006), and earn a bachelor’s degree

(Swanson, 2008). A fair criticism of all these studies is that they may conflate student motivation with

participation in explaining the differences in post-secondary outcomes. Nevertheless, the positive

correlations have motivated educator and policymaker interest in the Early College High School

model.




Early College High Schools

The Early College High School Initiative was founded in 2002 by the Bill & Melinda Gates

Foundation with support from the Carnegie Corporation of New York, the Ford Foundation, and the

W.K. Kellogg Foundation, among others. All ECHS schools partner with a higher education

institution to offer high school students from underrepresented populations the opportunity to earn

one or two years of free, transferable college course credit in four or five years’ time. Students can rely

on a comprehensive support system to develop college-level skills (ECHS Initiative, 2010). The model

emphasizes rigorous instruction, relevant curricula, and supportive relationships between students and

staff, all of which are aided by the schools’ small size – no more than 100 students per grade. The

organization Jobs for the Future (JFF) offers technical assistance to intermediary organizations, each

providing ground-level support for a set of Early College schools.

A multi-year evaluation of the ECHS Initiative conducted for the Gates Foundation provides

the most comprehensive nationwide look at the model’s implementation and its potential effects on

student achievement (American Institutes for Research & SRI International, 2009; Berger, Adelman,

& Cole, 2010). A survey of all 157 Early College campuses open in 2007/08 reveals two-thirds were

created new for the purpose of implementing the model, with many of the other pre-existing schools

converting from Middle College High Schools.1 The Early Colleges have successfully enrolled high

percentages of minorities and low-income students serving on average 70 percent minority and 57

percent low-income students, versus district school averages of 64 percent and 55 percent,

respectively, between 2006 and 2009 (Berger et al., 2010). Almost two-thirds of the schools formed

partnerships with two-year colleges and universities and just over half are located on a college campus.

1 Middle Colleges differ from Early Colleges in two key respects. First, students typically do not enroll in Middle Colleges in the 9th grade. Second, Middle College students are only eligible to earn some college credit but not a full two-year’s worth as offered by Early Colleges.




Seventy-three percent of ECHS 12th and 13th graders in 2007-08 completed at least one college course,

50 percent of whom took a college course in a core academic subject (i.e., social science, history,

science, English, and mathematics). The average Early College graduate in 2007 earned 23 college

credits.

Unfortunately, data availability seriously restricted the study’s ability to access the impact of

Early Colleges on student academic outcomes (American Institutes for Researcher & SRI

International, 2009; Berger et al., 2010). Only school-level aggregate data on Early College and

conventional high schools in the same districts were available, not the preferred student-level data.

The analysis, like earlier evaluations of dual-enrollment programs, is unable to control for pre-existing

differences among students. Such comparisons are likely biased in favor of the Early Colleges because

the average Early College student is likely different from the average conventional high school student

in meaningful ways related to her performance (e.g., Early College students may be more motivated).

With that in mind, the analysis finds students attending Early College schools outperform their peers

at other high schools within the same district on statewide assessments in both mathematics and

English language arts. Also, Early College schools located on college campuses have higher

performance relative to non-Early College schools than Early College schools not located on college

campuses.

A more recent study of schools in North Carolina provides stronger evidence of the ECHS

model’s positive impact on student high school achievement. Although the school sample is much

smaller (12 schools), the analysis exploits a random assignment sample generated by over-subscription

lotteries at each of the schools to isolate the model’s causal impacts (Edmunds et al., 2012). ECHS

students both take and pass algebra I at higher rates, by 10 and 6 percentage points respectively, and

are significantly more likely to take and pass both at least one college preparatory course and at least




two college preparatory courses (i.e., algebra I, geometry, algebra II, one mathematics course beyond

algebra II) by the end of 9th grade. Taking and passing rates for English I do not differ significantly

between the treatment and control groups. Although the study’s generalizability is limited by the

selective small sample, it provides compelling evidence that the ECHS model improves student

academic achievement in high school.

Our study expands our understanding of this increasingly popular dual-enrollment/dual-credit

program by improving upon prior evaluations. Our sample includes all 33 Early College schools

opened in school years 2005/06 and 2006/07 as part of a statewide innovative high school initiative.

Hence, the external validity of our results is wider than the randomized North Carolina evaluation.

And leveraging the student-level longitudinal data maintained by the North Carolina Education Data

Research Center allows us to control for observed pre-existing differences between ECHS

participants and non-participants that limited the nationwide evaluation. We employ propensity score

matching to identify similar students attending non-ECHS schools to mimic random assignment and

then utilize a difference-in-difference approach as a robust check on these results. Our sample, data,

and methodology are described below.

ECHS in North Carolina

Although North Carolina adopted a dual-enrollment policy in 1983 as part of the Huskins bill,

the rapid growth of Early College High Schools in the state has been driven by the Learn and Earn

initiative launched in September 2004 by Governor Mike Easley. The initiative, funded by both state

monies and a $20 million grant from the Bill & Melinda Gates Foundation, seeks to increase high

school graduation rates and college readiness through the creation of smaller, innovative high schools

that provide high school students with access to college courses. Schools supported through this

effort have a priority to serve students from groups underrepresented in college—first-generation




college-goers, lower-income, minority, and students struggling in conventional high schools. Each

school is partnered with and many are located on a 2- or 4-year college campus, serves no more than

100 students per grade (or 400 students per school), and provides the opportunity for every student to

earn a 2-year college degree or two years of college transferrable credit within 4 or 5 years. In the fall

of 2011, 74 Early College schools served more than 12,000 students throughout the state, the most of

any state in the U.S. (ECHS Initiative, 2010; State Board of Education and North Carolina

Department of Public Instruction, 2011).

Each ECHS enters into a six-year partnership with the North Carolina New Schools Project

(NCNSP), a public-private organization created in 2003. The partnership includes one year of

planning and five years of implementation support. NCNSP provides each school with a variety of

services to ensure adherence to five design principles: (1) the school exists to prepare students for

college and work, (2) the school is characterized by powerful learning and teaching, (3) the staff uses

their knowledge of the students to improve student learning, (4) the staff’s shared vision of the

innovative high school is reflected in everything they do at the school, and (5) the school’s

organizational structure and allocation of resources are designed to facilitate and support the other

four design principles.2 Toward that goal, each ECHS receives funding for two school-based

positions—an additional school counselor and a work-based learning experiences coordinator—plus a

college liaison. Each school is also assigned, during the first two years, a school change coach who

serves as the change facilitator at the site and an instructional coach who works with staff on-site

during the last four years. Professional development activities for teachers and principals are provided

through the Integrated System of School Support Services (IS4) which combines the instructional

2 In 2010 (outside this study’s observation period), NSP added a sixth design principle – shared leadership working to ensure the success of every student.




coach with peer school reviews (based on the medical school rounds model) and leadership

facilitators.

The first 33 Early College schools opened through the Learn and Earn initiative comprise our

analytic sample—13 opened in Fall 2005 and 20 in Fall 2006.3 Four schools partnered with a member

school of the University of North Carolina system with the others partnering with a NC community

college. Spread throughout the state, the schools are located in 31 districts but serve students from 39

districts. Almost all the schools began with only 9th graders in the first year and have added an

additional grade each year.

Roughly 4,400 students enrolled as freshmen in these 33 Early College schools between

2004/05 and 2006/07 (see Table 1). As for the recruitment of students from subgroups

underrepresented in college, the schools have been more successful with some subgroups than others.

Compared to 9th graders statewide, ECHS freshmen are more likely to be potential first-generation

college-goers (56 versus 51 percent), have parents without bachelor’s degrees (74 versus 66 percent),

and slightly more likely to be eligible for free/reduced-price lunch (48 versus 44 percent). There is no

difference in the students’ racial and ethnic background (56 percent are white). ECHS students,

however, score far above the statewide average on both the 8th grade mathematics and reading exams

(0.29 and 0.34 standard deviations units higher, respectively) and would likely have done well in

conventional high schools. It is important to match these students to similar students at conventional

high schools to assess the impact of attending an ECHS on student pipeline progression.

(Insert Table 1 about here)

3 One school opened in Fall 2004 but did not start working with the New Schools Project until fall 2005. Seven schools existed prior to the initiative as Middle College High Schools which are quite different from ECHS. Students usually do not attend an MCHS for four years and are only eligible to earn some college credit, not the 2 years’ worth at the center of the ECHS model.




Data and Methodology

Longitudinal student-level data from North Carolina allow us to observe students’ pipeline

progression as they advance through high school. The data indicate the school and grade level in

which the student is enrolled as well as her mathematics and science course-taking and performance

on the statewide End-of-Course (EOC) exams. The data panel covers students enrolled in any public

school in the state at any time between 2004/05 and 2008/09, providing data on several 9th grade

cohorts in each Early College High School. We identify three cohorts of 9th graders (enrolling in

2004/05 through 2006/07) and follow them through the high school grades.4

We define two measures of pipeline progression—persistence and proficient performance—

that both impose a timeline for the completion of a sequence of college-preparatory courses. A

separate set of pipeline progression measures are defined for mathematics and science. Each pipeline

consists of three courses—algebra 1, geometry, and algebra 2 in mathematics, and biology and any

two of physical science, chemistry and physics in science. In order for students to persist in the

pipeline, they simply must take an additional pipeline course each year. In order for students to

progress through the pipeline with proficient performance, they must demonstrate mastery of the

course material by performing proficiently on the course’s EOC exam. The state defines proficiency

as scores at either Proficiency Level III or IV. Students must take the course in order to take the

exam. If they do not take the course, they do not demonstrate proficient performance by default.

Thus, proficiently performing students are a subset of persisting students.

The pipeline concept flows from the fact that high school mathematics and science curricula

are sequential; therefore, it is just as important when students take the courses as whether they take

4 Our statewide cohort identification strategy requires students to be observed in either the 8th or the 9th grade. We exclude several groups of students: (1) those never observed in high school, (2) those who were ever enrolled in special state schools (e.g., schools for the deaf) or in vocational, special education, or alternative high schools), and (3) those few students with highly irregular (and suspect) grade patterns (skipping or regressing grades).




them. For both measures, we judge a student’s pipeline progression at the end of each grade relative

to the timelines indicated in Figure 1. On-track pipeline progression in mathematics, for example,

requires a student take (persistence) and demonstrate mastery of the course material (proficient

performance) in at least one course by the end of 9th grade, in two by the end of 10th grade, and in all

three by the end of 11th grade. Once a student’s progression falls off-track, she remains off-track. The

motivation behind the pipeline concept is that schools should help students avoid discovering later in

high school that certain opportunities (e.g., courses with prerequisites, college admissions,

scholarships, etc.) are now closed to them because they did not take specific courses in earlier grades.5

In addition, research suggests that once a student falls off-track it is extremely hard for a student to

get back on-track. For example, a study examining high school transcripts in California found that, of

the students who did not take algebra I by the end of the 9th grade, less than 6 percent had completed

the courses necessary for college by the end of grade 12 (Finkelstein & Fong, 2008). The pipeline

timelines allow students to comply with the North Carolina Department of Public Instruction’s

(NCDPI) graduation requirements for the College Preparatory (CP) Course of Study, the eligibility

requirements for the North Carolina Academic Scholars Program, the NCDPI recommended science

sequence for CP students, and admissions guidance for the University of North Carolina system.6

(Insert Figure 1 about here.)

The pipeline progression timelines are compatible with the variety of course sequences North

Carolina high school students are encouraged to follow. In order to graduate from the CP course of

study, students must complete algebra 1, geometry, algebra 2 and another mathematics course with

5 Very few students are observed making up missed coursework after falling off-track to be deemed persisting in the pipeline and even fewer are observed proficiently performing on the state-wide exams. 6 The requirements of the College Preparatory course of study applied to the students in our analytic sample. This was replaced by the Future-Ready Core course of study beginning with the 2008/09 9th grade cohort.




algebra 2 as a prerequisite.7 Therefore, students must complete all three mathematics pipeline courses

by the end of 11th grade to avoid taking more than one mathematics class a year.8 With respect to

science, the pipeline timeline allows students to take the three science courses required for

graduation—biology, a physical science course, and earth/environmental science—plus a fourth

course that is increasingly becoming the norm for college admission.9 There is no EOC for the

earth/environmental course thus the timeline permits a student to take that course at any time during

high school and still be on-track at the end of each grade.10

North Carolina’s detailed longitudinal data on students permits a rigorous examination of the

Early College High Schools on student pipeline progression in mathematics and science. We are able

to follow students over time and observe their course-taking behavior in both mathematics and

science (for the persistence measure). And we are able to observe their mastery of the subject matter

taught in each of these classes (for the proficient performance measure). Peculiarities in the state’s

end-of-course testing system, however, place some limitations on our ability to measure the ECHS

effect on pipeline progression, especially in science. Because the state did not release scores for the

physical science, chemistry and physics EOC exam in 2006/07, we can only assess the effect on

science pipeline proficient performance at 10th grade. This does not impact the science persistence

effect estimate as we observe course-taking in other administrative data.

7 We modified our math pipeline indicators to allow for the test-taking pattern of students enrolled in the three-course Integrated Mathematics (IM) sequence. Students in the IM sequence take the algebra 1 EOC at the end of the second course, the geometry EOC in the middle of the third course, and the algebra 2 EOC at the end of the third course. This modification was applied only to students enrolled in schools where the IM sequence is offered – less than 3 percent of all student-year observations statewide. 8 This four-course requirement first applied to the 2002/03 9th grade cohort. Prior cohorts needed only three units (algebra 1, geometry, and algebra 2). 9 The physical science course requirement can be fulfilled by three state-tested courses—physical science, chemistry and physics—and one non-state-tested subject—Principals of Technology (a Career and Technical Education course). 10 State law only requires these students to take these courses, not to demonstrate subject matter mastery as is required by our definition of pipeline proficient performance. Beginning with 9th graders in 2006/07, North Carolina requires students to earn a passing score (i.e., Proficiency Level III or IV) on the algebra 1 and biology EOC. There are no performance requirements for the other pipeline courses.




Methodology

ECHS’s effect on student pipeline progression is the difference between the predicted pipeline

progression among ECHS students and that of similar matched students attending traditional high

schools within the same district. This effect may vary across grades. For example, ECHS and matched

students may be equally likely to demonstrate mastery in one college preparatory mathematics course

by the end of 9th grade while ECHS students are more likely to continue to persist in the 10th and 11th

grades. Survival analysis aligns nicely with our interest in a student’s probability of progressing on-

track through the pipeline at the end of each high school grade (Singer & Willett, 2003).

We estimate a two-level hazard model (students in cohorts-by-schools) separately for each of

the four pipeline measures—persistence (course-taking) and proficient performance (demonstrating

mastery) in mathematics and science (Figure 2). Each model predicts the probability that a student

who began the year on-track falls off-track by the end of the year. Our outcome measure is therefore

equal to 0 if the student ends the year on-track and equal to 1 if she ends the year off-track. Every

student is linked to an individual data record for each year of the pipeline until her on-track pipeline

progression halts (she falls off-track), meaning students are likely to have more than one and up to

three records in the dataset. Data pertaining to school years after a student falls off-track are censored

because she is no longer at any risk of falling off-track.

(Insert Figure 2 about here)

The probability student i in cohort j at school k will end grade g off-track given that he or she

started the grade on-track is a function at Level 1 of the baseline hazard, g , and a set of student

characteristics, c

iX . The baseline hazard is simply a series of grade-specific indicator variables which

in turn is modeled at Level 2 as a function of a reform school indicator, kR , equal to 1 if the school

has implemented the reform, 0 otherwise; a series of 9th grade cohort indicators, [ ]; and




cohort-by-school random effects. The estimated coefficients ( ) trace the student’s risk (or

hazard) of falling off-track at the end of each grade assuming he or she has remained on-track (or

survived) up to that grade. All student characteristics are measured in the 8th grade except for less than

ten percent of the sample that is not observed in the 8th grade. For these students, the characteristics

are measured when they first enroll in 9th grade. Student characteristics include gender, race/ethnicity

(white, black, Hispanic or other race), parental education (high school dropout, high school graduate,

some college or Bachelor’s degree or higher), an indicator of whether they entered the 9th grade over-

age (a proxy for being previously retained in grade), 8th grade End of Grade mathematics score

standardized (across full state sample) within testing year, and an indicator of whether or not the

student took Algebra I in the 8th grade.11

North Carolina’s ECHS initiative was motivated in large part by the desire to close persistent

achievement gaps between student subgroups (white and minority students, children of college and

high school graduates). The model specification above assumes that the ECHS effect is the same for

all students. We relax this proportional hazard assumption by interacting the student characteristics

with the baseline hazard at Level 1 and modeling these coefficients at Level 2 as a function of the

reform school indicator. This allows for the ECHS effect to vary across the grades within student

subgroups.

As already mentioned, the validity of these effect estimates depends a great deal on the quality

of matches of Early College and traditional high school students. Observed outcomes among the

traditional high school students are assumed to represent Early College students’ outcomes if they had

opted to attend a traditional high school. We therefore must choose these students carefully in order

11 We tested linear, quadratic and cubic relationships between 8th grade mathematics exam test scores and pipeline progression. Based on model fit, we specify a linear relationship with science pipeline progression and a quadratic relationship with mathematics pipeline progression.




to remove from our impact estimates the potential bias introduced by the fact Early Colleges are not

randomly assigned to districts, and students are not randomly assigned to an ECHS. We identify

matched students from the same district(s) to hold district level factors constant between the two

groups of students. Opening an ECHS is a district-level initiative. The district must agree to create a

new school, hire a new principal and new staff, and enter into an agreement with a local post-

secondary institution at which Early College students can take college-level coursework. Not all

districts will have the resources and/or desire to implement this reform, factors that are conceivably

related to student achievement.

We use propensity score matching to select comparison students from the same 9th grade

cohort for each ECHS student who enrolls in the 9th grade at an ECHS (Becker & Ichino, 2002;

Dehejia & Wahba, 2002; Rosenbaum & Rubin, 1983). The pool of potential matches included all

members of the same cohort enrolled in any school within the same district (including charter

schools). We identify ten nearest-neighbor matches for each ECHS student using a thorough set of

student characteristics that included two proxies for motivation: amount of time spent on homework

and 8th grade teacher’s judgment of student reading ability.12 Descriptive statistics of the student-level

samples are provided in Table 1. The resultant sample of ECHS and matched comparison students

mimics random assignment on the observables included in the selection model, and the effect

estimates are average treatment effects on the treated.

The statistics in Table 1 support our decision to pull same-district matches rather than

matching Early College students to observably similar students in districts without an ECHS option.

Under both scenarios, we can only test for balance on the observables. We know same-district

12 The ten nearest-neighbor matches used in these analyses were identified with replacement in 26 districts and without replacement in the remaining districts. The final matches were chosen without a caliper; however, over 85 percent of matches have propensity scores within 0.01 of their ECHS match. The specification of the selection model predicting the propensity scores used to identify the matches varied across the districts.




matches achieve balance on a large number of observable characteristics. Judgment as to the relative

strength of the two matching strategies therefore lies with assumptions about each strategy’s ability to

balance the unobservable characteristics predictive of both ECHS enrollment and academic

performance. Once matched on the observables, it is random which student is identified as a match.

Inter-district matching would be preferable if the selection process was such that students observed

selecting into an early college are completely dissimilar to all students not selecting into an early

college. We do not believe this to be the case. In fact, many of the Early Colleges are oversubscribed

(a fact Edmunds and her colleagues have exploited in their study of Early College’s impacts on

student academic performance). Given the balance we achieve with the same-district sample, a

random-draw would determine if an inter-district sample would permit a better balance on the

student-level unobservables. What is quite clear, though, is that an inter-district sample would not be

balanced on district-level unobservables, a weakness our same-district sample does not suffer.

The propensity score matching approach leverages detailed information on a large pool of

comparison students (including information on aspects related to their motivation to enroll in an

Early College school) to identify a comparison student sample; however, the resultant average

treatment on the treated effect estimates (ATT) may still be biased should we lack critical information

on the selection process. As a check on this potential bias, we also estimate average treatment effects

(ATE) via a difference-in-difference approach that compares pipeline progression rates between all

members of a 9th grade cohort with access to an Early College school and all members of a prior 9th

grade cohort which did not have access between districts with and without an Early College school

(Figure 3). The benefit of this approach is that our effect estimates will not be biased by non-random

student selection on unobservables into the district’s ECHS; however, the effect estimates may still be

biased for other reasons. For the difference-in-difference treatment effects to be unbiased, we must




assume that any state-, regional-, or national-level policy changes coinciding with the opening of the

Early College schools have the same effect on pipeline progression rates in the non-ECHS districts as

they do in the ECHS districts so that the difference in pipeline progression between the two cohorts

in districts without an ECHS is the same difference we would have observed in the ECHS districts if

they had not opened an ECHS. To support this identifying assumption, we select as non-ECHS

districts the 32 districts that open an ECHS at some point after 2006/07 and which do not allow

members of the Fall 2006 9th grade cohort to enroll. These districts are similar to the ECHS districts

in that they demonstrate the desire and ability to partner with a university and create a new school;

they just open their Early College school later. The timing of the school’s opening, however, is non-

random, and the difference-in-difference effects will be biased if there are factors connected to the

timing decision that are related to pipeline progression and which interact with these other policy

changes. Even if our ATT effect estimates are unbiased, we expect attenuation in the effects (ATE

relative to ATT) given the positive selection in Early College schools and Early College students being

a small proportion of a district’s total enrollment.

(Insert Figure 3 about here)

We estimate a separate difference-in-difference model predicting the probability a student is

on-track for each grade, subject, and pipeline progression measure. We compare the pipeline

progression of the Fall 2004 and Fall 2006 9th grade cohorts, meaning we are comparing the second

cohort with ECHS access to the last cohort without access in districts opening their ECHS in Fall

2005 and comparing the first cohort with access to the second-to-last cohort without access in the

districts opening their ECHS in Fall 2006.13 The sample includes 33 of the 39 ECHS. We exclude the

13 Given the same set of districts serve as comparisons for both sets of ECHS districts, selecting these two cohorts allows us to estimate one set of average treatment effects for the pooled sample. If we had selected the first cohort with access and the last cohort without access in each district, the pipeline progression of the Fall 2005 9th grade cohort in the non-




five ECHS districts in which upperclassmen were allowed to enroll in the ECHS, violating the

assumption that the Fall 2006 cohort was exogenously assigned to the treatment condition and the

Fall 2004 cohort to the control condition, as well as one district where nearly all non-ECHS students

attend 10-11th grades in a different school than where they attended the 9th grade. Only a very small

proportion, roughly 4 percent, of the 2006/07 9th grade cohorts in the ECHS districts choose to enroll

in an Early College and receive the treatment.

For further assurance of comparability between the ECHS and matched comparison student

samples, we assign each student a propensity score weight to account for non-random censoring and

student mobility out of their initial 9th grade school (Brunell & DiNardo, 2004; DiNardo, 2002;

DiNardo, Fortin, & Lemieux, 1996). Each weight is equal to the inverse of the student’s propensity to

be censored (i.e., lost to follow-up) multiplied by the inverse of the student’s propensity to separate

from her initial 9th grade school.14

Finally, our interest is in the effect of attending an ECHS on a student’s probability of being

on-track at the end of each high school grade. Using the estimated coefficients from the hazard

models, we convert the predicted log odds ratios to predicted hazard probabilities and then to survival

probabilities to calculate the treatment effect in terms of percentage points change in the probability

of being on-track (i.e., predicted on-track probability if attending an ECHS minus predicted on-track

probability if not attending an ECHS).We set all student characteristics equal to the analytic sample

mean to predict these probabilities. Standard errors around these treatment effects are calculated by

applying the delta method to the treatment effect using the estimated variance-covariance matrices (Ai

& Norton, 2003). Because the maximum treatment effect in percentage points is determined by the

ECHS districts would represent both post-treatment outcomes for the earlier ECHS-implementing districts and pre-treatment outcomes for the later ECHS-implementing districts. 14 We use the full state population (not just our analytic sample) to estimate these propensities.




predicted probability in the absence of reform (e.g., 20 percentage points max if 80 percent probability

of being on-track in absence of reform, 40 percentage points max if 60 percent probability, etc.), we

also present the treatment effect as the percentage change in the risk of being off-track to facilitate

comparisons across effect estimates.

Results We present three sets of results. In order to situate the ECHS students within the broader

state context, we first compare observed pipeline progression rates among ECHS students to all

students statewide. These simple difference-in-means results show ECHS students are more likely to

progress successfully through the pipelines, especially in mathematics, than students statewide on both

pipeline progression measures, especially with respect to persistence (course-taking). Next, we

compare ECHS students’ observed pipeline progression rates to those of the matched non-ECHS

students who were carefully selected using propensity score matching to represent the pipeline

progression we would expect of the ECHS students had they not attended an ECHS. These simple

comparisons reveal higher rates of mathematics on-track progression and generally lower rates of

science on-track progression among ECHS students, suggesting the model has a positive impact in

mathematics and a negative impact in science. Finally, we present the estimated treatment effects

generated by the two-level hazard model of pipeline progression that accounts for student

characteristics, cohort effects, and school random effects. These multivariate results confirm the

simple means comparisons between ECHS versus matched non-ECHS students.

Comparing ECHS Students to All Students Statewide

Elsewhere we detail statewide dynamics in pipeline progression rates between 1998 and 2008

(Miller & Milton, 2011). Over this period, on-track progression rates—both persistence (course-

taking) and proficient performance (subject matter mastery)—increased in mathematics but decreased




in science. The decrease in science progression rates coincides with the 2003 change in state

graduation requirements that deemphasized chemistry and physics as standalone courses. The trends

also show positive correlations of pipeline progression and both parental education and 8th grade

mathematics achievement test scores as well as higher on-track progression rates among white than

non-white students.

Table 2 details the observed progression rates for three groups of students— all students in

the state, ECHS students, and matched non-ECHS students. We provide rates for two cohorts of

students—9th graders in 2006 and 2007—in order to highlight sources of variation which our hazard

model takes into account when predicting pipeline progression rates. The 2006 cohort is the only 9th

grade cohort we are able to follow through the 12th grade; yet we only observe this cohort in 13 of the

33 Early College schools in our sample. The rates for the 2007 9th grade cohort on the other hand

include all 33 schools, yet can only be calculated through the 11th grade. Comparisons across the two

cohorts highlight both cohort effects as well as the introduction of new mathematics exams which

decreased proficiency rates in 2007.


ECHS students are more likely to persist and perform proficiently in the college preparatory

mathematics courses than students statewide, which is not surprising given their stronger performance

on the 8th grade mathematics exam (more than a quarter standard deviations higher). Over 90 percent

of ECHS students have completed all three courses by the end of the 11th grade (and thus have the

necessary prerequisites to take an advanced math course in the 12th grade) compared to less than 60

percent statewide. Half of ECHS students master all three math courses by the end of the 11th grade

versus about 36 percent of students statewide.




The pipeline concept is motivated by the belief that students need to be taking college

preparatory courses in the 9th grade and continue to take them each year until graduation. ECHS

students and students statewide are at greatest risk of falling off-track at different stages of the

mathematics pipeline (i.e., they begin the year on-track but fall off-track by year’s end). Students

statewide are at greatest risk of falling off-track in 9th grade in both persistence (course-taking) and

proficient performance (subject matter mastery) when students are likely taking algebra I.15 In fact,

almost 30 percent of students statewide do not take a pipeline course and roughly 45 percent do not

demonstrate mastery in one subject by the end of the 9th grade. ECHS students, on the other hand,

are at greatest risk of falling off-track in the 11th grade when students are likely taking algebra II. The

risk an ECHS student falls off-track with respect to persistence in the 11th grade is 4 percent and close

to 25 percent with respect to proficient performance.

Although ECHS students successfully progress through the mathematics pipeline at higher

rates than students statewide, they fall behind in science after the 10th grade. ECHS students are more

likely to have taken at least one college preparatory science course by the 10th grade than students

statewide (11 points in 2006, 8 points in 2007); however, their on-track persistence rates decline

sharply thereafter. The risk the average ECHS student will not take a second course by the 11th grade

and thus fall off-track is more than 2 times greater for the 2006 cohort and more than 3 times greater

for the 2007 cohort than the risk the average student statewide will fall off-track. By the end of the

12th grade, 23 percent of students statewide have taken three science pipeline courses compared to 19

percent of ECHS students. Because of the lack of science test scores other than biology in 2007, we

cannot observe if the same comparative reversal occurs in the proficient performance progression

15 The pipeline measures require at least one course be taken/mastered by the end of the 9th grade. Therefore, for a student who took algebra I in the 8th grade and another pipeline course in the 9th grade to be off-track with respect to proficient performance in the 9th grade, they must have scored below proficiency on both exams.




rates. However, 10-13 percentage points more ECHS students than students statewide demonstrate

mastery in one course by the 10th grade.

Comparing ECHS Students to Matched Comparison Students

Successful mathematics pipeline progression rates for both persistence and proficient

performance among ECHS students exceed those of their non-ECHS matched peers, even though

the matched students also have 8th grade mathematics achievement test scores a quarter of a standard

deviation on average above the statewide average. Whereas over 90 percent of ECHS students take all

three mathematics courses by the end of the 11th grade, only about 70 percent of the matched

students do the same. Although ECHS proficient performance rates also exceed those of non-ECHS

matched students, the ECHS students’ advantage decreases as they advance through high school.

Among the 2006 9th grade cohort, ECHS students were 8.3 percentage points more likely to be on-

track in 9th grade yet only 4.8 percentage points more likely in the 11th grade. The gap dwindles to 0.2

percentage points by the 11th grade for the 2007 9th grade cohort.

ECHS students are less likely than their non-ECHS matched peers to successfully progress

on-track through the science pipeline. The exception is 10th grade persistence. Consider the 2006 9th

grade cohort. Although the ECHS students are 4.3 percentage points more likely than matched non-

ECHS students to have taken one science course by the end of the 10th grade, they are 16.1

percentage points less likely to have taken two courses by the 11th grade and 10.8 points less likely to

have taken three by the end of the 12th grade. These ECHS students are also less likely than matched

non-ECHS students to have demonstrated mastery in one course by the 10th grade (61.7 versus 72.4

percent).




Estimating the Effect of ECHS on Pipeline Progression

Results of the two-level hazard model of pipeline progression largely confirm the simple

comparisons above. ECHS has positive effects on mathematics pipeline progression at each grade

level for persistence (course-taking) and at 9th and 10th grade for proficient performance (subject

matter mastery) and a negative effect on science pipeline persistence at the 12th grade and for

proficient performance at the 10th grade. Furthermore, ECHS has differential impacts on the

successful pipeline progression of various student subgroups defined by race/ethnicity, parental

education and prior mathematics achievement. ECHS generally decreases group differences in

mathematics (both pipeline progression measures) and science persistence but widens them in science

proficient performance. We present the results separately by subject.


While ECHS improves mathematics course-taking at each point in the pipeline, its positive

effect on performance disappears in the 11th grade. The average Early College student is 11.0

percentage points more likely (or 12.6 percent more likely, p<.001) to have taken all three college-

preparatory mathematics classes by the end of the 11th grade than a similar student attending a non-

ECHS (97.2 versus 86.2 percent) as shown in Table 3. This average Early College student is also 9.9

percentage points more likely (or 13.7 percent more likely) to have demonstrated mastery in at least

two courses by the end of the 10th grade (82.2 versus 72.3 percent, p<.001). Although the treatment

effect on proficient performance is positive at the 11th grade (4.3 percentage points), it is not

statistically significant.

The treatment effects increase with grade level and are larger for pipeline performance than

persistence (11th grade performance excepted). Early Colleges, however, are far more successful at

ensuring students take college-preparatory mathematics courses than they are at helping them master




the content. If we judge Early Colleges’ effects by how many students are kept on-track that are

predicted to be off-track in the absence of ECHS (in other words, reducing the risk of being off-

track), early colleges have the greatest impact at the 9th grade and on pipeline persistence. For

example, the smaller 5.1 percentage point treatment effect on 9th grade performance converts to a 54.4

percent reduction in the risk the average student fall out of the pipeline whereas the larger 9.9

percentage point treatment effect at the 10th grade converts to a smaller 35.8 percent risk reduction.

Similarly, the risk reductions are larger for persistence than proficient performance (e.g., 86.1 versus

35.8 percent at the 10th grade) while the treatment effects are smaller (e.g., 7.8 versus 9.9 percentage

points).

At least with respect to mathematics, North Carolina’s Early College schools are meeting their

directive to serve students from groups underrepresented in college—first-generation college-goers,

minorities, and students struggling in conventional high schools. Results from models testing for

heterogeneous effects confirm that the pipeline progression of almost all subgroups defined by

race/ethnicity, parental education, and 8th grade mathematics achievement benefits from the ECHS

model. However, effects vary across the subgroups changing their relative progression.


Group differences in persistence and proficient performance through the mathematics

pipeline are narrowed by ECHS with the exception of the differences between white and black and

Hispanic students. The positive treatment effect and the reduction in the risk of being off-track are

smaller for black and Hispanic students than for whites (see Table 4). With respect to mathematics

pipeline persistence at the 11th grade, the black-white difference increased by 3.5 percentage points

and the Hispanic-white difference increases by 1.7 percentage points at the 11th grade. With respect to

mathematics pipeline proficient performance at the 10th grade, the black-white difference grew by 3.5




percentage points and the Hispanic-white difference grew by 3.6 percentage points. ECHS has no

impact on the white-versus-non-white proficient performance differences at the 11th grade.

As a result of attending an ECHS, students who stand to become first-generation college-

goers successfully progress through the mathematics pipeline at rates much closer to their peers

whose parents graduated from college (see Table 5). Consider the students with parents who earned a

BA and those whose parents did not complete high school. The difference in persistence rates

between them narrowed by 2.6 percentage points in the 9th grade and by 8.5 percentage points in the

11th grade. The extent of narrowing of the difference was very similar with respect to proficient

performance.


Finally, ECHS narrows differences in mathematics persistence and proficient performance

across the distribution of 8th grade mathematics achievement test scores (see Table 6). The effect on

the top performers is nil to negligible. Almost all these students are, in the absence of ECHS,

predicted to be on-track. However, the size of the treatment effects grows as one moves down the

score distribution causing the differences in the persistence and proficient performance rates to

decrease. For example, ECHS increases the likelihood a student scoring at the 75th percentile will have

demonstrated mastery in all three courses by the end of the 11th grade by 3.4 percentage points and

increases the likelihood by 17.7 percentage points for a student at the 25th percentile. These results are

based on a model specifying a quadratic relationship between 8th grade test scores and on-track

pipeline progression.


The consistent positive effects on mathematics pipeline progress do not carry over to the

science pipeline. Early Colleges have no effect on science course-taking at the 10th and 11th grades yet




significantly decrease the likelihood the average student will have taken three college-preparatory

science classes by the end of 12th grade by 16.6 percentage points (p<.01), which is associated with an

increase in the risk of being off-track of 21.6 percent (see Table 3). Early Colleges have no effect on

the average student’s probability of demonstrating mastery in at least one course by the end of 10th

grade.

Early Colleges’ negative impact on 12th grade science persistence generally decreased student

subgroup differences as almost all students took fewer science pipeline courses for high school credit.

For example, compared to students whose parents earned a bachelor’s degree, the difference with

students whose parents graduated high school shrank by 3.6 percentage points (see Table 5). Gaps in

on-track persistence rates between the top 8th grade mathematics scorers and other students also

narrowed with size of negative impacts being largest for top scorers and insignificant for the bottom

scorers (see Table 6). An exception is the black-white difference which increased by 5.2 percentage

points (see Table 4).

Robustness Checks

We explored the robustness of our main results in several ways.

First, we tested the assumption the treatment effects did not differ between the 13 Early

Colleges that opened in Fall 2005 and the 20 schools that opened a year later. There are several

reasons why the effects may differ between the two groups. The first group may have been able to

open earlier because of pre-existing connections with the partner institute of higher education, which

may have helped the districts implement the ECHS model. Or perhaps the second group took the

extra year to better plan how the partnership would work, which may have facilitated implementation.

Another explanation could be that the extra year of operation allowed the first group of schools to

iron out kinks that constrained their students’ successful pipeline progression. We therefore estimated




the models separately by implementation cohorts. The results support the equal effects assumption

with the effects being quite similar between the two school groups.

Second, we test our assumption that Early Colleges recommend a four-course science

curriculum for students. Our science pipeline expects students to complete the three state-required

courses plus a fourth course. The descriptive pipeline statistics presented earlier indicate Early College

students are less likely than non-ECHS matched students (and students statewide) to go beyond the

minimum state requirements and take a fourth science course for high school credit. It may be,

however, that Early College students are more likely to meet the state graduation requirements by the

12th grade than non-ECHS matched students. (Note, we do not expect all students in our sample to

meet this goal as students who drop out of high school are considered to be off-track and students

only need to demonstrate mastery of biology to graduate.) We examine this possibility by shifting our

science pipeline out one year to require one course be taken or mastered by 11th grade and two

courses be taken or mastered by 12th grade.

Early College students perform slightly better under this more lenient definition of pipeline

progression, although the difference in on-track rates is quite small (see Table 7). Persistence rates

among ECHS students are predicted to be 1.7 percentage points higher (p<.05) at the 11th grade than

non-ECHS matched students (99.2 versus 97.5 percent) with no significant difference at the 12th grade

(87.8 versus 87.5 percent). ECHS also had no impact on proficient performance at the 11th grade (91.9

versus 92.9 percent). In other words, while Early College students are substantially less likely to go

beyond the minimum graduation requirements and take a fourth course for high school credit, they

are equally likely as similar non-ECHS students to satisfy the minimum science requirements.

Another possible driver of the science results is that Early College students are taking other

types of high school science courses than those included in our pipeline. Exploratory analyses of




course membership files for 2007 and 2008 rule this out, however. A greater percentage of Early

College students are not taking any high school science course than among non-ECHS matched

students: 14 versus 7 percent in 9th grade, 6 versus 5 percent in 10th grade, 34 versus 8 percent in 11th

grade, and 65 versus 45 percent in the 12th grade. These numbers may reflect Early College students

taking science courses through the schools’ partner colleges and universities that are not listed in the

course membership files. While we do observe students in courses described as “Community College

Science,” we do not observe the courses students are taking strictly for college (and not high school)

credit. Therefore, our science results should be interpreted as Early College’s impacts on high school

science courses only.

Finally, we estimate average treatment effects via a difference-in-difference approach to assess

the extent to which our main results may be driven by selection bias (Table 8). The pattern of the

results mimics those based on the matched student sample. The effects are statistically insignificant,

however, owing in part to the expected attenuation of the point estimates but also substantial losses in

precision. Therefore, we cannot rule out the possibility selection is biasing our main results; however,

nor do the ATE results prove our main results are all selection bias. Only 4 percent of our treatment

group actually attended an Early College school. The ECHS effect could be too small to detect when

diffused over a group almost 24 times larger.


Discussion and Conclusion

Early College High Schools are small, innovative public schools that offer students the

opportunity to earn both a high school diploma and two years of college credit in four or five years

free-of-charge. Hoping to improve college preparedness and ensure successful transition to college,

especially among students from populations underrepresented in post-secondary institutions, the




model has been replicated more than 230 times in 28 states and the District of Columbia. We study 33

Early Colleges in North Carolina to measure their effect on student progression through the

mathematics and science pipelines, each of which consist of three college-preparatory courses. The

pipeline concept stresses the importance of when a student takes the courses, not just if she does.

Our results indicate large, positive effects of attending an Early College on students’ successful

progression through the mathematics pipeline. Early College students, regardless of race and ethnicity,

parental educational attainment, and 8th grade mathematics achievement, are more likely than similar

students attending other high schools in the same district to take three college preparatory

mathematics classes by the end of the 11th grade and demonstrate mastery in two of them by the end

of the 10th grade. However, attending an Early College has a nil to negative effect on science pipeline

progression. As with all propensity score-based treatment effects, there is the possibility our results

are driven by selection on the unobservables. We estimate average treatment effects via a difference-

in-difference model as a diagnostic test. These results do not allow us to rule out bias from selection

on unobservables, but our diagnostic test is not without its own imperfections. The pattern of the

average treatment effects combined with the previously mentioned positive experimental results

reported by others analyzing a subset of the Early College schools in our sample provide suggestive

evidence our results are not all selection. Four key questions guide our discussion of these results and

what actions they suggest for policy. We consider each in turn below.

First, what explains the difference in effects between mathematics and science? Financial

constraints on schools’ staffing decisions coupled with the federal requirement that all teachers be

“highly qualified” may be one reason. These Early College schools are small by design and even

smaller during our study’s observation period as they ramp up to full enrollment. Early Colleges offer

fewer sections of any one course than are offered at a large comprehensive high school, meaning full-




time ECHS teachers are likely expected to teach multiple courses rather than multiple sections of a

single course. North Carolina’s teacher certification system makes it easier to hire a single “highly

qualified” teacher for all three mathematics pipeline courses than for all the science pipeline courses.

There is one certification area for all secondary mathematics teachers, and holding that certification

makes teachers eligible to be highly qualified in all three of the mathematics pipeline courses. In

contrast, science teachers certified in a specific course (earth science, biology, chemistry, or physics)

are not automatically eligible for “highly qualified” status in the other courses. This may lead to a

shortage of science teachers and, in turn, fewer science course offerings in Early College schools.

The fact Early College schools enroll students in grades 9 through 13 may also explain the

lower science course-taking rates through the 12th grade among Early College than matched non-

ECHS students. Early College schools are public schools and thus dependent on state funding. State

funding is based on student enrollment, and Early College schools can only include 13th graders in

their enrollment figures if these students are taking courses toward their high school diploma.

Therefore, if Early College students wait until their 5th year of high school to complete their high

school science coursework, we will not observe the full effect on science course-taking rates at the

end of the 12th grade. Science courses do not build upon knowledge from prior classes to the same

extent as mathematics courses, which may make it easier for students to take one or more science

classes in the 13th grade.

Second, what about the manner in which Early College schools operate is driving the positive

effects in mathematics? As mentioned earlier, evidence from the national evaluation suggests the

effect of ECHS on standardized test scores is greater when they are located on college campuses,

which many Early College schools in North Carolina are. And while these campuses are certainly




different from the comprehensive schools the non-ECHS matched students attend, it is not entirely

clear how this proximity would be beneficial for mathematics and not science.

Another feature of the Early College schools that sets them apart from comprehensive high

schools is the policies and practices implemented at the school to support student performance. In a

related study, we surveyed school principals to collect data on policies and practices in five core

areas—course-taking requirements, rigorous instruction, academic support, personalization, and

relevance—which previous research has shown are positively correlated with student outcomes

(Arshavsky, Edmunds, Miller, & Corritore, 2012).The results reveal Early College schools have

implemented policies and practices in each area to a higher degree than a sample of comprehensive

high schools. Additionally, the results show that when jointly implemented these policies and practices

predict successful mathematics pipeline progression.

Third, our study focuses on the effect of Early Colleges on high school performance, yet this

is only half of the intended benefits of the ECHS model. What is the effect on students’ post-

secondary experiences? How many college courses are they taking, and how are they performing in

them while enrolled in an ECHS? Are Early College students more likely to complete a 4-year degree

than their matched non-ECHS peers? Data available for the current analysis preclude an analysis of

post-secondary outcomes as the first cohort of students to attend ECHS for 5 years graduated in

2011.

Fourth, given the average student selecting into North Carolina’s Early Colleges is better

prepared academically than the average student statewide, can the positive effect identified here be

replicated with students not as well prepared? All the schools included in this analysis are schools of

choice. For example, consider a non-choice school (e.g., a neighborhood school serving a catchment

area) that adopts the ECHS model for all its students similar to the way it might adopt a




comprehensive school reform model (e.g., America’s Choice, High Schools That Work, Talent

Development, etc.). Will ECHS improve the average student’s pipeline progression in this school with

more low-performing students? Will ECHS narrow achievement differences between student

subgroups? It is vital we develop an understanding of Early College’s effects in other school contexts

in order to better inform decisions concerning how widely to scale up the ECHS model.

This brings us to our final question. What is the optimal amount of model replication? The

current study demonstrates the model’s success in preparing students for college by changing the high

school experience. Yet, in changing the high school experience, the model also fundamentally changes

the college experience for these students. Assuming positive benefits on post-secondary educational

attainment, students enter and exit college at an earlier age and split their college education between

two campuses and two sets of peers. These changes may limit the benefits of a more traditional path

to a degree such as access to and acceptance in social and professional networks. Early College’s

accelerated timeline forces tradeoffs that may not be optimal for all students. If a student’s choice is

between getting a jumpstart on her college degree by attending an ECHS and not going to college at

all, the accelerated program is preferable. However, the same might not be true if the student’s choice

is between Early College’s accelerated path and the traditional path (four years of high school and

four years of college).




References

Adelman, C. (2006). The toolbox revisited: Paths to degree completion from high school through college. Washington, DC: U.S. Department of Education.

Ai, C., & Norton, E. C. (2003). Interaction terms in logit and probit models. Economics Letters, 80, 123-129.

American Institutes for Research & SRI International. (2009). Six years and counting: The ECHSI Matures. Seattle: The Bill & Melinda Gates Foundation.

Arshavsky, N., Edmunds, J., Miller, L. C., & Corritore, M. (2012). Success in college preparatory mathematics pipeline: The role of policies and practices employed by three high school reform models. Greensboro, NC: SERVE Center.

Bailey, T., & Karp, M. M. (2003). Promoting college access and success: A review of credit-based transition programs. Washington, DC: Office of Vocational Education, U.S. Department of Education.

Becker, S. O., & Ichino, A. (2002). Estimation of average treatment effects based on propensity scores. The Stata Journal, 2(4), 358-377.

Berger, A., Adelman, N., & Cole, S. (2010). The Early College High School Initiative: An overview of five evaluation years. Peabody Journal of Education, 85(3), 333-347. doi:10.1080/0161956X.2010.491697

Brunell, T. L., & DiNardo, J. (2004). A propensity score reweighting approach to estimating the partisan effects of full turnout in American presidential elections. Political Analysis, 12(1), 28-45.

Bureau of Labor Statistics. (2012, March 23). Education pays. Retrieved November 19, 2012, from Bureau of Labor Statistics: http://www.bls.gov/emp/ep_chart_001.htm

Carnevale, A. P., Smith, N., & Strohl, J. (2010). Help wanted: Projections of jobs and education requirements through 2018. Washington, DC: Georgetown University Center on Education and the Workforce.

Conger, D., Long, M. C., & Iatarola, P. (2009). Explaining race, povery, and gender disparities in advanced course-taking. Journal of Policy Analysis and Management, 28(4), 555-576. doi:10.1002/pam.20455

Dehejia, J., & Wahba, S. (2002). Propensity score-matching for nonexperimental causal studies. Review of Economics and Statistics, 84(1), 151-161.

DiNardo, J. (2002). Propensity score reweighting and changes in the wage distribution. Latin American Meeting of the Econometrica Society. Sao Paulo, Brazil.




DiNardo, J., Fortin, N. M., & Lemieux, T. (1996). Labor market institutions and the distribution of wages, 1973-1992: A semiparametric approach. Econometrica, 64(5), 1001-1044.

ECHS Initiative. (2010). Early College High School: A portrait in numbers. Washington, DC: Jobs for the Future.

Edmunds, J., Bernstein, L., Unlu, F., Glennie, E., Willse, J., Smith, A., & Arshavsky, N. (2012). Expanding the start of the college pipeline: Nineth grade findings from an experimental study of the impact of the Early College High Schools model. Journal of Research on Educational Effectiveness, 5(2), 136-159. doi:10.1080/19345747.2012.656182

Eimers, M. T., & Mullen, R. (2003). Dual credit and advanced placement: Do they help prepare students for success in college? 43rd Annual Association for Institutional Research Fall Conference. Tampa, FL.

Finkelstein, N. D., & Fong, A. B. (2008). Course-taking patterns and preparation for postsecondary education in California's public university systems among minority youth. Washington, DC: U.S. Department of Education, Institute of Educational Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory West.

Hoffman, N. (2005). Add and substract: Dual enrollment as a state strategy to increase postsecondary success for underrepresented students. Washington, DC: Jobs for the Future.

Hoffman, N., Vargas, J., & Santos, J. (2008). On ramp to college: A state policymaker's guide to dual enrollment. Washington, DC: Jobs for the Future.

Hoffman, N., Vargas, J., & Santos, J. (2009). New directions for dual enrollment: Creating stronger pathways from high school through college. New Directions for Community Colleges, 145, 43-58. doi:10.1002/cc.354

Iatarola, P., Conger, D., & Long, M. C. (2011). Determinants of high schools' advanced course offerings. Educational Evaluation and Policy Analysis, 33(3), 340-359. doi:10.3102/0162373711398124

Karp, M. M., Calcagno, J. C., Hughes, K. L., Jeong, D. W., & Bailey, T. (2007). The postsecondary achievement of participants in dual enrollment: An analysis of student outcomes in two states. St. Paul, MN: National Research Center for Career and Technical Education.

Kentucky Council on Postsecondary Education. (2006). The dual enrollment of high school students in postsecondary education in Kentucky, 2001-02 to 2004-05. Frankfort, KY: Kentucky Council on Postsecondary Education.

Kisker, C. B. (2006). Integrating high school and the community college: Previous efforts and current possibilities. Community College Review, 34(1), 68-86. doi:10.1177/0091552106289821

Klopfenstein, K. (2004). Advanced Placement: Do minorities have equal opportunity? Economics of Education Review, 23, 115-131. doi:10.1016/S0272-7757(03)00076-1




Learner, J. B., & Brand, B. (2006). The college ladder: Linking secondary and postsecondary education for success for all students. Washington, DC: American Youth Policy Forum.

Long, M. C., Conger, D., & Iatarola, P. (2012). Effects of high school course-taking on secondary and postsecondary success. American Educational Research Journal, 49(2), 285-322. doi:10.3102/0002831211431952

Long, M. C., Iatarola, P., & Conger, D. (2009). Explaining gaps in readiness for college-level math: The role of high school courses. Education Finance and Policy, 4(1), 1-33. doi:10.1162/edfp.2009.4.1.1

Michalowski, S. (2007). Positive effects associated with College Now participation. New York: The City University of New York, Collaborative Programs Research & Evaluation, Office of Academic Affairs.

Miller, L. C., & Milton, R. (2011). Who graduates high school college and career ready? An examination of mathematics and science pipeline progression in North Carolina. Charlottesville, VA: University of Virginia.

Partnership for 21st Century Skills. (2008). 21st Century skills, education & competitiveness. Tucson, AZ: Partnership for 21st Century Skills.

Robins, J. M., Hernan, M. A., & Brumback, B. (2000). Marginal structural models and causal inference in epidemiology. Epidemiology, 11(5), 550-560.

Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41-55.

Sherman, A., & Stone, C. (2010). Income gaps between very rich and everyone else more than tripled in the last three decades. Washington, DC: Center on Budget and Policy Priorities.

Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurance. New York: Oxford University Press.

State Board of Education and North Carolina Department of Public Instruction. (2011). Report to the North Carolina General Assembly: Early College High School (Learn and Earn) (Report No. 40). Raleigh, NC: North Carolina General Assembly.

Swanson, J. L. (2008). An analysis of the impact of high school dual enrollment course participation on post-secondary academic success, persistence and degree completion. Chapel Hill, NC: National Alliance of Concurrent Enrollment Partnerships.

U.S. Census Bureau. (2012, May 23). Table 3. Detailed Years of School Completed by People 25 Years and Over by Sex, Age Groups, Race and Hispanic Origin, 2011. Retrieved November 19, 2012, from U.S. Census Bureau: http://www.census.gov/hhes/socdemo/education/data/cps/2011/Table3.xls




Waits, T., Setzer, J. C., & Lewis, L. (2005). Dual credit and exam-based courses in U.S. public high schools: 2002-03. Washington, DC: National Center for Education Statistics.

Western Interstate Commission for Higher Education. (2006). Moving the needle on access and success. Boulder, CO: Western Interstate Commission for Higher Education.




By the end of… Student has taken courses in and/or demonstrated mastery in…

Mathematics Science

9th grade At least one subject 10th grade At least two subjects At least one subject 11th grade All three subjects At least two subjects 12th grade All three subjects

Figure 1. Timeline required for successful pipeline progression

Level 1 Model (student)

( )

[ ( )] ( ) ∑

, where

( ) [ ] [ ] [ ] for mathematics; or

( ) [ ] [ ] [ ] for science

Level 2 Model (cohort-by-school)

∑ [ ] , for g = 1, 2, 3

for all c

Figure 2. Two-level hazard model of pipeline progression

Level 1 Model (student)

( | )

(

( )) ∑

Level 2 Model (district-by-cohort)

, for some c

, for all other c

Figure 3. Two-level difference-in-difference model of pipeline progression




Table 1 Average student characteristics of 9th grade cohorts by school subsamples, 2004/05 to 2006/07 All students

in state (%) Early College High

School Students (%) Matched Comparison

Students (%)

N 474,965 4,402 44,020

Observed in 8th grade 89.6 95.1 95.1

Female 48.6 59.3 59.0

Over-age 21.2 13.5 13.6

Limited English Proficient 4.0 3.2 2.9

Free/Reduced-price Lunch Eligible 43.5 47.9 47.3

Race/Ethnicity

White 56.1 56.2 56.5

Black 29.7 26.5 26.4

Hispanic 7.1 8.0 7.7

Other Race 7.1 9.4 9.4

Parental Educational Attainment

No High School 8.8 8.7 8.6

High School 42.5 47.2 46.9

Some College 14.4 18.2 18.1

Bachelor’s Degree or More 26.6 20.8 20.7

{missing} 7.8 5.1 5.7

Time Spent Weekly on Homework in 8th Grade (0-5 scale) a

2.4 2.5 2.5

{missing} 14.4 6.5 7.1

8th Grade Teacher’s Judgment of Student’s Reading Ability

Does not have sufficient mastery 2.3 0.6 0.6

Demonstrates inconsistent mastery 12.3 5.1 5.1

Consistently demonstrates mastery 39.5 40.0 40.1

Performs in a superior manner 31.6 47.9 47.0

{missing} 14.2 6.5 7.2

Took Algebra I in 8th grade 22.3 23.7 23.7

8th Grade Math Score (z-score) 0.023

(0.989)

0.289 (0.815)

0.293 (0.845)

8th Grade Reading Score (z-score) 0.020

(0.986)

0.340 (0.782)

0.350 (0.802)

{missing 8th grade test scores} 14.7 6.2 7.1*

Note: Standard deviations in parentheses. a Response scale: 0 = no time, 1 = less than one hour, 2 = between one and three hours, 3 = more than three but less than five hours, 4 = between five and ten hours, 5 = more than ten hours * Significantly different from treatment group at 5%




Table 2 Pipeline progression rates for all students statewide, ECHS students, and matched comparison students by 9th grade cohort and grade, 2005/06-2008/09

2005/06 9th Grade Cohort 2006/07 9th Grade Cohort

All Students Statewide

(%)

ECHS

Students (%)

Matched Students (%)

All Students Statewide

(%)

ECHS Students (%)

Matched Students (%)

Mathematics

Persistence

9th Grade 72.8 97.1 86.1 70.4 97.8 83.0

10th Grade 62.3 92.6 76.4 61.3 94.8 73.3

11th Grade 55.9 92.9 71.4 58.5 91.2 70.5

Proficient Performance

9th Grade 63.2 87.3 79.0 50.7 77.8 67.5

10th Grade 44.2 66.7 56.4 43.3 65.3 56.7

11th Grade 35.1 50.5 45.7 37.7 50.1 49.9

Science

Persistence

10th Grade 83.7 94.5 90.2 82.1 90.0 90.6

11th Grade 72.8 68.5 84.6 75.6 66.0 86.4

12th Grade 22.8 19.4 30.2 N/A N/A N/A


10th Grade 51.6 61.7 72.4 58.5 71.5 72.4

Notes: All percentages calculated on the weighted sample. This explains the slight up-tick in the on-track percentage for mathematics persistence between 10th and 11th grades for the 2005/06 9th grade cohort enrolled in Early College schools.

Table 3 ECHS’s effects on the probability of on-track pipeline progression

Mathematics Science

Persistence Prof. Performance Persistence Prof. Performance

N 97,454 89,232 61,936 39,061

9th Grade 2.5 (0.4)***

-88.6%

5.1 (0.7)***

-54.4%

N/A N/A

10th Grade 7.8 (0.9)***

-86.1%

9.9 (1.9)***

-35.8%

1.4 (0.8) †

-34.3%

0.4 (3.0)

1.9%

11th Grade 11.0 (1.1)***

-79.8%

4.3 (3.6)

-10.1%

-12.7 (10.3)

145.9% N/A

12th Grade N/A N/A -16.6 (6.1)**

21.6% N/A

Notes: Separate models estimated by subject and pipeline progression measure. The first number in the first row of each cell is the treatment effect in percentage points; second number is the effect’s robust standard error. The number in the second row is the percent change (reduction if negative or increase if positive) in the students risk of being off-track in the absence of reform. Model fit statistics presented in Table A1 in the appendix. † p<.10, * p<.05, ** p<.01, *** p<.001




Table 4 ECHS’s effects on the probability of on-track pipeline progression by student race/ethnicity

Persistence Proficient Performance

Mathematics 9th Grade 10th Grade 11th Grade 9th Grade 10th Grade 11th Grade

N 97,454 89,232

White 2.2 (0.3)***

-81.9%

7.4 (1.2)***

-78.7%

10.6 (1.5)***

-70.7%

5.1 (0.8)***

-55.6%

10.7 (2.3)***

-38.5%

6.1 (4.0)

-13.7%

Black 2.0 (0.3)***

-76.5%

5.6 (1.0)***

-73.0%

7.1 (1.4)***

-65.1%

4.5 (1.2)***

-43.7%

7.2 (3.1)*

-25.2%

0.4 (4.5)

-0.9%

Hispanic 2.0 (0.3)***

-79.4%

6.4 (1.3)***

-75.9%

8.9 (2.3)***

-67.4%

4.0 (1.3)**

-47.0%

7.1 (3.8) †

-29.0%

1.4 (6.4)

3.6%

Other 2.6 (0.4)***

-80.7%

7.4 (1.4)***

-77.5%

9.7 (1.9)***

-70.5%

5.1 (1.1)***

-56.7%

10.8 (3.2)***

-40.1%

6.7 (5.1)

-15.8%

Science 10th Grade 11th Grade 12th Grade 10th Grade 11th Grade 12th Grade

N 61,936 39,061

White 0.6 (1.0)

-13.5%

-15.2 (12.1)

171.6%

-14.5 (5.8)*

18.2%

-1.6 (3.3)

8.1% N/A N/A

Black -0.1 (1.1)

1.4%

-15.7 (11.7)

208.2%

-19.7 (7.03)**

27.0%

-2.2 (4.1)

8.8% N/A N/A

Hispanic 1.1 (1.1)

-27.9%

-11.1 (10.4)

134.5%

-13.9 (7.7) †

18.0%

4.0 (3.9)

-20.6% N/A N/A

Other 0.6 (1.4)

-11.9%

-14.5 (12.0)

160.2%

-17.5 (7.6)*

23.5%

-1.1 (4.7)

4.9% N/A N/A





Table 5 ECHS’s effects on the probability of on-track pipeline progression by parental educational attainment



N 97,454 89,232

BA plus 1.2 (0.2)***

-74.0

3.9 (0.7)***

-70.0%

5.3 (1.1)***

-59.6%

3.8 (0.7)***

-55.2%

8.7 (2.4)***

-37.8%

4.3 (4.0)

-11.0%

Some College 1.6 (0.2)***

-81.1%

5.8 (1.0)***

-77.8%

8.6 (1.5)***

-69.1%

4.1 (0.8)***

-52.2%

8.6 (2.6)***

-34.0%

3.0 (4.6)

-7.2%

High School 2.7 (0.4)***

-82.4%

8.8 (1.4)***

-79.1%

12.5 (1.8)***

-71.3%

5.3 (1.0)***

-49.4%

9.5 (2.8)***

-30.8%

2.9 (4.8)

-6.2%

No High School 3.8 (0.5)***

-83.8%

10.4 (1.8)***

-80.9%

13.8 (2.6)***

-74.9%

8.2 (1.4)***

-59.3%

14.8 (3.3)***

-44.6%

11.7 (4.6)*

-25.6%


N 61,936 39,061

BA plus 0.7 (0.7)

-21.4%

-10.3 (9.3)

156.0%

-17.9 (8.9)*

25.1%

-0.2 (4.1)

8.8% N/A N/A

Some College -0.2 (0.8)

9.2%

-14.9 (11.1)

266.6%

-19.6 (6.4)**

26.6%

-2.1 (3.5)

11.9% N/A N/A

High School 0.5 (1.2)

-9.7%

-16.9 (12.8)

172.5%

-14.4 (5.4)**

18.0%

-1.0 (3.8)

4.0% N/A N/A

No High School 1.7 (1.8)

-24.1%

-13.3 (12.5)

105.3%

-15.6 (8.4) †

20.6%

-4.0 (5.5)

15.5% N/A N/A





Table 6 ECHS’s effects on the probability of on-track pipeline progression by 8th grade mathematics achievement



N 97,454 89,232

95th Percentile 0.0 (0.1)

-33.1%

0.2 (0.3)

-23.0%

-0.2 (0.9)

14.1%

0.1 (0.0)*

-49.2%

0.3 (0.3)

-30.2%

-0.4 (1.2)

13.9%

75th Percentile 0.7 (0.1)***

-71.7%

2.4 (0.5)***

-67.4%

3.4 (0.9)***

-55.1%

1.3 (0.3)***

-51.1%

3.4 (1.3)**

-33.1%

0.0 (3.2)

-0.1%


-82.1%

7.8 (1.2)***

-79.0%

10.5 (1.3)***

-72.1%

5.7 (0.9)***

-51.3%

10.7 (2.4)***

-32.9%

4.2 (3.9)

-8.5%


-87.4%

20.0 (2.5)***

-84.5%

25.3 (2.5)***

-79.8%

16.3 (2.2)***

-46.4%

17.7 (3.4)***

-26.4%

7.9 (3.4)*

-9.7%


-90.3%

54.6 (4.0)***

-86.1%

60.0 (3.8)***

-81.9%

18.2 (3.1)***

-22.1%

6.6 (4.3)

-6.8%

1.9 (2.5)

-1.9%


N 61,936 39,061

95th Percentile -1.2 (0.7)

101.8%

-16.5 (11.2)

585.9%

-34.3 (7.4)***

58.4%

-1.6 (1.3)

50.7% N/A N/A

75th Percentile -0.6 (0.8)

23.6%

-16.2 (11.4)

298.0%

-22.8 (6.6)***

32.5%

-2.2 (2.4)

21.0% N/A N/A


11.5%

-14.9 (11.7)

174.5%

-15.8 (6.2)*

20.5%

-1.1 (3.3)

5.4% N/A N/A

25th Percentile 2.4 (1.2) †

-36.2%

-12.1 (12.1)

90.9%

-9.8 (6.1)

11.8%

1.9 (4.0)

-4.9% N/A N/A


-60.2%

-3.4 (13.1)

13.3%

-2.6 (6.0)

2.9%

6.9 (4.2)

-9.7% N/A N/A





Table 7 ECHS’s effects on the probability of on-track progression through the alternate science pipeline

Alternate Science Persistence Alternate Science Proficient Performance

11th 12th 11th 12th

N 24,620 20,132

Average Student 1.7 (0.9)*

-67.8%

0.3 (1.4)

-2.4%

-1.0 (2.9)

14.1% N/A

Race/Ethnicity

White 4.6 (3.2)

-62.7%

-0.2 (6.9)

1.4%

0.3 (1.8)

-2.5% N/A

Black 6.0 (5.7)

-52.1%

0.8 (10.4)

-4.9%

-1.4 (2.5)

11.9% N/A

Hispanic 0.1 (0.3)

-19.4%

-5.6 (7.3)

182.6%

-1.2 (3.2)

16.3% N/A

Other Race 3.8 (3.5)

-77.2%

4.1 (8.8)

-34.0

-0.4 (3.1)

3.7% N/A

Parental Education

BA Plus 1.2 (0.7)

-85.5%

2.9 (2.8)

-49.1%

2.0 (1.4)

-28.5% N/A

Some College 0.5 (1.6)

-52.3%

-1.0 (2.4)

52.2%

-2.7 (2.0)

35.6% N/A

High School 5.8 (4.8)

-47.5%

-4.6 (9.6)

19.5%

-1.9 (1.9)

17.0% N/A

No High School

27.1 (16.5)

-35.9%

15.4 (18.0)

-18.4%

3.0 (4.3)

20.4% N/A

8th grade Mathematics Achievement


-52.3%

-1.2 (3.1)

38.7%

-0.4 (0.4)

32.2% N/A


-57.4%

-1.3 (4.9)

17.0%

-0.6 (0.8)

13.6% N/A


-60.2%

-0.6 (6.3)

4.2%

-0.3 (1.5)

2.8% N/A


-61.9%

1.6 (8.3)

-6.2%

1.3 (2.7)

-5.6% N/A

5th Percentile 19.1 (9.9) †

-61.2%

8.3 (12.6)

-15.7%

5.9 (5.7)

-10.4% N/A





Table 8 ECHS’s effects on the probability of on-track pipeline progression based on difference-in-difference approach

Mathematics Science 9th Grade 10th Grade 11th Grade 10th Grade 11th Grade 12th Grade

N 116,292 105,515 97,347 105,515 97,347 53,070

Persistence 0.5 (1.5) -8.4%

0.5 (2.3) -3.7%

1.4 (3.1) -6.7%

0.2 (0.9) -2.8%

-0.2 (1.7) 1.6%

-6.8 (8.4) 9.9%


-0.5 (2.8) 2.6%

5.2 (3.7) -12.2%

5.0 (4.0) -8.6%

1.0 (3.0) -4.0%

N/A N/A

Notes: Separate models estimated by grade, subject, and pipeline progression measure. The first number in the first row of each cell is the treatment effect in percentage points; second number is the effect’s robust standard error. The number in the second row is the percent change (reduction if negative or increase if positive) in the students risk of being off-track in the absence of reform. Model fit statistics presented in Table A2 in the appendix.




Appendix

Table A1 Model fit statistics for ECHS’s effects on the probability of on-track pipeline progression

Mathematics Science Alternate Science

Persistence Proficient

Performance Persistence

Proficient

Performance

Persistence Proficient

Performance

Average Student Effects

Variance, τ01 1.269 0.502 1.445 2.211 0.832 0.464

Reliability, β1 0.770 0.736 0.778 0.893 0.527 0.641

Variance, τ02 1.042 0.632 2.178 -- 0.973 --

Reliability, β2 0.718 0.717 0.796 -- 0.428 --

Variance, τ03 0.967 0.704 1.828 -- -- --

Reliability, β3 0.606 0.686 0.842 -- -- --

Effects by Student Subgroups

Variance, τ01 1.235 0.497 1.415 2.111 0.796 0.458

Reliability, β1 0.767 0.734 0.774 0.889 0.521 0.638

Variance, τ02 1.048 0.632 1.981 -- 1.035 --

Reliability, β2 0.720 0.717 0.784 -- 0.439 --

Variance, τ03 0.959 0.709 1.687 -- -- --

Reliability, β3 0.605 0.687 0.835 -- -- --

Table A2 Model fit statistics for ECHS’s effects on the probability of on-track pipeline progression, difference-in-difference approach

Mathematics Science 9th Grade 10th Grade 11th Grade 10th Grade 11th Grade 12th Grade

Persistence Variance, τ00 0.523 0.319 0.269 0.190 0.200 0.848 Reliability, β0 0.958 0.931 0.915 0.855 0.888 0.957 Proficient Performance Variance, τ00 0.219 0.160 0.176 0.193 Reliability, β0 0.914 0.858 0.853 0.903

assessing the impact of north carolina's early college high

Documents