1

Promise kept? Free community college, attainment, and earnings in Tennessee1

February 2020

WORK IN PROGRESS

Celeste K. Carruthers University of Tennessee

William F. Fox

University of Tennessee

Christopher Jepsen University College Dublin, IZA, and CES-Ifo

JEL Codes: I22, I28, D83, H31

Abstract

We examine the effect of a local “free community college” program on college credit accumulation, completion, and short-term labor market outcomes. Knox Achieves pledged tuition-free community college to any Knox County, Tennessee high school graduate and served as the model for the statewide Tennessee Promise program as well as a growing number of state initiatives across the U.S. We find that Knox Achieves eligibility or participation led to a higher likelihood of attaining a two-year college credential alongside heterogeneous changes in the likelihood of attaining a bachelor’s degree. Eligible higher-achieving students, some of whom may have been swayed to start at a community college rather than a four-year school, were less likely to complete a bachelor’s degree after the introduction of Knox Achieves, particularly if they were economically disadvantaged. Lower-achieving students, however, were more likely to complete two-year and four-year degrees, as were eligible black and Hispanic students. We also find that Knox Achieves is associated with significantly higher in-state earnings up to 7 years after high school, although estimated effects taper thereafter. These medium-term labor market effects are driven by differential retention in in-state work, as well as higher earnings conditional on working.

1 We gratefully acknowledge tnAchieves, the Tennessee Higher Education Commission, Tennessee Department of Education, and the Tennessee Department of Labor and Workforce Development for providing data used in this study. We are thankful for comments and suggestions from Nicolas Salamanca, Josh Goodman, Michael Kofoed, Judy Scott-Clayton, conference/seminar attendees at the Southern Economic Association, Association for Education Finance and Policy, Association of Public Policy Analysis and Management, Irish Economic Association, Labor Econometrics Workshop, Melbourne Institute, University of Michigan, American University, University of New Hampshire, and Upjohn Institute. Bilal Celik and Tom Jenkins provided outstanding research assistance. Finally, we are grateful for funding from Upjohn Institute and Strada Education Network. All errors are our own, and the opinions and findings discussed in this study do not represent the opinions of our funders, tnAchieves, or any Tennessee state agency.

2

I. Introduction

The history of higher education is threaded with private and public efforts to make the

endeavor “free” of tuition for needy students, students studying for particular vocations,

returning veterans, and students enrolling in new and growing institutions. Individual need-based

scholarships date back to the 12th century when students at the University of Bologna (originally

but temporarily a free institution) pooled resources to help countrymen repay their debts (Fuller,

2014). The tradition of private altruistic scholarships arrived in America when Harvard

University established the colonies’ first endowed scholarship for poor students in the late 17th

century (Wilkinson, 2005). Today, many nations go well beyond need-based aid: free or nearly

free postsecondary education is a defining feature of university systems in Scandinavia, Brazil,

Mexico, and elsewhere.

While the ethos of need-based aid has a long precedence in the United States, the idea

that education should be free for any student was rarely institutionalized beyond secondary

grades.2 Until recently, the most sweeping example of public support for free higher education

was contained in California’s 1960 Master Plan for Higher Education, which stipulated that state

residents pay no tuition. Escalating enrollment fees have since eroded the Master Plan’s intent.

Although national or statewide free higher education has remained infeasible for most of

the U.S. timeline, private philanthropists found a way to provide college tuition as an entitlement

to traditional-aged students residing in particular cities. The beginning of the modern movement

for free higher education is often attributed to the Kalamazoo Promise of 2005, which committed

to pay the college tuition of Kalamazoo high school graduates in the classes of 2006 and later

who enrolled in one of Michigan’s public colleges or universities.3 Kalamazoo Promise was

2 Scattered individual exceptions include military academies, Berea College, the Free Academy in New York City (today’s tuition-collecting City University of New York), the Williamstown Free School of 1791 (now Williams College, which charged tuition starting in 1793), and Cooper Union in New York City (which began charging tuition in 2014). 3 Kalamazoo Promise was one of the more prominent modern scholarships with “place-based” eligibility criteria, but it was not the only one. Earlier examples of place-based grants include the Clemens Foundation program of Philomath, Oregon, which originated in 1959 to benefit the small logging town’s high school graduates; the 1990 Philadelphia Education Fund, Online to College, a 1998 last-dollar scholarship for California’s Chaffey College students who graduated from the Ontario-Montclair school district east of Los Angeles; and the Ayres Foundation Scholars Program, a last-dollar scholarship started in 1999 for students in three rural Tennessee counties. Another early program is the Oklahoma Promise, which started in 1991. It targets low-income students, with a maximum family income of $50,000 per year. See Mendoza and Mendes (2012) for an analysis of college persistence in the wake of the Oklahoma Promise.

3

envisioned in equal parts as a community revitalization tool and financial aid package for the

erstwhile manufacturing hub’s needy and middle-income students (Zipp, 2005). Between 2005

and 2016, place-based scholarships proliferated across the country. Many, like Kalamazoo

Promise, were intended to retain residents and students in shrinking city schools. Most were

privately, and generously, funded.

In 2014, Tennessee passed the first state legislation since California’s Master Plan

devoting public resources toward making college free for all state residents. At the time it was

implemented, Tennessee Promise fit into the state’s “Drive to 55” campaign to increase

postsecondary attainment among the working-age population from 39.3%4 to 55% by the year

2025. Unlike the California plan of the late 1960s, Tennessee Promise covered tuition and

required fees, but only for high school graduates making a seamless transition to public two-year

postsecondary institutions in the state. As a “last-dollar” scholarship, Tennessee Promise pays for

the gap between the value of two-year tuition plus fees and a student’s entitled grants and

scholarships from federal, state, and institutional sources. The program’s first beneficiaries—

numbering over 16,000 according to state figures—enrolled in college in the fall of 2015,

contributing to a 10% year-over-year rise in first-time freshmen enrollment. Tennessee Promise

was followed by several similar initiatives in other states. To date, there are 15 active and 18

proposed statewide promises of free college, most of which are structured as last-dollar aid

(Jones and Berger, 2018).

Tennessee Promise was closely modeled after a privately funded last-dollar program in

Knox County, Tennessee, known as “Knox Achieves” from 2008-2011, and then “tnAchieves”

from 2011 thereon.5 Rather than city revitalization or the ethos of free higher education per se,

Knox Achieves, tnAchieves, and Tennessee Promise were motivated by workforce development,

the idea that higher education was necessary for individual and state prosperity, and evidence

pointing to a skills gap between employer needs and the state’s human capital (Carnevale and

Smith, 2012).

4 https://www.luminafoundation.org/stronger_nation2016 5 tnAchieves and the Ayres Foundation Scholars Program remain in operation as Tennessee Promise partnering organizations that coordinate mentoring, community service, and other student supports. tnAchieves serves 90 counties across the state, and the Ayres Foundation serves five.

4

It is too early to assess whether Tennessee Promise will have the intended effect of

increasing postsecondary attainment for traditional college students, although enrollment gains

give cause for optimism. But the effect of Knox Achieves availability on college persistence and

completion may foreshadow the effect of current and proposed statewide programs like the

Tennessee Promise. Knox Achieves raised college enrollment among eligible high school

graduates by about 3 percentage points, or 5-6% of average matriculation rates in the Knoxville

and East Tennessee area (Carruthers and Fox, 2016).

By assessing the effect of Knox Achieves availability and participation on college

completion, we provide the first insights on how a last-dollar scholarship program shapes student

attainment and earnings. Our fundamental empirical challenge is characterizing the

counterfactual without the benefit of plausibly random variation in eligibility. Like many place-

based programs, eligibility for Knox Achieves was transparent and unbounded by income or

merit criteria. Every Knox County 12th grader beginning with the class of 2009 was eligible, and

students participated from every high school in the county. Universal eligibility was a critical

feature of the program, which sought to simplify the question of college price with a powerful

message of “free” community college for any interested student. We take advantage of

geographic and intertemporal variability in access to Knox Achieves to estimate the effect of the

program on college and labor outcomes, relying on difference-in-difference and matching

identification alongside a host of robustness and falsification tests.

Relative to the best available counterfactual, Knox Achieves eligibility is associated with

a higher likelihood of attaining a two-year postsecondary credential, a negative but imprecisely

estimated change in the likelihood of attaining a four-year college degree within 6 years of high

school, and higher earnings 7 years after high school, tapering in years 8-9. These helicopter-

level intent-to-treat effects mask heterogeneity with important policy implications. Moderately

low-income and lower-achieving students appear to have benefitted most from access to free

community college, exhibiting significant increases in all levels of attainment. For these

students, the last-dollar scholarship may have met financial needs not covered by family

resources available to higher-income students, need-based aid available to lower-income

students, or merit-based aid available to higher-achieving students.

5

Democratization of two-year college access comes at some cost, however, as eligible

students were no more or less likely to complete a bachelor’s degree within 6 years, and the

lowest-income students actually saw significant declines in four-year college completion rates.

Students who started in a community college rather than a four-year school because of Knox

Achieves would have had to navigate a transfer pathway to attain a bachelor’s degree, which can

be a complicated, opaque process (Boatman and Soliz, 2018), and they may have been further

dissuaded by steep changes in out-of-pocket tuition between the two sectors. As a result, our

back-of-the-envelope calculations suggest that the net benefits of participating in a program like

Knox Achieves, as measured by expected changes in lifetime earnings, are uncertain and will

depend critically on the extent of substitution between community colleges and four-year

degrees, as well as future returns to those degrees.

In terms of short-term earnings, Knox Achieves eligibility appears to significantly raise

in-state earnings covered by Unemployment Insurance (UI) 4-7 years after high school, tapering

in the 8th and 9th years. We delve into two potential mechanisms pushing earnings up:

participation in UI-covered work, and industry of employment. We find that Knox Achieves

eligibility and participation are associated with a sustained higher likelihood of any employment

in UI-covered jobs 1-9 years after high school, as well as lower rates of attrition. Program

participants tend to work in more lucrative industries (i.e., industries with higher U.S. average

earnings) than other students, and this tendency grows over time, but it may reflect positive

selection rather than a causal effect of free community college. Countywide, the introduction of

Knox Achieves did not significantly change our summary measure of industry quality.

Except where noted otherwise, results are robust to a large and flexible set of observable

student features, school features, and interactions therein, and we estimate that non-zero

treatment effects are robust to a substantial degree of selection on unobservables. Estimated

effects of Knox Achieves eligibility on two-year attainment and earnings are large and atypical

among random placebo permutations, whereas findings for college credits within four years and

bachelor’s degree completion within six are less distinguishable from permutation noise.

Preferred specifications are well balanced and we do not detect consistently significant OLS

“effects” on pseudo-outcomes estimated from pre-program predictions. Matching treatment

effects tend to be no smaller than linear difference-in-difference estimates.

6

Nearly all of the previous work on Promise programs focuses on generous first-dollar

scholarships funded by private sources and targeting students in particular cities. Last-dollar

scholarships are more financially viable for statewide efforts, however, because they can make

similar promises of free college but contain costs by counting other grants and scholarships

against potential benefits and limiting those benefits to less costly postsecondary options. As our

results suggest, this subsidized but constrained set of choices can benefit students in the middle

of the income distribution and the bottom half of the achievement distribution, who might not

have the family resources, need-based aid opportunities, or merit-based aid opportunities of

others. Policymakers need to be aware of this large but often-overlooked group of potential

college students, whose enrollment and completion may determine the degree to which the

college-educated population will expand in the future. Estimated effects are larger for black and

Hispanic students as well, who have been traditionally underrepresented among college

completers. At the same time, evident substitution from four-year schools to two-year

community colleges, and lower bachelor’s degree completion among higher-achieving and

lower-income students, highlights a potential consequence of restricted, last-dollar scholarship

programs as well as the costs and hurdles that students face when they consider transferring from

two-year to four-year schools.

II. Policy Background and Related Research

Knox Achieves launched with the 2008-2009 class of Knox County 12th graders after a

group of local business and civic leaders contributed funds to cover community college tuition

and fees for up to 500 students. Any public high school senior in the county was eligible to

apply. The 500-student ceiling was nonbinding for the class of 2009 (493 signed up and 241 later

enrolled) and eliminated for later, larger cohorts. Interested students signed up for Knox

Achieves in the fall of their senior year of high school. This necessary first step—signing up for

more information and support leading to free community college—will serve as one measure of

the Knox Achieves treatment in analyses to follow. A broader and preferred treatment indicator

covers all students who had access to Knox Achieves, i.e., all high school seniors in the

graduating classes of 2009-2011.

In order to retain eligibility, participants from the fall signup lists were expected to meet

with mentors (volunteers drawn from the community), file a Free Application for Federal

7

Student Aid (FAFSA) in the spring of their senior year, graduate from high school, verify their

FAFSA if necessary, complete an eight-hour community service project, and then seamlessly

enroll in a community college. In return, Knox Achieves paid for tuition and mandatory fees for

up to five semesters. Between the fall of senior year and the fall of the following year, program

attrition measured 45 percent; that is, 55 percent of fall signees fulfilled program requirements

and enrolled in a community college the following year, while others enrolled in a different

sector, did not enroll in college, or enrolled in a community college without Knox Achieves.

By studying Knox Achieves, we contribute to a small but growing literature on location-

based “promise” programs. A common theme of these programs is that a locality makes a

promise to provide postsecondary funding, often in conjunction with other services, for students

who meet noncompetitive eligibility requirements in addition to living and attending secondary

school in the location. There are over 140 local place-based programs that meet this broad

definition,6 in addition to at least 15 active and 16 proposed statewide programs (Jones and

Berger, 2018). Promise programs exhibit many differences regarding the amount of aid, student

GPA and residency requirements, and the type and number of institutions covered. Some, like

the El Dorado Promise, commit to cover costs at any accredited college or university in the

country. Others, such as the Kalamazoo Promise, limit benefits to institutions within a state. And

still others, such as the Chaffey College Online to College initiative, are limited to one

institution. Swanson et al. (2016) survey the landscape of Promise initiatives in the U.S. as well

as the limited research base seeking to assess the impact of Promise aid on student outcomes.

Exposure to a place-based scholarship opportunity has been shown to increase the

likelihood of college enrollment in several settings across the U.S. (Carruthers and Fox, 2016;

Daugherty et al., 2016; Bartik et al., 2019; Page et al., 2018). Regarding college completion,

Bartik et al (2019) find the Kalamazoo Promise raised the likelihood of completing a bachelor’s

degree by 25 percent, and Swanson and Ritter (2018) find that the El Dorado Promise led to 32

percent gain in bachelor’s degrees. Those two applications each focus on a generous and

privately funded “first dollar” free college initiative that pays for a student’s college tuition

irrespective of need-based and merit-based aid from other sources, and irrespective of whether a

student enrolls in a community college or a more costly four-year university. Last-dollar

6 See the Upjohn Institute Promise Database here: https://www.upjohn.org/promise/.

8

programs like Knox Achieves and Tennessee Promise cover any gap between college tuition and

financial aid from institutions, states, and the federal government, and further contain costs by

restricting benefits to community college students. Knox Achieves requires that applicants fill

out a FAFSA, and any federal or state aid received by the student as a result of FAFSA

processing reduces the amount of aid provided by the program. Eligible students still receive

tuition-free community college, but the cost of the program is then shared with other sources. As

a result, program costs for Knox Achieves were just under $1,000 per enrolled student per year,

versus up to $10,000 per student-year in the New Haven and Pittsburgh Promise programs,

$8,000 per student-year in Kalamazoo, and up to $9,000 in El Dorado. Students in a first-dollar

program, but not in a last-dollar program such as Knox Achieves, can use scholarship funds in

excess of tuition to offset expenditures for books, supplies, transportation, and living expenses.

We add to what we know about college financial aid in general, and promises of free

college in particular, in a few ways. Foremost, we examine the effect of Promise availability on

longer-term credit accumulation and college completion; the Bartik et al. (2019) study of the

first-dollar Kalamazoo Promise and Swanson and Ritter (2018) study of the first-dollar El

Dorado Promise are the only others we are aware of to do so quasi-experimentally. We analyze

students’ college persistence and completion over a time period of at least six years following

high school.

Second, we provide the first estimates of college completion for a last-dollar scholarship

program that only covers community colleges, extending results reported by Carruthers and Fox

(2016) for college enrollment. Knox Achieves is a lower-cost, lower-benefit aid vehicle with a

broader base of eligible students, compared with most first-dollar city programs, such as

Kalamazoo and El Dorado, that entail high school GPA requirements and/or residency for

several years. Knox Achieves was available to every high school senior regardless of GPA or

tenure in the county, but the program provides scholarships only for community colleges. About

half of enrolled participants qualify for enough Pell aid that they receive no funding from Knox

Achieves itself, which is also true for the program’s successor, statewide Tennessee Promise.

Our findings are particularly useful for policymakers considering publicly-funded, last-dollar

programs where first-dollar programs are prohibitively expensive.

9

Third, we characterize the effect of free community college on employment, earnings,

and industries of occupation up to 9 years after high school, quantifying the medium-term

tradeoff between democratization of college access and, for some students, diversion from what

is traditionally a higher-return path through four-year college.

And finally, a necessary methodological difference from the most related literature is

how we characterize the counterfactual to Knox Achieves access. Universal place-based

eligibility is a core feature of the program, and one that is certainly worth study, but it does not

permit the kind of difference-in-difference identification used to evaluate programs where

benefits are conditioned on GPA or how long a student lived in the qualifying city. Rather, the

first conditions of Knox Achieves are (1) attending a Knox County high school as a senior in the

class of 2009, 2010, or 2011, and (2) signing up for the program in the fall of senior year. We use

these initial points of access as sources of identifying variation in the Knox Achieves treatment.

Recognizing that both plausibly coincide with other determinants of college success, we

scrutinize ex ante expectations for participating students, we explore the extent to which linear

regression results are sensitive to much more flexible functional forms alongside data-driven

model selection (Hansen et al., 2014), and we gauge the potential severity of selection on

unobservables using coefficient stability methods proposed by Oster (2017). We also relax

linearity assumptions by pursuing complementary matching techniques. Matching identification

rests on a similar unconfoundedness assumption as linear difference-in-differences, but it relaxes

functional form assumptions and minimizes the influence of outliers. This is particularly

valuable when covariate distributions are distinct across treatment and control units (Imbens,

2015), which is certainly the case with Knox Achieves participants and other students. A

remarkably consistent story echoes across these different approaches, where free community

college leads to large gains in two-year college completion, at best no change in four-year

college completion, and better work outcomes several years after high school.

III. Methods

III.A. Data and Summary Statistics

We begin with longitudinal administrative data describing the universe of 2006-2007

through 2010-2011 high school seniors in Tennessee’s public school systems, linked to their

10

enrollment records in public two-year and four-year higher education institutions throughout the

state and in-state earnings covered by Unemployment Insurance. We join these data with Knox

Achieves participant lists from the senior classes of 2009-2011. Participant lists include all

students who signaled their interest in the program in the fall of 2008-2010, by signing up to

receive more information and a volunteer mentor. In some analyses to follow, these lists

compose the Knox Achieves treatment group, even though just 55 percent of fall participants

later enrolled in a community college as a Knox Achieves student. Additionally, we observe

initial college enrollment details from the National Student Clearinghouse. Clearinghouse data

on hand do not extend beyond the year immediately following high school. Accordingly, our

focus is on postsecondary outcomes in Tennessee’s public colleges and universities. The

Tennessee Higher Education Commission (THEC), a coordinating body spanning the state’s

university and college systems, maintains data on these students. Ninety-one percent of college-

going students in the state enroll in a THEC institution, and results to follow account for the

plausible idea that Knox Achieves swayed some higher achieving students to enroll in a public

institution rather than a private or out-of-state institution (details are provided in Section III.B).

Table 1 provides descriptive statistics for all Tennessee 12th graders (Column 1), Knox

County Knox Achieves signees (Column 2), and Knox County non-participants (Column 3),

omitting high school graduates who enrolled seamlessly in a non-THEC higher education

institution. Knox Achieves participants accounted for 16.8% of eligible cohorts. As plainly seen

from the table, participants were substantially more likely to enroll in college (two or four year)

than their peers in Knox County and throughout Tennessee. Just under half of Knox County non-

participants seamlessly enrolled in college, versus 75 percent of program participants.

Outcomes of interest include THEC accumulated credits within two and four years of

high school, any enrollment in a four-year institution, postsecondary certificate or associate’s

degree attainment within four years of high school, bachelor’s degree attainment within six

years, and labor market outcomes up to nine years after high school. Labor market outcomes are

derived from in-state earnings recorded in the state’s Unemployment Insurance system, which

covers the vast amount of earnings from work among state residents, with notable exceptions of

earnings from self-employment, some agricultural or federal employers (including the military),

and any earnings from working in other states. Age-earnings profiles vary across industries,

11

which complicates the comparison of early-career earnings. To address this, we compute

earnings by fiscal year (third quarter of each calendar year through second quarter of the next

calendar year) in the nine years following high school as well as an industry score, which we

construct from a worker’s industry (specifically, its 4-digit NAICS code) and U.S. average

earnings among workers covered by Unemployment Insurance. U.S. figures are drawn from the

Census Quarterly Workforce Indicator. For each year where a worker has in-state, UI-covered

earnings, we compute their industry score as the average annual U.S. earnings in that industry

that year, or as a weighted average if they worked in multiple industries. UI-covered earnings

and industry scores are inflation-adjusted to 2019 dollars using the Consumer Price Index for

Urban Consumers.7

Knox Achieves participants outperformed other students in terms of two-year

postsecondary credentials. Knox Achieves students are 3-6 times more likely than their peers to

attain a certificate or associate’s degree within four years, at least 200% of the “normal time” to

complete those credentials. Relative to the state and non-participating peers, however, Knox

Achieves participants are less likely to graduate with a four-year college degree. We find that 17

percent of participants receive a bachelor’s degree within six years of completing high school,

noticeably below the rate of 19 percent for all 12th graders and 25 percent for non-participants in

Knox County. Earnings in the year immediate after high school are similar across groups,

measuring $4,470-5,440 on average, which includes missing earnings recorded as zero.8 We

observe earnings up to seven years after high school for all 2006-7 through 2010-11 seniors, up

to eight years after high school for the first four cohorts, and up to seven years after high school

for the first three. Results in tables to follow focus on the estimated effects of Knox Achieves on

earnings seven years later, which we can quantify for all three Knox Achieves cohorts, although

Figures 2-3 trace effect estimates for all observed years.

Looking to student characteristics, the profile of a typical Knox Achieves student is one

who is socioeconomically disadvantaged but performing close to or better than par in terms of

standardized achievement. Program participants are more likely to be female, non-white, and

7 https://fred.stlouisfed.org/series/CPIAUCSL 8 Results to follow include analyses of log-earnings among those with non-missing earnings as well as the binary incidence of having any UI-covered earnings in a given fiscal year.

12

lower-income than their peers throughout the county or state, but they have similar or better

composite scores on end-of-course exams and the ACT.

III.B. Linear Estimation

To begin assessing the potential impact of Knox Achieves availability on students’

college success, we estimate the following student-level model for the 12th grade classes of 2007-

2011, which includes all eligible Knox County seniors from 2009-2011 as well as ineligible

students from other counties and pre-program cohorts:

𝑌𝑌𝑖𝑖 = 𝛼𝛼0+ 𝛼𝛼𝑡𝑡 + 𝛼𝛼𝑠𝑠 + 𝐾𝐾𝐾𝐾𝑖𝑖𝛾𝛾 + 𝑋𝑋𝑖𝑖𝑠𝑠𝑡𝑡𝛽𝛽 + 𝜀𝜀𝑖𝑖𝑠𝑠𝑡𝑡, (1)

where Yi is a college outcome such as credits earned or associate degree received for student i,

KAi is either an indicator for Knox Achieves availability (equal to one for all Knox County

seniors in the classes of 2009-2011) or an indicator for seniors who signed up for Knox Achieves

in the fall, αt is a cohort fixed effect, αs is a school fixed effect, and Xist is a vector of observable

features describing student i and his or her school s. Students with no record of college

enrollment have Yi coded as zero. Robust standard errors allow for εist residuals to be correlated

within counties. Student controls include continuous or binary variables describing gender, race,

Hispanic ethnicity, repeating the 12th grade, free or reduced-price lunch eligibility at any time

since 8th grade, earnings during junior and senior year, the number of districts and counties

attended between 8th and 12th grades, ACT composite and subject scores, standardized end-of-

course exam scores (English, math, science), the number of end-of-course records overall, and

indicators for missing data on these variables.9 These features are intended to characterize

students’ basic demographics, ability, and need, each of which is expected to be important

signals or determinants of college success. School controls include the student-teacher ratio, and

the share of students who are white, Hispanic, or eligible for free or reduced-price lunch. We

also control for schools’ urbanicity (indicators for city or suburban locales), schoolwide Title I

status (another signal of economic disadvantage), and contemporaneous county unemployment

rates.

9 Missing ACT and end-of-course scores are imputed at the 25th percentile.

13

One threat to interpreting γ estimates as the causal effect of Knox Achieves is the

possibility that the sample of students with observable college credits and degrees is itself

influenced by Knox Achieves. Although we can identify the immediate college destinations of

almost all 2007-11 high school graduates through the National Student Clearinghouse, we only

observe credit accumulation and completion for students who enroll in Tennessee’s public

universities and colleges. And since Knox Achieves emphasized community college enrollment,

it is plausible that our capacity to observe a student’s postsecondary progression was influenced

by the program itself. Indeed, using the Equation (1) model, we find that Knox Achieves

participation reduced the likelihood of enrolling in a private or out-of-state institution by 5

percentage points, nearly equal to the 5.6% share of all ineligible students who enroll as such.

Estimates of γ would overstate the effect of Knox Achieves on success in college if students who

decided to enroll in a public in-state institution because of Knox Achieves were innately higher-

achieving, i.e., positively selected, relative to their peers. To account for this possibility, main

results are reported for a truncated sample, where we exclude a portion of Knox Achieves

participants with the largest college credit accumulation after four years, under the assumption

that participants who were convinced to enroll in one of Tennessee’s public colleges or

universities would have persisted in college longer than all other program participants. How

much longer is not clear because we cannot observe accumulated credits in the National Student

Clearinghouse records on hand. Main results are reported with the top 5% of participants

excluded.

Another hazard of using Equation (1) to estimate the effect of Knox Achieves

engagement on college outcomes is the idea that unobserved factors coincide with program

availability or take-up as well as subsequent college outcomes. Differences in the college

prospects of Knox County students or participants’ inherent interest in a community college

education are chief examples of such factors. We respond to this threat in a few ways. First, we

look for differences in expected college behavior by Knox Achieves availability or participation.

Findings discussed in Section IV indicate that Knox Achieves participants and Knox County

students more broadly were on roughly the same trajectory as their respective counterparts who

did not engage with the program, and any differences are too small to explain away estimated

program effects. Second, we estimate Equation (1) with and without school fixed effects.

Controlling for school fixed effects limits identifying variation in Knox Achieves to non-

14

experimental student interest in the program more so than quasi-experimental program

availability. Nevertheless, Equation (1) appears to be more balanced with school fixed effects

(see Section IV), and results are broadly similar across specifications with and without school

fixed effects. Third, we test whether results from linear estimates are robust to much more

flexible specifications of the control vector, adding several hundred non-linear terms and

interactions. And finally, we estimate the extent of selection on unobservables, as a proportion of

selection on observables represented by Xist, that would imply a true γ of zero for each outcome.

We make this inference using Oster’s (2017) approach, which compares unconditional and

conditional γ estimates along with unconditional and conditional R2 values, to gauge the threat

from selection on unobservables.10 A high degree of selection on unobservables proportionate to

the estimable degree of selection on observables would be necessary to explain away the

magnitude of γ in settings where coefficients change very little but explained variance (R2) grows

much larger between unconditional and conditional specifications.

As a two-way fixed effects estimator, under the identifying assumption that successive

treated and untreated cohorts would have followed the same conditional trajectory of outcomes

(“common trends”), 𝛾𝛾� can be expressed as a weighted average of two-by-two difference-in-

difference estimates between treated and untreated schools (Chaisemartin & d’Haultfoeuille,

2019). Weights can be negative, and if average treatment effects are heterogeneous, this means

that 𝛾𝛾� can be biased or even of the opposite sign as average treatment effects. Using the

procedure outlined by Chaisemartin et al. (2019), we find that just 4% of weights in our

application are negative, with values that sum to 1.6E-05.

The Knox Achieves treatment affected a small number of groups: 18 out of 369 schools

statewide. As Conley and Taber (2011) and Ferman and Pinto (2019), among others, have

shown, cluster-robust standard errors can over-reject the null when there are a small number of

treatment groups. To assess whether our conclusions are vulnerable to this, for each outcome of

interest we estimate Equation (1) over 300 permutations, where the 𝐾𝐾𝐾𝐾𝑖𝑖 treatment is randomly

10 This method is an extension of work by Altonji et al. (2005) inferring the necessary degree of selection from coefficient stability. To proceed, we must assume linear functional forms, a proportionate selection relationship, and an upper bound on R2. Following Oster (2017), we assume that the maximum R2 is equal to 1.3 times the R2

observed in the fully conditioned specification of Equation (1).

15

reassigned to other schools. Results broadly support our main inferences, and are discussed in

more detail in Section IV.C.

III.C. Matching Estimation

The statistical profile of Knox Achieves participants summarized in Table 1 suggests that

they were notably different than non-participants in terms of demographics, achievement, and

financial security. This difference may mean that models such as Equation (1) will depend too

heavily on extrapolation when forming estimates of the conditional gap in outcomes between

treated and untreated students. In such cases matching estimators can be a suitable alternative to

linear models.

The intuition with matching is to pair each Knox Achieves participant with a

quantitatively similar non-participant and interpret the average difference in outcomes across

matched pairs as the treatment effect. We take two complementary approaches to defining

similarity between treated and control students. We first pair participants to similar students in

terms of several observable features in Xist: gender, race, Hispanic ethnicity, repeating the 12th

grade, free or reduced-price lunch, junior year earnings, number of districts since 8th grade, ACT

composite, English and math end-of-course exam scores, the number of end-of-course exams on

record, schoolwide percent white, schoolwide percent with a history of free lunch eligibility, and

the county unemployment rate. Mahalanobis matching computes a distance metric describing the

similarity of this control vector between every i, j pair of students and matches treated students to

the untreated nearest neighbor in terms of this distance metric. For Mahalanobis treatment effect

estimates, we compute standard errors according to Abadie and Imbens (2006). Figure A1 in the

appendix depicts the average standardized gap in observable features before and after matching

Knox Achieves participants to non-participants enrolled in the same county. As one would hope,

matching on these covariates limits observable differences between Knox Achieves participants

and comparison students.11

Our second matching approach is to pair Knox Achieves participants with non-

participants according to their estimated likelihood of participating in the program. We estimate

11 Mahalanobis matching minimizes the distance metric between matches without regard for which variables in the matching vector are most important for predicting college outcomes. With this in mind, our preferred Mahalanobis matching vector is limited to a subset of key Xist variables.

16

this propensity by logit for seniors who had the opportunity to participate, i.e., Knox County

classes of 2009-2011. Predictive factors include all variables represented in Xist. Parameter

estimates are mapped to seniors in other counties and pre-program cohorts. We use a nearest

neighbor propensity score estimator where matches are constrained to be within two percentage

points (i.e., a caliper of 0.02), within the range of overlap between treated and control propensity

(i.e., with common support), and below the top one percent of treated students’ propensity

distribution. Figure A2 in the appendix plots the distribution of predicted propensity by Knox

Achieves participation. While participants generally have a higher propensity than non-

participants, the area of common support is inclusive of them all.

IV. Results

IV.A. Balancing Tests

The critical identification assumption allowing γ estimates from Equation (1) to be

interpreted as a causal effect of Knox Achieves is that participation or access to the program was

conditionally independent of unobserved determinants of college persistence and completion.

This assumption would be violated if, for example, Knox County students were following a

divergent path relative to the rest of the state, in terms of expected college outcomes, or if

unobserved interest in a community college education coincided with expected college

persistence and completion. Although these scenarios and others like them are untestable, we can

utilize two pre-program cohorts to get a sense of how college outcomes would have changed in

Knox County in the absence of Knox Achieves. Specifically, we estimate Equation (1) for the

Tennessee classes of 2007 and 2008, substituting a linear time trend for cohort fixed effects. We

then map coefficient estimates to the classes of 2009-2011, when Knox Achieves was available

to Knox County seniors. Finally, we estimate Equation (1) for fitted outcomes of all students

from 2007 to 2011. Standard errors are computed by bootstrap from 200 replications of this last

step on random subsamples of 10,000 students. This test is a more parametric version of

common identification checks that visualize parallel trends across treatment and control groups.

Table 2 lists results for the pseudo-effect of Knox Achieves on college credit accumulation,

program completion, and earnings.

17

In Columns 1 and 3 the measure of Knox Achieves treatment is the broadest measure of

eligibility, i.e., an indicator for all Knox County high school seniors in the classes of 2009-2011.

The Column 2 and 4 models use a finer measure of treatment: an indicator for students who

signed up for Knox Achieves in the fall of their senior year. Columns 1-2 include county fixed

effects rather than school fixed effects.

The pattern of results in Column 1 suggests that ex ante predictions from an intent-to-

treat specification with county fixed effects are fairly unbalanced, in that we would expect Knox

Achieves eligibility to be associated with 6.5 additional credits within four years, a 0.6

percentage point lower likelihood of associate degree completion, and a 5.3 percentage point

higher likelihood of bachelor’s degree completion, based solely on post-treatment observable

student features and not necessarily the program itself. In other words, even in the absence of

Knox Achieves, there would have been a strong tendency for eligible students to accumulate

more college credits and attain four-year college degrees at a higher rate than the counterfactual.

Knox Achieves participants do not appear to share that tendency (Column 2), although their

(imprecise) expected gains in college credits and earnings after college are large.

Once we control for fixed but unobserved properties of treated schools (as seen in

Columns 3-4), we find just one statistically significant difference in expected outcomes across

treated and comparison students. Based on observable characteristics and school fixed effects,

Knox County students in the classes of 2009-11 were ex ante 0.3 percentage points more likely

to attain a postsecondary certificate than students in other counties (Column 3). As we show

below, however, actual treatment effect estimates for that outcome are many times greater than

0.3 percentage points. Conditional expectations for Knox Achieves participants were not

significantly different than for non-participants throughout the county and state (Column 4),

although as in the Column 2 specification, participants were expected to exhibit large gains in

earnings based on observable characteristics alone, with or without Knox Achieves.

Going forward, our preferred specification of Equation (1) controls for school fixed

effects and estimates either the intent-to-treat effect of eligibility, where KAi is an indicator for

Knox County seniors in the classes of 2009-11, or the treatment-on-the-treated effect of

participation, where KAi is an indicator for students who signed up to receive mentoring and

more information about the program.

18

IV.B. Equation (1) results

Table 3 lists linear estimates of γ for two different specifications of Equation (1), where

we include the full sample of students from 2007 to 2011. For each outcome and each

specification, coefficients are listed first, with robust standard errors in parentheses, and control

means below standard errors. Column 1 reports results where KAi is specified as a broad

eligibility indicator, meaning that γ estimates in Column 1 can be interpreted as intent-to-treat

estimates. We find that Knox Achieves availability preceded a countywide 1.6-credit, or 9%, rise

in college credits two years after high school, along with a 1-3 point rise in postsecondary

certificate or associate’s degree attainment within four years, and a statistically insignificant 0.5-

percentage point fall in bachelor’s degree attainment within six years of high school. Estimated

effects on certificate completion are quite large, exceeding the certificate attainment rate for

ineligible cohorts, and estimates for effects on associate degree completion are 25% of the

control mean. The imprecise coefficient estimate for bachelor’s degree attainment reported in

Column 1 suggests that eligibility was associated with a fall in bachelor’s completion as large as

1.4 percentage points (7%), or a gain no larger than 0.3 percentage points (2%). Turning to

earnings after high school, the Column 1 specification finds a $732.70 conditional increase in

annual earnings for eligible cohorts, or 5% of the control mean.

Column 2 reports conditional differences in college outcomes between program

participants and non-participants for the model with school fixed effects. As noted in Section

III.B, school fixed effects sacrifice cross-county identifying variation in Knox Achieves

availability for a more flexible representation of unobserved school-wide heterogeneity in factors

that determine college enrollment, persistence, and completion. The model suggests that Knox

Achieves participation preceded a 2.5 typical credit gain over non-participants after two years

(14% of the control mean), a small and weakly significant decline in accumulated credits after

four years, and significantly more presence and success in two-year college institutions than in

four-year institutions. Estimated gains in two-year degree and certificate attainment of eight and

nine percentage points, respectively, well exceed typical attainment rates for those credentials.

Participants also exhibit 7-percentage-point reductions in the likelihood of four-year degree

receipt, and they earned $1,456.49 more than their peers seven years after high school.

19

If countywide trends in postsecondary outcomes were driven entirely by Knox Achieves,

and if spillover effects were minimal between participants and non-participants, we would expect

Column 2 estimates to be about six times as large as those in Column 1, based on 16.8%

participation among eligible cohorts. For earnings, treatment effects implied by Column 1 are

much larger than what we observe among participants: over $4,000 in additional earnings per

year, of which observed Knox Achieves participation can explain just one-third. By contrast,

intent-to-treat effects for associate and bachelor’s degree completion imply notably smaller

treatment effects than what we estimate for those outcomes: associate gains of 6 percentage

points rather than 9, and lower bachelor’s attainment by 3 percentage points rather than 7. We

can reconcile these differences with two plausible varieties of unobserved factors that co-moved

with Knox Achieves eligibility or participation. First, it is reasonable to think that something

other than the program affected postsecondary interests and later employment opportunities

among Knox County students beginning with the class of 2009; locally favorable growth

following the Great Recession, perhaps (Murray, 2013). Second, it is even more reasonable to

think that students who signaled their interest in free community college were more inclined

toward associate programs than college and university programs in ways that were unrelated to

other observable controls. In Section IV.C we show that inclination would have to be many times

more influential on attainment than observable characteristics, but nevertheless, for much of this

section we prioritize inferences from intent-to-treat estimates. The effect of universal free

community college availability, versus take-up, is more relevant for policy, and more plausibly

independent of unobserved factors.

Figure 1 illustrates the relationship between Knox Achieves and γ estimates on earnings

for 1-9 years following high school. Specifically, left-hand Panels I, III, and V contain the

regression coefficients for Knox Achieves eligibility, and right-hand Panels II, IV, and VI trace

results for participation. We see a positive association between Knox Achieves eligibility and

earnings starting 5 years after high school but tapering by 8-9 years after high school. The

highest benefit is $732.70 in additional earnings the 7th year after high school (also reported in

Table 3), representing 5% of mean annual earnings among ineligible students. The increase in

earnings for Knox Achieves participants is more than twice as large as for eligible students, and

exhibited a similar taper 8-9 years after high school.

20

The earnings outcomes depicted in Panels I-II of Figure 1 are inclusive of unobserved

earnings, where we assign a value of zero. This is an informative summary measure of potential

effects on in-state earnings, but one that combines extensive and intensive margins of work. And

typical of earnings more broadly, the distribution of dollar values is right skewed. The second

two rows of Figure 1 depict estimated effects on the binary incidence of having any in-state UI-

covered earnings (Panels III-IV) and log earnings among those with any earnings (Panels V-VI).

Results indicate that Knox Achieves eligibility and participation are both associated with a

higher likelihood of working in Tennessee’s UI-covered occupations 1-9 years after high school,

as well as higher log earnings among UI-covered workers 1-8 years after high school.

Knox Achieves is linked to a lower likelihood of attrition from UI-covered work, not just

temporally as in Figure 1 Panels III-IV, but in a more lasting sense as we show in Figure 2

Panels I-II. Following Grogger (2012), we define attrition from the earnings sample as the

beginning of a terminal run of zero or missing earnings. We then apply Equation (1) to the

incidence of attrition 1-6 years following high school. Regardless of whether Knox Achieves is

measured in terms of eligibility (Panel I) or individual-level participation (Panel II), the effect is

similar: Knox Achieves is associated with a reduction of attrition of at least 2 percentage points.

However, the drop attrition is noticeably more pronounced when measuring individual-level

participation in Knox Achieves. Higher earnings shown in Figure 1 are in part a function of a

significantly higher likelihood of observing Knox County graduates and Knox Achieves

participation over time. It is not yet clear, however, if affected students are more likely to work at

all, or if they are less likely to leave the sample of Tennessee UI-covered workers for other

opportunities. In Equation (1) subgroup results not shown, we find that Knox Achieves is

associated with differentially less attrition among lower-achieving students, and more attrition

among low-income, higher-achieving students. This is consistent with negative selection into

observed earnings, which would tend to bias results for earnings inclusive of zero downward.

Recognizing that earnings in the decade after high school might not be indicative of long-

term returns, we estimate Equation (1) for each worker’s industrial score, equal to average US

earnings in their industry. Intent-to-treat effects depicted in Panel III of Figure 2 are small and

close to zero; when combined with results for other outcomes, this implies that eligibility is

linked to more work but not necessarily more lucrative industries. Knox Achieves participants,

21

however, exhibit higher industrial scores immediately after high school, and the gap grows to

over $3,000 by the 9th year after high school.

As shown in our earlier work, the message underlying free community college policies is

evidently a compelling factor that can sway students on the fence about going to college at all,

but it can also reroute students to begin their postsecondary education in a two-year college

rather than a four-year college (Carruthers and Fox, 2016). We found that while much of the

effect of Knox Achieves was to push extramarginal students into college, there was a discernible

negative effect on the likelihood of starting in a four-year college, and this substitution was more

pronounced for higher-income and higher-achieving students. Shifting preferences for where to

start college, combined with ambiguous findings for eventual bachelor’s degree receipt, speaks

to the long-term return to last-dollar programs like Knox Achieves because the earnings

premium that students receive from a bachelor’s degree is high relative to the return from

community college completion. From a social perspective, higher earnings from more college

completers will not necessarily offset losses from fewer four-year completers. In Section IV.D

we explore this tradeoff more formally, relying on estimates of lifecycle returns to education

rather than relatively short-run gains in UI-covered earnings.

Focusing on the intent-to-treat specification of Equation (1), where KAi is an indicator for

Knox Achieves availability, Table 4 partitions the Tennessee sample of 2007-2011 seniors into

three groups: those who did not participate in the free school lunch program between 8th and 12th

grades (Column 1), those who were eligible for free lunch for 1 or 2 years in that time (Column

2), and those who were eligible for at least 3 years (Column 3). Moving left to right, family

income likely falls and subsamples are successively more disadvantaged (Michelmore and

Dynarski, 2017). For ease of discussion, we label these three groups higher income, middle

income, and lower income. Notably, more disadvantaged students in last-dollar Promise

programs such as Knox Achieves tend to qualify for federal Pell grants at or exceeding

community college tuition. Any discernible treatment effect among these subpopulations may be

more attributable to non-financial aspects of Knox Achieves such as mentoring and the upfront

message about the cost of college for program participants.

The availability of Knox Achieves is associated with the most improvement in

postsecondary attainment for middle-income students (Column 2), compared with higher income

22

(Column 1) or lower-income (Column 3) students. For example, the increase in accumulated

credits is 4.2 (22%) for the middle income group, versus 1.9 (8%) and 1.5 (15%) for higher and

lower-income students, respectively. Similarly, the middle income group is the only one where

Knox Achieves is associated with an increase in the likelihood of completing a bachelor’s

degree, at 1.8 percentage points. Both the higher-income and middle-income groups have sizable

increases of roughly three percentage points for certificates, compared with an increase for

lower-income students of only 1.1 percentage. The lower-income group is associated with a

decrease in likelihood of receiving an associate’s degrees, in contrast to an increase of 1.3-1.4

percentage points for the other two income groups.

Why might higher-income students be as affected as needier peers, if not more affected,

by Knox Achieves support? The program was founded with the intent of helping disadvantaged

and first-generation college students, but its universal message and benefits do not exclude

students with higher baseline rates of college success. Students without subsidized lunch are not

necessarily affluent but will have less support from federal Pell grants. As such, they would

receive more in last-dollar aid from Knox Achieves, which could amplify the program’s intended

effects on 2-year college completion as well as its unintended effects on 4-year college

completion.

Table 5 summarizes results of Equation (1) by subgroups according to measures of

college readiness: expected persistence in college (Columns 1 and 3), which was estimated using

the same pre-program mapping as in the Table 2 balance tests, and average standardized end-of-

course achievement prior to high school exit (Columns 2 and 4). Columns 1-2 report results for

students above the median of each measure, and Columns 3-4 report results for students below

the median. Although results vary greatly across the columns, a general pattern is that estimated

effects on two-year college completion are larger in absolute and relative terms for higher-

achieving students, whereas four-year completion and 7th-year labor market gains are largest for

lower-achieving students. One inference is that higher-achieving students responded to tangible

last-dollar financial support in ways that led to more completion in targeted schools, but their

initial matriculation shift toward two-year schools set them on a path that led to certificates and

associate’s degrees at the (imprecisely estimated) expense of bachelor’s degrees. For lower-

achieving students, the decision of whether to go to college at all may have been more relevant

23

than the choice between starting in a community college versus a four-year school, and Columns

3-4 of Table 5 are consistent with the idea that the program’s effects on initial enrollment

translated to higher completion rates several years later. A 1-percentage-point effect on

bachelor’s degree attainment for lower-achieving students is a meaningful increase over 7%

baseline rates for this group (versus 31-32% for higher-achieving students).

Table 6 reports results for Equation (1) estimates by race, Hispanic ethnicity, and gender.

We show that that estimated effects on college credit accumulation and completion are larger for

black and Hispanic students, both in absolute terms and relative to sample averages. As seen

from control means below standard errors, bachelor’s attainment rates are typically 8-10

percentage points lower than white completion rates (11-13% versus 21%) for ineligible cohorts

in other counties, or in Knox County prior to the introduction of Knox Achieves. Intent-to-treat

estimates of 1.3 and 3.0 percentage points for black and Hispanic students, respectively, would

narrow that gap by 12-23%. College persistence and completion results are similar for men and

women (Columns 4-5), although estimated effects on earnings 7 years after high school are

entirely driven by men.

IV.C. Equation (1) robustness

Knox Achieves affected a small number of groups—18 schools and 1 county—and in an

intent-to-treat specification like Equation (1) when KAi is a binary indicator for 2009-11 Knox

seniors, this circumstance can result in standard errors that are too small, leading us to over-

reject the null and infer significant treatment effects where there might be none. Cluster-robust

standard errors such as the ones reported in Tables 3-6 are derived from variance estimators that

sum the pre/post residual difference from treated schools with the pre/post residual difference

from comparison schools, and by construction the former is very small with few treated groups

(Cameron & Taber, 2011; Ferman & Pinto, 2019).

In order to get a different perspective on the precision of our main intent-to-treat findings,

we re-estimate Equation (1) over 300 permutations where 18 schools from across the state are

randomly selected to be treated by a Knox Achieves placebo. Figure 3 plots the distribution of

placebo treatment effects for six outcomes, marking the Table 3 Column 1 point estimate for

each with a dashed vertical line. The percent of placebo treatment effects that are larger than the

24

corresponding Table 3 estimate in absolute value is listed in parentheses for each panel. As

expected, these shares are larger than p-values implied by Table 3 standard errors, and treatment

effect estimates that were imprecisely estimated in Equation (1) with cluster-robust standard

errors—credits accumulated within four years and bachelor’s degree attainment, are likewise

somewhat central in the distribution of placebo effects. Treatment effect estimates for credits

within two years, certificate or associate attainment, and earnings 7 years after high school are

atypical relative to placebos, however, indicating that our inferences about those outcomes thus

far are robust in spite of the tendency for estimates to stand out from the conditional mean when

a small number of groups are treated.

Permutations reported in Figure 3 do not account for heteroskedasticity with respect to

school size, i.e., the idea that random permutations would discount the importance of treatment

effects among larger groups, whose outcomes are less susceptible to noise and short-term

deviations than smaller groups (Ferman & Pinto, 2019). As one of the larger metro areas in the

state, Knox County high schools’ senior cohorts tend to be larger than the untreated average, and

so we expect that adjusting for heteroskedasticity would make attainment and earnings results in

Figure 3 stand out even more.

Looking to the other specification of Equation (1) reported in Table 3, where KAi is an

indicator flagging students who signed up to learn more about Knox Achieves, treatment-on-the-

treated effect estimates rely on a strong assumption that signup was conditionally independent of

influential and unobserved factors, such as inherent interest in associate programs or jobs that are

well-aligned with such programs. To make these inferences with confidence, we rely heavily on

the Xist vector of observable students and school features to control for factors that may influence

students’ interest in a free community college program as well as their later success in college.

Table 7 scrutinizes observable controls more closely and explores the extent to which non-zero

treatment effects are robust to more flexible specifications and/or a proportionate degree of

selection on unobservables. Here, the focus is on individual-level participation in Knox

Achieves.

Column 1 of Table 7 lists Equation (1) results when the Xist vector is limited to cohort

fixed effects. This is a nearly unconditional estimate of relative college credit and accumulation

gaps between Knox Achieves participants and other students. Knox Achieves participants were

25

academically similar to their non-participating peers in Knox County but socioeconomically

more disadvantaged (Table 1). Nevertheless, Table 7 shows that they accumulated 2-4 more

college credits than their cohorts at large, they were 10 percentage points more likely to attain a

two-year credential, they were 5 percentage points less likely to receive a bachelor’s degree, and

they were earning $1,699 more than their peers 7 years after high school.

These gaps narrow when we add the basic set of controls used in Tables 3-6, except for

participants’ shortfall in bachelor’s degree receipt, which widens from 5 to 7 percentage points

(Column 2). This moderate degree of attenuation combined with large gains in explained

variance leads to the inference that controls are informative and that a sizable degree of selection

on unobservables would be necessary to explain away Column 2 conditional gaps. Estimates of

Oster’s (2017) δ parameter are below R2 values in Columns 2-4. If the true treatment effect on

college credits earned within two years of high school is zero, selection into Knox Achieves

based on unobservable determinants of credit gains within two years would have to be 4.6 times

as informative as selection according to the basic Xist vector components. Likewise, selection on

unobservables would need to be more than 5 times as potent as selection on observables to

explain conditional gaps in two-year college certificate receipt, and 9 times as potent to explain

why Knox Achieves participants were 8 percentage points more likely to earn an associate’s

degree. The δ parameter is negative for bachelor’s degree receipt in Column 2 of Table 7, in part

because the negative estimated effect of program participation on bachelor’s degree attainment

widens from 5.3 to 6.9 percentage points when controls are added. That is, participants were ex

ante more likely to complete a bachelor’s degree than non-participants based on observable

features and their rate of bachelor’s degree completion fell short of expectations (we saw

evidence of this in Table 2 as well). Nonetheless, controls considerably increase explained

variance in this outcome, and it would presumably take a very large degree of unobservable

aversion to four-year degrees to attribute the negative estimated effect of Knox Achieves

participation to omitted variable bias.

Specifications reported in Column 3 of Table 7 expand the Xist vector from 30 elements to

over 800 by including quadratic functions of continuous variables and interactions between and

among all binary and continuous variables other than school or cohort fixed effects. Under this

more saturated and more flexible model, point estimates for college credit gains within two years

26

grow from 2.8 to 3.4, and the negative effect of participation on bachelor’s receipt shrinks from

6.9 to 5.4 percentage points. Results for other outcomes change very little. Even though we

added many more observable controls to the linear model, proportionate selection on

unobservables would still have to be quite large to completely explain treatment-on-the-treated

effect estimates.

Lastly, Column 4 reports results from a specification of Equation (1) with an intermediate

degree of flexibility relative to baseline and saturated models. Specifically, we estimate Equation

(1) using a set of controls identified by least absolute shrinkage and selection (LASSO). LASSO

is known for improving prediction accuracy more so than for interpreting particular parameters,

in part because shrunken coefficients are difficult to translate into policy predictions and more

importantly because LASSO parameters may be sensitive to imperfect feature selection (Hansen

et al. 2014). We follow the post-double-selection method prescribed by Hansen et al. (2014) to

recover more stable and interpretable Equation (1) estimates of γ from a large, data-driven set of

controls. Point estimates are little changed relative to the very flexible Column 3 specification,

although the penalty to bachelor’s degree receipt widens to 6.1 percentage points.

IV.C. Matching results

Treatment-on-the-treated estimates from Mahalanobis and propensity score matching

procedures are reported in Table 8. The primary takeaway is that treatment effect estimates are

similar under these two different matching schemes and—with some notable exceptions—very

similar to conditional differences from linear estimates of Equation (1) for the effect of Knox

Achieves participation. Columns 1 and 3 report matching results under Mahalanobis covariate-

based matching, where treated students are paired with untreated students with quantitatively

similar features. Columns 2 and 4 report treatment effects from propensity score matching,

where the counterfactual to a Knox Achieves participant is a student with nearly the same

propensity to engage with the program, but who did not participate because of choice or

availability.

Columns 1-2 of Table 8 include Knox County non-participants in the donor pool,

allowing participants to be matched to eligible students who chose not to participate. This

construct allows for demonstrably better matching on observables (shown in Figure A1) but may

27

exacerbate selection on unobservables by identifying treatment effects from participation as well

as willingness to engage with Knox Achieves when it was available. When the donor pool

consists of Tennessee 12th graders from the classes of 2007-2011 including Knox County non-

participants, we find that Knox Achieves students accumulate 3-4 more college credits within

two years of high school, and in a departure from linear results, up to 3 more credits within four

years of high school. Participant rates of certificate and associate’s degree completion are 8-10

percentage points higher than the counterfactual, in agreement with linear treatment effect

estimates. Bachelor’s degree completion is 3-6 percentage points lower among participants than

matched peers, a narrower gap than reported from most OLS models. Column 1-2 results for

earnings 7 years after high school are notably smaller than comparable linear findings for

participants: $829-1,014 in additional earnings per year, versus $1,457 reported in Table 3.

A plausible concern with the matching identification strategy is the idea that the choice

dimension of participating in Knox Achieves is endogenous to future college success.

Specifically, some program participants are paired with Knox County students who were eligible

for Knox Achieves but chose not to participate for unobserved reasons that may include interest

in community college. Results in Columns 3 and 4 report estimates from matching models that

remove eligible non-participants from the donor pool. Counterfactual outcomes are drawn

entirely from pre-program cohorts and Tennessee students outside of Knox County. Comparisons

of Column 1 with 3 and Column 2 with 4 imply that this distinction has little bearing on

inferences from matching-based identification, except for 7th year earnings, where there is

disagreement across matching methods and donor pools as to the size and precision of the gap

between Knox Achieves participants and other students. When we exclude Knox County non-

participants from the donor pool, that gap ranges from $184-1,263, favoring non-participants and

bookending Column 1-2 results where participants are more apt to be evaluated against their

non-participating Knox County peers.

IV.D. Benefit-cost comparison

Eligibility for or participation in Knox Achieves is associated with an increased

likelihood of receiving a certificate or associate’s degree, and potentially, a decreased likelihood

of receiving a bachelor’s degree. The returns to a four-year degree dominate returns to an

associate’s degree, and starting in a community college sets students up to face transition costs if

28

they choose to transfer to a four-year institution. Our findings for in-state earnings and industries

are limited to early-career years and potentially influenced by differential attrition and retention

in UI-covered Tennessee jobs. Therefore, an open question about the efficacy of last-dollar

scholarships that spotlight community colleges is whether the benefits from increased

community college completion offset the reduction in bachelor’s degree completion. In this

section, we explore how the known costs of the program measure up to its potential benefits in

terms of higher lifetime earnings for participants.

We note up front that the only benefit under consideration is changes in lifetime earnings

associated with higher levels of education. Our benefit-cost computations omit any non-financial

benefits of higher education, which are thought to be large and diverse, as well as the public cost

of subsidizing higher education for more students.

We calculate Tennessee-specific lifetime earnings by education level following

Blomquist et al. (2014), who apply coefficients from log earnings regressions using 2000 Census

data, arriving at the earnings premium for different education levels including associate’s

degrees. We use the same regression equation, but with 2012-2016 data for Tennessee from the

American Community Survey. This technique produces the following estimates of lifetime

earnings, each in 2016 dollars: $1,334,758 for a high school diploma, $1,875,023 for an

associate’s degree, and $2,318,398 for a bachelor’s degree.12 Because the ACS data do not

include certificates, we begin by assuming that a certificate returns lifetime earnings half-way

between a high school diploma and an associate’s degree. A certificate takes, on average, about

half as long as an associate’s degree to complete, and in a review of the literature on returns to

sub-baccalaureate attainment, Belfield and Bailey (2017) report that the short-term earnings

premium from certificate receipt is up to 46% of the return to an associate’s degree. Recognizing

widely varying returns to postsecondary certificates across studies, we also produce benefit-cost

estimates where the return to certificate attainment is (1) one-fourth of the gain (relative to high

school) of an associate’s degree, (2) one-eighth of the gain of an associate’s degree, and (3)

zero—that is, that certificates yield the same lifetime earnings as a high school degree.

12 Comparable national estimates from Carnevale et al. (2011) are $1,304,000 for a high school diploma, $1,727,000 for an associate’s degree, and $2,268,000 for a bachelor’s degree.

29

We compute the expected increase in earnings as the product of the increased likelihood

of obtaining a given award multiplied by the gain in lifetime earnings from that award relative to

high school. A further conversion is needed because the ACS measures education as the highest

degree obtained, and our estimates thus far are for the likelihood of receiving credentials at all.

Thus, we re-estimate linear models for the highest degree obtained within six years of high

school. For linear eligibility results, reported in the top panel of Table 9, we estimate the effect of

individual Knox Achieves eligibility in a model with school fixed effects, analogous to the

specification in Table 3, Column 1. Knox Achieves is associated with an increase of 0.789

percentage points in the likelihood of attaining an associate’s degree and no higher degree within

six years. Thus, the expected benefit from more associate’s degrees is:

E(benefit) = 0.000789 * (1,875,023 – 1,334,758) = 0.078 * 540,265 = 4,263

By a similar calculation, the expected per-participant benefit from higher likelihood of certificate

attainment is $1,353, assuming that a postsecondary certificate is worth half of an associate’s

degree. And the expected loss in earnings from a lower likelihood of attaining a bachelor’s

degree is -$4,997, meaning that the net change in expected earnings for Knox Achieves

participants is $619.

As mentioned earlier, 55 percent of senior-year participants enroll in a community

college with Knox Achieves, and the cost to the funder of this last-dollar scholarship is around

$1,000 per enrolled participant per year. We assume that this cost is measured in 2009 dollars,

the first full year of scholarship, an amount equal to $1,119 in 2016 dollars (as benefits are

measured in 2016 dollars). Using the calculation that 55 percent of Knox Achieves participants

receive the scholarship and enroll in community college, along with a maximum eligibility of

five semesters (or 2.5 years), and a participation rate of 16.8% among eligible students, we

calculate a potential cost of $1,119 * 0.55 * 2.5 * 0.168 = $258 per eligible student. For

participants only, the expected cost is $1,538.

Therefore, the benefit-cost ratio is $619/258 = 2.40 under the preferred, conservative

linear specification of attainment effects arising from eligibility. Note, however, that because of

the imprecision of eligibility effects on bachelor’s degrees, the benefit-cost ratio is much larger

at the upper limit of confidence intervals for certificate, associate’s, and bachelor attainment: 0.6,

30

1.0, and 0.3 percentage points, respectively. In that scenario, expected benefits per student are

over $10,000 in lifetime earnings, more than 40 times the per-eligible-student cost.

Moving down the top panel of benefit-cost estimates, we find that the sign of the net

return is also sensitive to assumptions about the value of postsecondary certificates. If these

credentials are worth no more than a high school diploma, the expected drop in lifetime earnings

per participant is 2.8 times the cost to the funder.13

Turning to the bottom panel of benefit estimates, we also find that results are very

different under estimated attainment effects from participation rather than eligibility. There,

Knox Achieves participation precedes certificate (and no higher) attainment 2.3 percentage

points higher than expected, associate attainment 5.5 percentage points higher, and bachelor’s

attainment of 7.2 percentage points lower, as reported in Table 3. In this model, returns to

certificates and associate degrees are dominated by estimated earnings penalties from lower

bachelor’s degree receipt, and the net expected change in earnings is at least 22 times larger than

the per-participant cost.

Looking across benefit-cost models, changes in expected lifetime earnings arising from

Knox Achieves participation are not conclusively positive or negative. Net benefits would be

positive and potentially large if Tennessee’s economy over the next several decades places an

atypically high premium on associate’s degrees and certificates, or more importantly, if true

effects on bachelor’s degree receipt are different than we estimate in the preferred, conservative

linear model (such as they are in some matching and less truncated analyses).

Another important caveat to recognize is that net benefits, either positive or negative,

come with notable distributional effects. Tables 4-6 report evidence that students with lower ex

ante college engagement are more likely to graduate with a bachelor’s degree than they would

otherwise, as are moderately low-income students, and non-white students who are

underrepresented among the state’s college graduates. Given low program costs, expected

benefits for these students are unambiguously positive. We do not observe program spending for

individual students, but benefit-cost computations would likely find a greater return for lower-

13 If certificates were equivalent to associate’s degrees in terms of the lifetime earnings premium over a high school degree, the benefit-cost ratio as computed in the top panel of Table 9 would be -0.66.

31

achieving students, as these participants tend to substitute fewer bachelor’s degrees for two-year

college awards with lower lifetime earnings. By contrast, we would expect larger negative

returns among higher-income and higher-achieving students through last-dollar programs like

Knox Achieves. These students collect more from the program in last-dollar aid and tend to

exhibit a larger absolute shortfall in bachelor’s degree completion six years after high school.

An additional caveat that bears repeating is that many potential costs and benefits are

omitted from this analysis. For example, the decision to attend community college or university

more than six years after high school is not included, nor are any delayed effects on bachelor’s

completion after six years. We ignore costs to the state government of providing additional

funding to community colleges as well as any individual-level costs, including nonmonetary

costs, not covered by the program. Nonmonetary benefits of higher education are also omitted.

Benefits of each level of attainment are assumed to be uniform, ruling out the possibility of

heterogeneous benefits that could correspond with student interest and aptitude for different

community college or university programs. Any benefits of multiple credentials (two distinct

certificates, for example, or a certificate and an associate’s degree) or graduate degrees are

omitted by our focus on highest undergraduate attainment. All the true costs and benefits of the

program are not known, and these additional considerations further cloud our expectations about

the net return to Knox Achieves participation.

V. Conclusion

In this study, we estimate the medium-term effects of a low-cost last dollar scholarship,

Knox Achieves, on college completion up to 6 years after high-school graduation as well as in-

state UI-covered earnings up to 9 years after high school. Key elements of Knox Achieves—

universal place-based eligibility and last-dollar support—match statewide programs that

followed in Tennessee and elsewhere throughout the United States. Estimated effects vary in

magnitude across different approaches, but a consistent story emerges where the introduction of

Knox Achieves led to more persistence and completion at two-year community colleges,

heterogeneous effects on success at four-year colleges and universities, and short-term gains in

employment and earnings alongside uncertain long-term returns.

32

On average, students eligible for Knox Achieves earned 1.6 more college credits within

two years of high school but the gap disappears after four. Modest gains in credit accumulation

nonetheless translate into large changes in the likelihood of completing a two-year college

program. Eligible students were 1-3 percentage points more likely to earn a postsecondary

certificate or associate’s degree, which represent sizable improvements over control means of 2-

4 percent, respectively. They were no more likely to complete a bachelor’s degree, however,

although college completion effects varied widely across student subgroups. Moderately low-

income students, lower-achieving students, and black and Hispanic students were significantly

more likely to complete a bachelor’s degree if they had access to free community college

through Knox Achieves, whereas more advantaged groups of students were less likely to finish a

bachelor’s degree, with varying degrees of precision. Apparent substitution for a two-year

college education over a four-year university education was most pronounced for higher-

achieving, higher-income students, who stood to gain more financial aid from the program. Less

advantaged students who benefitted from access to Knox Achieves may have been on the fence

about going to college at all, more so than starting at a two-year versus a four-year school.

More generous first-dollar scholarship programs that cover four-year schools as well as

community colleges have been shown to increase college attainment by much more than Knox

Achieves. The increase in the likelihood of receiving a bachelor’s degree is 7.4 percentage points

for the Kalamazoo Promise (Bartik et al., 2019) and 10.7 percentage points for the El Dorado

Promise (Swanson and Ritter, 2018). The costs of guaranteeing first-dollar aid, however, are

much higher than comparable costs from last-dollar Knox Achieves and its successor the

Tennessee Promise. Kalamazoo Promise spent $17,620 per eligible student (Bartik et al., 2016,

Table 2), versus $1,538 for Knox Achieves. Bartik et al. (2016) find that the likely benefits from

higher lifetime earnings, driven by gains in bachelor’s degree receipt, outweigh the costs 4.66

times (their Table 5). Page et al. (2018) estimate a benefit-cost ratio of Pittsburgh Promise equal

to 1.35 based on gains in short-term college enrolment and persistence. Benefit-cost ratios for

last-dollar Knox Achieves are far too inconclusive to evaluate net returns against these more

costly Promise models, and whether higher lifetime earnings outweigh modest program costs

will hinge on the extent to which students are deterred or delayed from completing a bachelor’s

degree, as well as future returns to certificates and associate degrees.

33

Our findings suggest that a low-cost, last-dollar promise of free community college can

raise community college attendance and completion for participants regardless of income or

aptitude, and can improve bachelor’s degree attainment for underrepresented students or those

who would not be expected to persist in college. At the same time, however, we find evidence

supporting the idea that raising the incentives to start college in the two-year rather than four-

year college sector lowers bachelor’s degree attainment for some students, and in particular,

higher-achieving, lowest-income students.

Uncertainty around highest degree attainment has economic implications for last-dollar

programs modeled after Knox Achieves, including Tennessee Promise and several subsequent

state initiatives, all of which share the premise that free college will support workforce

development in an advancing economy. In order to do so unambiguously, the message of free

college that motivates students to go to college at all should be paired with efforts to support

students with transfer intentions through bachelor’s degree completion.

In the years since the Knox Achieves study population completed high school, many

statewide efforts have evolved in parallel to the expansion of the Knox Achieves, including K-12

reforms stemming from Race to the Top and the introduction of the “Drive to 55” postsecondary

attainment goal. Also notably, a statewide system of transfer articulation agreements between

community colleges and four-year universities was formalized, which should in principle ease

student transitions between college sectors (Jenkins et al., 2018). It remains to be seen if such

transfer pathways are an effective complement to free community college programs in practice,

although our findings suggest that strong links between tertiary sectors will be critical to their

success.

34

References

Abadie, Alberto, and Guido W. Imbens. (2006). Large Sample Properties of Matching Estimators for Average Treatment Effects. Econometrica, 74(1), 235-267.

Altonji, Joseph. G., Todd E. Elder, and Christopher R. Taber. (2005). Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools. Journal of Political Economy, 113(1), 151-184.

Bartik, Timothy J., Brad Hershbein, and Marta Lachowska. (2019). The Effects of the Kalamazoo Promise Scholarship on College Enrollment, Persistence, and Completion. Forthcoming, Journal of Human Resources.

Bartik, Timothy J., Brad Hershbein, and Marta Lachowska. (2016). The Merits of Universal Scholarship: Benefit-Cost Evidence from the Kalamazoo Promise. Journal of Benefit Cost Analysis, 7(3), 400-433. https://doi.org/10.1017/bca.2016.22

Bartik, Timothy. J., and Nathan Sotherland. (2015). Migration and Housing Price Effects of Place-Based College Scholarships. Upjohn Institute Working Paper 15-245. Kalamazoo, MI: W.E. Upjohn Institute for Employment Research. http://dx.doi.org/10.17848/wp15-245

Belfield, Clive, and Thomas Bailey. (2017). The Labor Market Returns to Sub-Baccalaureate College: A Review. A CAPSEE Working Paper. Center for Analysis of Postsecondary Education and Employment.

Blomquist, Glenn, Paul A. Coomes, Christopher Jepsen, Brandon C. Koford, and Kenneth R. Troske. (2014) Estimating the Social Value of Higher Education: Willingness to Pay for Community and Technical Colleges. Journal of Benefit Cost Analysis, 5(1), 3-42.

Boatman, Angela and Adela Soliz. (2018). Statewide Transfer Policies and Community College Student Success. Education Finance and Policy, 13(4), 449-483.

Conley, T. G., & Taber, C. R. (2011). Inference with “difference in differences” with a small number of policy changes. The Review of Economics and Statistics, 93(1), 113-125.

Carnevale, Anthony P., Stephen J. Rose, and Ban Cheah. (2011). The College Payoff: Education,

Occupations, Lifetime Earnings. Center on Education and the Workforce, Georgetown Public Policy Institute, Georgetown University.

Carnevale, Anthony P., and Nicole Smith. (2012). A Decade Behind: Breaking Out of the Low-Skill Trap in the Southern Economy. Center on Education and the Workforce, Georgetown Public Policy Institute, Georgetown University.

Carruthers, Celeste. K., and William F. Fox. (2016). Aid for All: College Coaching, Financial Aid, and Post-Secondary Persistence in Tennessee. Economics of Education Review, 51, 97-112.

35

Chaisemartin, de C., & d'Haultfoeuille, X. (2019). Two-way fixed effects estimators with heterogeneous treatment effects (No. w25904). National Bureau of Economic Research.

Chaisemartin, de C., X. D'Haultfoeuille, and A. Deeb. (2019). “TWOWAYFEWEIGHTS: Stata

module to estimate the weights and measure of robustness to treatment effect heterogeneity attached to two-way fixed effects regressions,” Statistical Software Components S458611, Boston College Department of Economics, revised 03 Nov 2019.

Daugherty, Lindsay, and Gabriella C. Gonzalez. (2016). The Impact of the New Haven Promise

Program on College Enrollment, Choice, and Persistence. RAND Corporation Working Paper.

Delaney, Jennifer A., and Bradley Hemenway. (2016). A Difference-in-Difference Analysis of “Promise” Financial Aid Programs on Postsecondary Institutions. Working Paper.

Ferman, B., & Pinto, C. (2019). Inference in differences-in-differences with few treated groups and heteroskedasticity. Review of Economics and Statistics, 101(3), 452-467.

Fuller, Matthew B. (2014). A History of Financial Aid to Students. Journal of Student Financial

Aid, 44(1), 4.

Grogger, Jeffrey. (2012). Bounding the effects of social experiments: Accounting for attrition in administrative data. Evaluation review, 36(6), 449-474.

Hansen, Christian, Victor Chernozhukov, and Alexandre Belloni. (2014). Inference on Treatment

Effects after Selection among High-Dimensional Controls. The Review of Economic Studies, 81(2), 608-650.

Imbens, Guido W. (2015). Matching Methods in Practice: Three Examples. Journal of Human Resources, 50(2), 373-419.

Jenkins, Davis, Amy E. Brown, John Fink, Hana Lahr, Takeshi Yanagiura. (2018). Building Guided Pathways to Community College Student Success: Promising Practices and Early Evidence from Tennessee. Community College Research Center, Columbia University Teachers College.

Jepsen, Christopher, Kenneth Troske, and Paul Coomes. (2014). The Labor-Market Returns to Community College Degrees, Diplomas, and Certificates. Journal of Labor Economics 32(1), 95-121.

Jones, Tiffany, and Katie Berger. (2018). A Promise Fulfilled: A Framework for Equitable Free College Programs. The Education Trust. Washington, D.C. https://edtrust.org/resource/a-promise-fulfilled/

LeGower, Michael, Randall Walsh. (2017). Promise scholarship programs as place-making policy: Evidence from school enrollment and housing prices. Journal of Urban Economics, 101, 74-89.

36

Mendoza, Pilar, and Jesse P. Mendez. (2012). The Oklahoma’s Promise Program: A National Model to Promote College Persistence. Journal of College Student Retention 14(3): 397-421.

Michelmore, Katherine, and Susan Dynarski. (2017). The Gap within the Gap: Using Longitudinal Data to Understand Income Differences in Educational Outcomes. AERA Open 3(1): https://doi.org/10.1177/2332858417692958

Musu-Gillette, Lauren, Cristobal de Brey, Joel McFarland, William Hussar, William Sonnenberg, and Sidney Wilkinson-Flicker, S. (2017). Status and Trends in the Education of Racial and Ethnic Groups 2017 (NCES 2017-051). U.S. Department of Education, National Center for Education Statistics. Washington, DC.

Murray, M. N. (2013). Fall 2013 Tennessee Business and Economic Outlook. Center for Business and Economic Research, University of Tennessee. https://haslam.utk.edu/sites/default/files/fall13.pdf

Oster, Emily. (2017). Unobservable Selection and Coefficient Stability: Theory and Validation. Journal of Business and Economic Statistics.

Page, Lindsay C., Jennifer E. Iriti, Danielle J. Lowry, and Aaron M. Anthony. (2018). The Promise of Place-Based Investment in Postsecondary Access and Success: Investigating the Impact of the Pittsburgh Promise. Education Finance and Policy, (Just Accepted), 1-60.

Stevens, Ann Huff, Michal Kurlaender, and Michel Grosz. (forthcoming). Career Technical Education and Labor Market Outcomes: Evidence from California Community Colleges. Journal of Human Resources.

Swanson, Elise, and Gary W. Ritter. (2018). Start to Finish: Examining the Impact of the El Dorado Promise on Postsecondary Outcomes. EDRE Working Paper Series, No. 2018-02

Swanson, Elise, Angela Watson, Gary W. Ritter, and Malachi Nichols. (2016). Promises Fulfilled? A Systematic Review of the Impacts of Promise Programs. University of Arkansas, Department of Education Reform Working Paper No. 2016-16.

Wilkinson, Rupert. (2005). Aiding Students, Buying Students: Financial Aid in America. Vanderbilt University Press.

Zipp, Yvonne. (2005). No Money for College? One Town's Reply. Christian Science Monitor. (November 16, 2005, Wednesday).

37

Table 1. Student summary statistics, by Knox Achieves participation (1) (2) (3)

Knox County Knox County

All Tennessee Knox Achieves non- Variable 12th Graders participants participants Seamlessly enrolled in college (0,1) 0.48 0.75 0.49 Attained a postsecondary certificate within 4 years (0,1) 0.02 0.12 0.03 Attained an associate's degree within 4 years (0,1) 0.04 0.15 0.05 Attained a bachelor's degree within 6 years (0,1) 0.19 0.17 0.25 In-state earnings one year after high school (000s) 4.47 5.44 4.60

(6.02) (5.57) (6.22) In-state earnings seven years after high school (000s) 15.96 17.84 14.95

(18.11) (16.01) (18.24) Female (0,1) 0.5 0.61 0.47 Hispanic (0,1) 0.23 0.22 0.14 Black (0,1) 0.03 0.02 0.02 White, non-Hispanic (0,1) 72.36 74.68 81.34 Free lunch for at least three years, 8th-12th grade (0,1) 17.15 31.44 13.1 Any free lunch, 8th-12th grade (0,1) 38.08 47.56 31.53 Any reduced-price lunch, 8th-12th grade (0,1) 0.12 0.2 0.11 ACT composite 17.36 18.17 17.11

(3.66) (4.00) (3.33) Average standardized EOC 0.06 0.18 0.28

(0.83) (0.66) (0.97) N (students) 307,798 2,109 18,105

Notes: Standard deviations are in parentheses below continuous-valued means. The sample includes 12th grade students throughout Tennessee, academic years 2006-2007 through 2010-2011. Other variables included in the analysis: the number of schools, districts, and counties a student attended since 8th grade, junior and senior year earnings, ACT subscores, EOC subject scores, an indicator for having repeated the 12th grade, indicators for cohort, indicators for missing ACT and EOC scores, and annual county unemployment rates. For college outcomes, samples exclude students who enrolled in non-THEC institutions.

38

Table 2: Balancing tests of ex ante expectations by Knox Achieves availability or participation (1) (2) (3) (4)

Measure of treatment Post-2008

Knox County

Individual KA

participation indicator

Post-2008 Knox

County

Individual KA

participation indicator

County fixed effects Yes Yes No No School fixed effects No No Yes Yes

Fitted THEC college credits within years 2.80*** 2.02 1.60 1.12

(0.85) (1.76) (1.01) (2.01)

Fitted THEC college credits within 4 years 6.48*** 3.95 3.36 2.05

(1.76) (3.64) (2.09) (4.22)

Fitted certificate attainment within 4 years -0.001** -2.0E-04 0.003*** 0.001

(0.000) (0.001) (0.001) (0.001)

Fitted associate's degree attainment within 4 years -0.006*** -0.001 1.00E-03 0.003

(0.002) (0.004) (0.002) (0.004)

Fitted bachelor's degree attainment within 6 years 0.053*** 0.024 0.024 0.010

(0.013) (0.027) (0.015) (0.030)

Fitted earnings 7 years after high school 45.20 723.91 173.47 705.32

(233.76) (596.74) (276.81) (587.23)

Students 307,727 307,727 307,727 307,727

Notes: The table lists γ estimates from Equation (1), where observed outcomes are substituted with fitted predictions from pre-program estimates (classes of 2007 and 2008). Outcomes are listed at left. Columns 1 and 3 reports results from a version of Equation 1 where the Knox Achieves treatment is a simple indicator for seniors in Knox County schools after the program was initiated. Columns 2 and 4 report results from a specification of Equation 1 where treatment status is assigned to Knox County seniors who signed up for Knox Achieves in the fall of their senior year. Standard errors are in parentheses below each coefficient.

*** significant at 1 percent, ** significant at 5 percent, * significant at 10 percent

39

Table 3: Average intent-to-treat and treatment-on-the-treated effect estimates

(1) (2)

Measure of treatment Post-2008 Knox County

Individual KA participation

indicator School fixed effects Yes Yes

THEC college credits within 2 years 1.62*** 2.48*** Control mean: 17.75 (0.42) (0.18)

THEC college credits within 4 years 0.76 -0.54* Control mean: 32.22 (0.61) (0.31) Postsecondary certificate attainment within 4 years 0.027*** 0.080*** Control mean: 0.016 (0.002) (0.002)

Postsecondary associate's degree attainment within 4 years 0.010*** 0.090*** Control mean: 0.036 (0.001) (0.002)

Postsecondary bachelor's degree attainment within 6 years -0.005 -0.072*** Control mean: 0.190 (0.004) (0.003)

In-state earnings 7 years after high school 732.70*** 1456.49*** Control mean: 15,976 (146.28) (73.21)

Treated students 12,343 2,038 All students 307,727 307,727

Notes: The table lists γ estimates from Equation 1, ITT and ATT estimates of the effect of Knox Achieves availability or participation on college persistence, attainment, and earnings. Outcomes are listed at left. Column 1 reports results from a version of Equation 1 where the Knox Achieves treatment is a simple indicator for seniors in Knox County schools after the program was initiated. Column 2 reports results from a specification of Equation 1 where treatment status is assigned to Knox County seniors who signed up for Knox Achieves in the fall of their senior year. Additional controls include cohort fixed effects, school fixed effects, observable student characteristics, school composition, and the county unemployment rate. Samples include students with no record of college enrollment, whose postsecondary outcomes are imputed to be zero. The top 5% of program participants, in terms of credit accumulation, are excluded. Robust standard errors, in parentheses below point estimates, allow for clustering by county.

*** significant at 1 percent, ** significant at 5 percent, * significant at 10 percent

40

Figure 3: ITT and ATT effect estimates on earnings, by years since high school In-state earnings 1-6 years after high school (including zero if missing)

I. Estimated effect of Knox Achieves availability II. Estimated effect of Knox Achieves participation

Any in-state earnings 1-9 years after high school (0,1) III. Estimated effect of Knox Achieves availability IV. Estimated effect of Knox Achieves participation

Log in-state earnings 1-9 years after high school (if non-missing) V. Estimated effect of Knox Achieves availability VI. Estimated effect of Knox Achieves participation

Figures display Equation (1) point estimates and confidence intervals for ITT estimated effects of Knox Achieves availability (panels I, III, V) and participation (II, IV, VI).

41

Figure 4: ITT and ATT effect estimates on attrition and industry, by years since high school Attrition from in-state earnings 1-6 years after high school (0,1)

I. Estimated effects of Knox Achieves availability II. Estimated effects of Knox Achieves participation

Industry score 1-9 years after high school (excluding zero and missing) III. Estimated effects of Knox Achieves availability IV. Estimated effects of Knox Achieves participation

Figures display Equation (1) point estimates and confidence intervals for ITT estimated effects of Knox Achieves availability (panels I, III) and participation (II, IV).

42

Table 4: Average intent-to-treat effect estimates, by free lunch status (1) (2) (3)

Measure of treatment Post-2008

Knox County

Post-2008 Knox

County

Post-2008 Knox

County Years eligible for free lunch, 8th-12th grade None 1-2 3+

THEC college credits within 2 years 1.88*** 2.71*** 1.47***

(0.53) (0.42) (0.41) 22.34 11.15 9.21

THEC college credits within 4 years 0.93 4.22*** 1.43*** (0.84) (0.56) (0.44) 41.27 19.34 15.17

Postsecondary certificate attainment within 4 years 0.033*** 0.025*** 0.011***

(0.003) (0.002) (0.001)

0.020 0.013 0.009

Postsecondary associate's degree attainment within 4 years 0.013*** 0.014*** -0.011***

(0.002) (0.002) (0.002)

0.046 0.023 0.014

Postsecondary bachelor's degree attainment within 6 years -0.004 0.018*** -0.011***

(0.006) (0.003) (0.004) 0.254 0.098 0.070

In-state earnings 7 years after high school 149.22*** 129.91* 194.51

(42.32) (78.03) (136.40)

4,602.40 4,538.00 3,892.97

Treated students 7,619 1,799 2,925 All students 190,543 64,424 52,760

Notes: The table lists γ estimates from Equation 1, ITT estimates of the effect of Knox Achieves availability or participation on college persistence, attainment, and earnings. Robust standard errors, in parentheses below point estimates, allow for clustering by county. Control means are listed below standard errors.

*** significant at 1 percent, ** significant at 5 percent, * significant at 10 percent

43

Table 5: Average intent-to-treat effect estimates, by achievement and ex ante persistence (1) (2) (3) (4)

Above median Below median

Expected college credits

End-of-course

achievement

Expected college credits

End-of-course

achievement

THEC college credits within 2 years 2.48*** 1.50*** 1.16*** 1.64*** (0.62) (0.53) (0.38) (0.41) 27.57 26.79 8.18 8.86

THEC college credits within 4 years 1.13 0.015 1.90*** 2.49*** (1.18) (0.94) (0.50) (0.55) 50.50 49.51 14.38 15.20

Postsecondary certificate attainment within 4 years 0.0410*** 0.037*** 0.002** 0.011***

(0.003) (0.003) (0.001) (0.001)

0.024 0.022 0.009 0.011

Postsecondary associate's degree attainment within 4 years 0.015*** 0.014*** (0.003) 0.004***

(0.002) (0.002) (0.002) (0.001)

0.055 0.051 0.018 0.021

Postsecondary bachelor's degree attainment within 6 years -0.005 -0.010 0.009** 0.010***

(0.008) (0.007) (0.004) (0.004)

0.316 0.312 0.067 0.071

In-state earnings 7 years after high school 301.19* 137.56 1017.49*** 1180.20***

(180.22) (164.82) (187.93) (165.00)

18,240 182,70 13,770 13,720

Treated students 7,951 7,303 4,392 5,040 All students 153,832 153,840 153,895 153,887

Notes: The table lists γ estimates from Equation 1, ITT estimates of the effect of Knox Achieves availability on college persistence, attainment, and earnings. Robust standard errors, in parentheses below point estimates, allow for clustering by county. Control means are listed below standard errors

*** significant at 1 percent, ** significant at 5 percent, * significant at 10 percent

44

Table 6: Average intent-to-treat effect estimates, by race, Hispanic ethnicity, and gender (1) (2) (3) (4) (5) Subsample Black Hispanic White Women Men

THEC college credits within 2 years 2.14*** 5.46*** 1.62*** 1.47*** 1.83***

(0.37) (0.52) (0.49) (0.46) (0.39) 13.40 10.48 19.29 20.71 14.79

THEC college credits within 4 years 2.85*** 4.54*** 0.89 0.53 1.14*

(0.45) (0.97) (0.79) (0.67) (0.61) 24.16 19.01 35.01 37.47 26.96

Postsecondary certificate attainment within 4 years 0.004*** 0.036*** 0.030*** 0.024*** 0.029***

(0.001) (0.003) (0.002) (0.002) (0.002) 0.005 0.010 0.021 0.018 0.015

Postsecondary associate's degree attainment within 4 years

0.010*** -0.019*** 0.012*** 0.009*** 0.011*** (0.002) (0.005) (0.002) (0.002) (0.001)

0.010 0.021 0.046 0.046 0.026

Postsecondary bachelor's degree attainment within 6 years

0.013*** 0.030*** -0.003 -0.002 -0.006 (0.004) (0.009) (0.005) (0.006) (0.004)

0.131 0.114 0.210 0.221 0.159

In-state earnings 7 years after high school 757.03*** 1,644.06*** 823.75*** -8.72 1,476.47*** (175.39) (430.01) (182.83) (150.86) (191.75)

13,940 11,690 16,900 14,620 17,330

Treated students 1,952 339 9,826 5,980 6,363 All students 72,116 9,463 222,663 153,838 153,889

Notes: The table lists γ estimates from Equation 1, ITT estimates of the effect of Knox Achieves availability on college persistence, attainment, and labor market measures. Robust standard errors, in parentheses below point estimates, allow for clustering by county. Control means are listed below standard errors. *** significant at 1 percent, ** significant at 5 percent, * significant at 10 percent

45

Figure 3: Permutation results for Equation (1): ITT effects of Knox Achieves versus placebos I. College credits within two years (9.7% greater) II. College credits within four years (66.7% greater)

III. Certificate attainment (0.0% greater) IV. Associate degree attainment (2.0% greater)

V. Bachelor's degree attainment (64% greater) VI. In-state earnings seven years after high school (6.0% greater)

Each panel illustrates the distribution of 300 iterations of Equation (1) results for Knox Achieves availability, with treated schools randomly reassigned across the state. Estimates of true ITT effects are marked with vertical dashed lines. The share of placebo estimates that are larger, in absolute value, than ITT estimates are listed in parentheses with each panel heading.

46

Table 7: Robustness of average treatment on the treated effect estimates (1) (2) (3) (4)

Measure of treatment

Individual KA

participation indicator

Individual KA

participation indicator

Individual KA

participation indicator

Individual KA

participation indicator

Controls Cohort fixed effects Basic Flexible

Selected features among

flexible set

THEC college credits within 2 years 4.16*** 2.78*** 3.38*** 2.92*** (0.43) (0.53) (0.53) (0.53)

Oster delta estimate 4.60 6.70 5.05 R-squared <0.01 0.27 0.31 0.28

THEC college credits within 4 years 1.96** -0.45 1.35 0.77

(0.89) (0.98) (0.98) (0.97) Oster delta estimate -0.56 4.34 1.69 R-squared <0.01 0.27 0.31 0.29 Postsecondary certificate attainment within 4 years 0.098*** 0.077*** 0.075*** 0.074***

(0.002) (0.003) (0.003) (0.003) Oster delta estimate 5.37 4.76 4.87 R-squared <0.01 0.03 0.03 0.03

Postsecondary associate's degree attainment within 4 years

0.097*** 0.084*** 0.084*** 0.081*** (0.005) (0.005) (0.005) (0.005)

Oster delta estimate

8.94 7.60 7.78 R-squared <0.01 0.04 0.05 0.04

Postsecondary bachelor's degree attainment within 6 years

-0.053*** -0.069*** -0.054*** -0.061*** (0.008) (0.009) (0.009) (0.009)

Oster delta estimate

-27.58 17.12 -388.43 R-squared <0.01 0.19 0.22 0.20 In-state earnings 7 years after high school 1,699*** 1,352*** 1,129*** 1,196

(422) (424) (433) (425) Oster delta estimate

7.50 4.02 5.12

R-squared <0.01 0.100 0.111 0.101

Treated students 1,917 1,917 1,917 1,917 All students 300,114 300,114 300,114 300,114 Notes: The table lists γ estimates from Equation 1, ATT estimates of the effect of Knox Achieves availability or participation on college persistence and attainment measures. Robust standard errors, in parentheses below point estimates, allow for clustering by county. *** significant at 1 percent, ** significant at 5 percent, * significant at 10 percent

47

Table 8. Matching results: Knox Achieves participation and postsecondary outcomes (1) (2) (3) (4) Control reservoir includes Knox County non-participants Yes Yes No No

Matching method Mahalanobis Propensity score Mahalanobis Propensity

score THEC college credits within 2 years 3.04 4.42 4.00 4.64

(0.67) (0.79) (0.71) (0.79)

THEC college credits within 4 years 0.79 2.88 2.44 2.96 (1.24) (1.42) (1.30) (1.42)

Postsecondary certificate attainment within 4 years 0.076 0.100 0.089 0.100

(0.009) (0.008) (0.008) (0.008)

Postsecondary associate's degree attainment within 4 years 0.081 0.089 0.087 0.087

(0.010) (0.010) (0.009) (0.010)

Postsecondary bachelor's degree attainment within 6 years -0.061 -0.028 -0.045 -0.029

(0.012) (0.012) (0.012) (0.012)

In-state earnings 7 years after high school 829.47 1013.69 183.97 1263.40 (534.96) (568.06) (550.01) (568.05)

All students on support 2,038 2,038 2,038 2,038 Treated students on support 307,183 304,643 296,859 294,422 Notes: The table lists Mahalanobis and propensity score matching results for postsecondary outcomes (average treatment on the treated, or ATT). We use a nearest-neighbor matching estimator, matching each Knox Achieves participant to one non-participant, with replacement in terms of the Mahalanobis distance metric or the propensty score. Standard errors for Mahalanobis matching are computed according to Abadie and Imbens (2006). Propensity score estimates accommodate a 2 percentage-point caliper of participation propensity and trim the top one percent of Knox Achieves participants in terms of participation propensity.

48

Table 9. Estimated Benefits and Costs of Knox Achieves Eligibility and Participation

Expected Change in Lifetime Earnings (in

thousands) Benefit

Certificate Associate's Bachelor's Total Cost

Ratio Linear estimates: Six-year highest attainment 0.0050 0.0079 -0.0051 (Table 3, Column 1 specification) (0.0007) (0.0010) (0.0043)

Certificate Benefit = 1/2 Associate's $1,353 $4,263 -$4,997 $619 2.40 Certificate Benefit = 1/4 Associate's $677 $4,263 -$4,997 -$58 -0.22 Certificate Benefit = 1/8 Associate's $338 $4,263 -$4,997 -$396 -1.53 Certificate Benefit = 0 $0 $4,263 -$4,997 -$734 -2.84

Linear estimates: Six-year highest attainment 0.0234 0.0550 -0.0721 (Table 3, Colunm 2 specification) (0.0008) (0.0010) (0.0027) Certificate Benefit = 1/2 Associate's $6,327 $29,715 -$70,881 -$34,840 -22.64 Certificate Benefit = 1/4 Associate's $3,163 $29,715 -$70,881 -$38,003 -24.70 Certificate Benefit = 1/8 Associate's $1,582 $29,715 -$70,881 -$39,585 -25.73 Certificate Benefit = 0 $0 $29,715 -$70,881 -$41,167 -26.76

Notes: The table reports estimates of benefits to program participation in terms of lifetime earnings. In the top panel, linear estimates replicate the Table 3, Colum 1 OLS model, but for highest certificate or degree attainment within 6 years of high school. Robust standard errors, in parentheses below point estimates, allow for clustering by county. The bottom panel reports highest attainment estimates arising from the participation model (Table 3, Column 2 specification). Costs of program participation are assumed to be $1,538 in the bottom panel, roughly the value of average program spending per student-year, in 2016 dollars, if each student attended for the maximum 5 semesters, or 2.5 years. Costs of eligibility are $1,538*0.168 = $257, which accounts for 16.8% participation rates among eligible students. Benefits are Tennessee-specific estimated lifetime earnings returns over a high school degree derived from the 2012-2016 ACS (Blomquist et al., 2014).

49

Appendix: Supplementary Figures and Results

Figure A1. Unmatched and matched balance in student and school observable features

Notes: The figure depicts standardized differences in observed student and school characteristics within unmatched (diamonds) and matched (x markers) samples. Treatment and control observations were matched by minimizing Mahalanobis distance metrics between vectors of these control variables.

50

Figure A2. Knox Achieves participation propensity

Notes: The figure depicts kernel densities of the estimated propensity to participate in Knox Achieves for actual participants (solid line) and non-participants (dashed line). Propensities were estimated by logit for Knox County 12th graders in the classes of 2009-2011. Factors included in the logit model included student characteristics (measures of demographics and need), student achievement on end-of-course and ACT exams, and school composition (including, among other controls, the percent white and the percent ever qualifying for free lunch).