do parents value school effectiveness?crwalters/papers/apsw_nyc.pdfdo parents value school...

59
American Economic Review 2020, 110(5): 1502–1539 https://doi.org/10.1257/aer.20172040 1502 Do Parents Value School Effectiveness? By Atila Abdulkadiro ˇ glu, Parag A. Pathak, Jonathan Schellenberg, and Christopher R. Walters* School choice may lead to improvements in school productivity if parents’ choices reward effective schools and punish ineffective ones. This mechanism requires parents to choose schools based on causal effectiveness rather than peer characteristics. We study relation- ships among parent preferences, peer quality, and causal effects on outcomes for applicants to New York City’s centralized high school assignment mechanism. We use applicants’ rank-ordered choice lists to measure preferences and to construct selection-corrected estimates of treatment effects on test scores, high school gradua- tion, college attendance, and college quality. Parents prefer schools that enroll high-achieving peers, and these schools generate larger improvements in short- and long-run student outcomes. Preferences are unrelated to school effectiveness and academic match quality after controlling for peer quality. (JEL D12, H75, I21, I26, I28) Recent education reforms in the United States, including charter schools, school vouchers, and district-wide open enrollment plans, increase parents’ power to choose schools for their children. School choice allows households to avoid undesir- able schools and forces schools to satisfy parents’ preferences or risk losing enroll- ment. Proponents of choice argue that this competitive pressure is likely to generate system-wide increases in school productivity and boost educational outcomes for students (Friedman 1962, Chubb and Moe 1990, Hoxby 2003). By decentralizing school quality assessment and allowing parents to act on local information, school choice is conjectured to provide better incentives for educational effectiveness than * Abdulkadiroˇ glu: Duke University, and NBER (email: [email protected]); Pathak: Massachusetts Institute of Technology, and NBER (email: [email protected]); Schellenberg: University of California, Berkeley (email: [email protected]); Walters: University of California, Berkeley, and NBER (email: crwalters@ econ.berkeley.edu). Thomas Lemieux was the coeditor for this article. We gratefully acknowledge funding from the National Science Foundation and the W. T. Grant Foundation. We also thank Nikhil Agarwal, Josh Angrist, Stephane Bonhomme, David Card, David Chan, Michael Dinerstein, Will Dobbie, James Heckman, Peter Hull, Pat Kline, Thibaut Lamadon, Magne Mogstad, Jack Mountjoy, Derek Neal, Ariel Pakes, Stephen Raudenbush, Jesse Rothstein, Alex Torgovitsky, Miguel Urquiola, Jeffrey Wooldridge, Danny Yagan, Seth Zimmerman, and seminar participants at the University of Chicago Committee on Education Workshop, the University of Chicago Harris School, UC Berkeley, the University of Chicago Econometrics Lunch, the University of Chicago Interactions Conference, UW Madison, the 2017 NBER Labor Studies fall meetings, the University of Virginia, the University of Hawaii, the University of Rochester, Princeton University, the University of Maryland, and Arizona State University for suggestions and comments. We’re grateful to Ray Han and Kate Bradley for outstanding research assistance, and to Eryn Heying for invaluable administrative support. Go to https://doi.org/10.1257/aer.20172040 to visit the article page for additional materials and author disclosure statements.

Upload: others

Post on 24-Jun-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

American Economic Review 2020, 110(5): 1502–1539 https://doi.org/10.1257/aer.20172040

1502

Do Parents Value School Effectiveness?†

By Atila Abdulkadiroglu, Parag A. Pathak, Jonathan Schellenberg, and Christopher R. Walters*

School choice may lead to improvements in school productivity if parents’ choices reward effective schools and punish ineffective ones. This mechanism requires parents to choose schools based on causal effectiveness rather than peer characteristics. We study relation-ships among parent preferences, peer quality, and causal effects on outcomes for applicants to New York City’s centralized high school assignment mechanism. We use applicants’ rank-ordered choice lists to measure preferences and to construct selection-corrected estimates of treatment effects on test scores, high school gradua-tion, college attendance, and college quality. Parents prefer schools that enroll high-achieving peers, and these schools generate larger improvements in short- and long-run student outcomes. Preferences are unrelated to school effectiveness and academic match quality after controlling for peer quality. (JEL D12, H75, I21, I26, I28)

Recent education reforms in the United States, including charter schools, school vouchers, and district-wide open enrollment plans, increase parents’ power to choose schools for their children. School choice allows households to avoid undesir-able schools and forces schools to satisfy parents’ preferences or risk losing enroll-ment. Proponents of choice argue that this competitive pressure is likely to generate system-wide increases in school productivity and boost educational outcomes for students (Friedman 1962, Chubb and Moe 1990, Hoxby 2003). By decentralizing school quality assessment and allowing parents to act on local information, school choice is conjectured to provide better incentives for educational effectiveness than

* Abdulkadiroglu: Duke University, and NBER (email: [email protected]); Pathak: Massachusetts Institute of Technology, and NBER (email: [email protected]); Schellenberg: University of California, Berkeley (email: [email protected]); Walters: University of California, Berkeley, and NBER (email: [email protected]). Thomas Lemieux was the coeditor for this article. We gratefully acknowledge funding from the National Science Foundation and the W. T. Grant Foundation. We also thank Nikhil Agarwal, Josh Angrist, Stephane Bonhomme, David Card, David Chan, Michael Dinerstein, Will Dobbie, James Heckman, Peter Hull, Pat Kline, Thibaut Lamadon, Magne Mogstad, Jack Mountjoy, Derek Neal, Ariel Pakes, Stephen Raudenbush, Jesse Rothstein, Alex Torgovitsky, Miguel Urquiola, Jeffrey Wooldridge, Danny Yagan, Seth Zimmerman, and seminar participants at the University of Chicago Committee on Education Workshop, the University of Chicago Harris School, UC Berkeley, the University of Chicago Econometrics Lunch, the University of Chicago Interactions Conference, UW Madison, the 2017 NBER Labor Studies fall meetings, the University of Virginia, the University of Hawaii, the University of Rochester, Princeton University, the University of Maryland, and Arizona State University for suggestions and comments. We’re grateful to Ray Han and Kate Bradley for outstanding research assistance, and to Eryn Heying for invaluable administrative support.

† Go to https://doi.org/10.1257/aer.20172040 to visit the article page for additional materials and author disclosure statements.

1503ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

could be achieved by a centralized accountability system (Peterson and Campbell 2001). Choice may also improve outcomes by allowing students to sort into schools that suit their particular educational needs, resulting in improved match quality (Hoxby 2000). These arguments have motivated recent policy efforts to expand school choice (e.g., DeVos 2017).

If choice is to improve educational effectiveness, parents’ choices must result in rewards for effective schools and sanctions for ineffective ones. Our use of the term “effective” follows Rothstein (2006): an effective school is one that generates causal improvements in student outcomes. Choice need not improve school effectiveness if it is not the basis for how parents choose between schools. For example, parents may value attributes such as facilities, convenience, student satisfaction, or peer composition in a manner that does not align with educational impacts (Hanushek 1981, Jacob and Lefgren 2007). Moreover, while models in which parents value schools according to their effectiveness are an important benchmark in the academic literature (e.g., Epple, Figlio, and Romano 2004), it may be difficult for parents to separate a school’s effectiveness from the composition of its student body (Kane and Staiger 2002). If parent choices reward schools that recruit higher-achieving students rather than schools that improve outcomes, school choice may increase resources devoted to screening and selection rather than better instruction (Ladd 2002, MacLeod and Urquiola 2015). Consistent with these possibilities, Rothstein (2006) shows that cross-district relationships among school choice, sorting patterns, and student outcomes fail to match the predictions of a model in which school effec-tiveness is the primary determinant of parent preferences.

This paper offers new evidence on the links between preferences, school effec-tiveness, and peer quality based on choice and outcome data for more than 250,000 applicants in New York City’s centralized high school assignment mechanism. Each year, thousands of New York City high school applicants rank-order schools, and the mechanism assigns students to schools using the deferred acceptance (DA) algo-rithm (Gale and Shapley 1962; Abdulkadiroglu, Pathak, and Roth 2005). The DA mechanism is strategy-proof: truthfully ranking schools is a weakly dominant strat-egy for students (Dubins and Freedman 1981, Roth 1982). This fact motivates our assumption that applicants’ rankings measure their true preferences for schools.1 We summarize these preferences by fitting discrete choice models to applicants’ rank-ordered preference lists.

We then combine the preference estimates with estimates of school treatment effects on test scores, high school graduation, college attendance, and college choice. Treatment effect estimates come from “ value-added” regression models of the sort commonly used to measure causal effects of teachers and schools (Todd and Wolpin 2003; Koedel, Mihaly, and Rockoff 2015). We generalize the conventional value-added approach to allow for match effects in academic outcomes and to relax the selection-on-observables assumption underlying standard models. Recent evi-dence suggests that value-added models controlling only for observables provide

1 As we discuss in Section I, DA is strategy-proof when students are allowed to rank every school, but the New York City mechanism only allows applicants to rank 12 choices. Most students do not fill their preference lists, however, and truthful ranking is a dominant strategy in this situation (Haeringer and Klijn 2009, Pathak and Sönmez 2013). Fack, Grenet, and He (2015) proposes empirical approaches to measuring student preferences with-out requiring that truth-telling is the unique equilibrium.

1504 THE AMERICAN ECONOMIC REVIEW MAY 2020

quantitatively useful but biased estimates of causal effects due to selection on unob-servables (Rothstein 2010, 2017; Chetty, Friedman, and Rockoff 2014a; Angrist et al. 2017). We therefore use the rich information on preferences contained in stu-dents’ rank-ordered choice lists to correct our estimates for selection on unobserv-ables. This selection correction is implemented by extending the classic multinomial logit control function estimator of Dubin and McFadden (1984) to a setting where rankings of multiple alternatives are known. We show that predictions from our models match the effects of randomized lottery assignment for a subset of schools where lottery quasi-experiments are available, suggesting that our methods provide accurate measures of causal effects.

The final step of our analysis relates the choice model and treatment effect esti-mates to measure preferences for school effectiveness. The choice and outcome models we estimate allow preferences and causal effects to vary flexibly with stu-dent characteristics. Our specifications accommodate the possibility that schools are more effective for specific types of students and that applicants choose schools that are a good match for their student type. We compare the degree to which parent preferences are explained by overall school effectiveness, match quality, and peer quality, defined as the component of a school’s average outcome due to selection rather than effectiveness. We explore these relationships for test scores as well as longer-run postsecondary outcomes, which is important in view of recent evidence that school quality is multidimensional and only imperfectly measured by effects on test scores (Beuermann et al. 2018).

We find preferences are positively correlated with both peer quality and causal effects on student outcomes. More effective schools enroll higher-ability students, however, and preferences are unrelated to school effectiveness after controlling for peer quality. We also find little evidence of selection on match effects: on bal-ance, parents do not prefer schools that are especially effective for their own chil-dren, and students do not enroll in schools that are a better-than-average match. These patterns are similar for short-run achievement test scores and longer-run postsecondary outcomes. Looking across demographic and baseline achievement groups, we find no evidence that any subgroup places positive weight on school effectiveness once we adjust for peer quality. Our estimates are also similar across applicant cohorts, suggesting that the relationship between demand and effective-ness is stable over time.

The factors driving school popularity we uncover are noteworthy, but to trans-late them into implications about the incentives that schools face from demand-side forces requires isolating the causal impacts of school attributes on preferences. Since effectiveness and peer quality are not randomly assigned to schools, our estimates need not capture causal effects of these attributes on preferences if other school characteristics that influence demand are correlated with effectiveness or peer qual-ity. We assess the potential for such omitted variables bias by conditioning on other school characteristics that predict demand, including measures of violent incidents, teacher education, and the school learning environment. This analysis reveals that parents prefer safer schools and schools with more educated teachers, but adding these covariates does not alter our main results characterizing the partial correla-tions between preferences, peer quality, and school effectiveness. This robustness exercise provides some reassurance that our estimates capture causal impacts of

1505ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

effectiveness on demand, though we cannot completely rule out the possibility that peer quality and school effectiveness are correlated with other unobservables.

It is worth cautioning that our findings do not mean that parents choose schools irrationally; they may use peer characteristics to proxy for school effectiveness if the latter is difficult to observe, or value peer quality independently of impacts on academic outcomes. Either way, our results imply that parents’ choices penalize schools that enroll low achievers rather than schools that offer poor instruction. As a result, school choice programs may generate stronger incentives for screening and selection than for improved academic quality. We provide suggestive evidence that schools have responded to these incentives by increasing screening in the years fol-lowing the introduction of centralized assignment in New York City.

Our analysis complements Rothstein’s (2006) indirect test with a direct assess-ment of the relationships among parent preferences, peer quality, and school effec-tiveness based on unusually rich choice and outcome data. The results also contribute to a large literature studying preferences for school quality (Black 1999; Figlio and Lucas 2004; Bayer, Ferreira, and McMillan 2007; Hastings and Weinstein 2008; Burgess et al. 2015; Imberman and Lovenheim 2016). These studies show that hous-ing prices and household choices respond to school performance levels, but they do not typically separate responses to causal school effectiveness and peer quality. Our findings are also relevant to theoretical and empirical research on the impli-cations of school choice for sorting and stratification (Epple and Romano 1998; Epple, Figlio, and Romano 2004; Hsieh and Urquiola 2006; Barseghyan, Clark, and Coate 2014; Altonji, Huang, and Taber 2015; Avery and Pathak 2015; MacLeod and Urquiola 2015; MacLeod et al. 2017). In addition, our results help to reconcile some surprising findings from recent studies of school choice. Cullen, Jacob, and Levitt (2006) finds limited achievement effects of admission to preferred schools in Chicago, while Walters (2018) documents that disadvantaged students in Boston are less likely to apply to charter schools than more advantaged students despite experiencing larger achievement benefits. Angrist, Pathak, and Walters (2013) and Abdulkadiroglu, Pathak, and Walters (2018) report on two settings where parents opt for schools that reduce student achievement. These patterns are consistent with our finding that school choices are not driven by school effectiveness.

Finally, our analysis adds to a recent series of studies leveraging preference data from centralized school assignment mechanisms to investigate school demand (Hastings, Kane, and Staiger 2009; Harris and Larsen 2014; Fack, Grenet, and He 2015; Abdulkadiroglu, Agarwal, and Pathak 2017; Glazerman and Dotter 2016; Kapor, Neilson, and Zimmerman 2017; Agarwal and Somaini 2018). Some of these studies analyze assignment mechanisms that provide incentives to strategically mis-report preferences, while others measure academic quality using average test scores rather than distinguishing between peer quality and school effectiveness or look-ing at longer-run outcomes. We build on this previous work by using data from a strategy-proof mechanism to separately estimate preferences for peer quality and causal effects on multiple measures of academic performance.

The rest of the paper is organized as follows. The next section describes school choice in New York City and the data used for our analysis. Section II develops a conceptual framework for analyzing school effectiveness and peer quality, and Section III details our empirical approach. Section IV summarizes estimated

1506 THE AMERICAN ECONOMIC REVIEW MAY 2020

distributions of student preferences and school treatment effects. Section V links preferences to peer quality and school effectiveness, and Section VI discusses impli-cations of these relationships. Section VII concludes and offers some directions for future research.

I. Setting and Data

A. New York City High Schools

The New York City public school district annually enrolls roughly 90,000 ninth graders at more than 400 high schools. Rising ninth graders planning to attend New York City’s public high schools submit applications to the centralized assignment system. Before 2003 the district used an uncoordinated school assignment process in which students could receive offers from more than one school. Motivated in part by insights derived from the theory of market design, in 2003 the city adopted a coordi-nated single-offer assignment mechanism based on the student-proposing deferred acceptance (DA) algorithm (Gale and Shapley 1962; Abdulkadiroglu, Pathak, and Roth 2005, 2009). Abdulkadiroglu, Agarwal, and Pathak (2017) shows that intro-ducing coordinated assignment reduced the share of administratively assigned stu-dents and likely improved average student welfare.

Applicants report their preferences for schooling options to the assignment mech-anism by submitting rank-ordered lists of up to 12 academic programs. An individual school may operate more than one program. To aid families in their decision-making, the New York City Department of Education (DOE) distributes a directory that pro-vides an overview of the high school admission process, key dates, and an information page for each high school. A school’s information page includes a brief statement of its mission, a list of offered programs, courses and extracurricular activities, pass rates on New York Regents standardized tests, and the school’s graduation rate (New York City Department of Education 2003). DOE also issues annual schools reports that list basic demographics, teacher characteristics, school expenditures, and Regents perfor-mance levels. During the time period of our study ( 2003–2007) these reports did not include measures of test score growth, though such measures have been added more recently (New York City Department of Education 2004, 2017).

Academic programs prioritize applicants in the centralized admission sys-tem using a mix of factors. Priorities depend on whether a program is classified as unscreened, screened, or an educational option program. Unscreened programs give priority to students based on residential zones and (in some cases) to those who attend an information session. Screened programs use these factors and may also assign priorities based on prior grades, standardized test scores, and atten-dance. Educational option programs use screened criteria for some of their seats and unscreened criteria for the rest. Random numbers are used to order applicants with equal priority. A small group of selective high schools, including New York City’s exam schools, admit students in a parallel system outside the main round of the assignment process (Abdulkadiroglu, Angrist, and Pathak 2014).

The DA algorithm combines student preferences with program priorities to gen-erate a single program assignment for each student. In the initial step of the algo-rithm, each student proposes to her first-choice program. Programs provisionally

1507ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

accept students in order of priority up to capacity and reject the rest. In subsequent rounds, each student rejected in the previous step proposes to her most-preferred program among those that have not previously rejected her, and programs reject pro-visionally accepted applicants in favor of new applicants with higher priority. This process iterates until all students are assigned to a program or all unassigned stu-dents have been rejected by every program they have ranked. During our study time period, students left unassigned in the main round participate in a supplementary DA round in which they rank up to 12 additional programs with available seats. Any remaining students are administratively assigned by the district. About 82 percent, 8 percent, and 10 percent of applicants are assigned in the main, supplementary, and administrative rounds, respectively (Abdulkadiroglu, Agarwal, and Pathak 2017).

An attractive theoretical property of the DA mechanism is that it is strategy-proof: since high-priority students can displace those with lower priority in later rounds of the process, listing schools in order of true preferences is a dominant strategy in the mechanism’s canonical version. This property, however, requires students to have the option to rank all schools (Haeringer and Klijn 2009, Pathak and Sönmez 2013). As we show below, more than 70 percent of students rank fewer than 12 programs, meaning that truthful ranking of schools is a dominant strategy for the majority of applicants. The instructions provided with the New York City high school applica-tion also directly instruct students to rank schools in order of their true preferences (New York City Department of Education 2003). In the analysis to follow, we inter-pret students’ rank-ordered lists as truthful reports of their preferences. We also probe the robustness of our findings to violations of this assumption by reporting results based on students who rank fewer than 12 choices.2

B. Data and Samples

The data used here are extracted from a DOE administrative information sys-tem covering all students enrolled in New York City public schools between the 2003–2004 and 2012–2013 school years (New York City Department of Education 2013). These data include school enrollment, student demographics, home addresses, scores on New York Regents standardized tests, Preliminary SAT (PSAT) scores, and high school graduation records, along with preferences submitted to the cen-tralized high school assignment mechanism. A supplemental file from the National Student Clearinghouse (NSC) reports college enrollment for students graduating from New York City high schools between 2009 and 2012. A unique student identi-fier links records across these files.

We analyze high school applications and outcomes for four cohorts of students enrolled in New York City public schools in eighth grade between 2003–2004 and 2006–2007. This set of students is used to construct several samples for statistical analysis. The choice sample, used to investigate preferences for schools, consists of all high school applicants with baseline (eighth grade) demographic, test score, and address information. Our analysis of school effectiveness uses subsamples of the choice sample corresponding to each outcome of interest. These outcome

2 Along similar lines, Abdulkadiroglu, Agarwal, and Pathak (2017) shows that preference estimates using only the top-ranked school, the top three schools, and all but the last ranked school are similar.

1508 THE AMERICAN ECONOMIC REVIEW MAY 2020

samples include students with observed outcomes, baseline scores, demographics, and addresses, enrolled for ninth grade at one of 316 schools with at least 50 stu-dents for each outcome. The outcome samples also exclude students enrolled at the nine selective high schools that do not admit students via the main DA mechanism. Online Appendix Section A and Appendix Table A1 provide further details on data sources and sample construction.

Key outcomes in our analysis include Regents math standardized test scores, PSAT scores, high school graduation, college attendance, and college quality. The high school graduation outcome equals 1 if a student graduates within five years of her projected high school entry date given her eighth grade cohort. Likewise, college attendance equals 1 for students who enroll in any college (two or four year) within two years of projected on-time high school graduation. The college quality variable, derived from Internal Revenue Service tax record statistics reported by Chetty et al. (2017), equals the mean 2014 income for children born between 1980 and 1982 who attended a student’s college. The mean income for the non-college population is assigned to students who do not enroll in a college. While this metric does not distinguish between student quality and causal college effectiveness, it provides a measure of the selectivity of a student’s college. It has also been used elsewhere to assess effects of education programs on the intensive margin of college attendance (Chetty et al. 2011; Chetty, Friedman, and Rockoff 2014b). College attendance and quality are unavailable for the 2003–2004 cohort because the NSC data window does not allow us to determine whether students in this cohort were enrolled in col-lege within two years of projected high school graduation.

Descriptive statistics for the choice and outcome samples appear in Table 1. These statistics show that New York City schools serve a disadvantaged urban population. Seventy-three percent of students are black or Hispanic, and 65 percent are eligible for a subsidized lunch. Data from the 2011–2015 American Community Surveys show that the average student in the choice sample lives in a census tract with a median household income of $50,136 in 2015 dollars. Observed characteristics are generally similar for students in the choice and outcome samples. The average PSAT score in New York City is 116, about 1 standard deviation below the US average (the PSAT is measured on a 240-point scale, normed to have a mean of 150 and a stan-dard deviation of 30). The five-year high school graduation rate is 61 percent, and 48 percent of students attend some college within two years of graduation.

C. Choice Lists

New York City high school applicants tend to prefer schools near their homes, and most do not fill their choice lists. These facts are shown in Table 2, which summarizes rank-ordered preference lists in the choice sample. As shown in col-umn 1, 93 percent of applicants submit a second choice, about one-half submit 8 or more choices, and 28 percent submit the maximum 12 allowed choices. Column 2 shows that students prefer schools located in their home boroughs: 85 percent of first-choice schools are in the same borough as the student’s home address, and the fraction of other choices in the home borough are also high. Abdulkadiroglu, Agarwal, and Pathak (2017) reports that for 2003–2004, 193 programs restricted eligibility to applicants who reside in the same borough. The preference analysis

1509ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

Table 1—Descriptive Statistics for New York City Eighth Graders

Outcome samples

Choice sample Regents math PSAT HS graduation College(1) (2) (3) (4) (5)

Female 0.497 0.518 0.532 0.500 0.500Black 0.353 0.377 0.359 0.376 0.372Hispanic 0.381 0.388 0.384 0.399 0.403Subsidized lunch 0.654 0.674 0.667 0.680 0.700Census tract median income $50,136 $50,004 $49,993 $49,318 $49,243Bronx 0.231 0.221 0.226 0.236 0.239Brooklyn 0.327 0.317 0.335 0.339 0.333Manhattan 0.118 0.118 0.119 0.116 0.116Queens 0.259 0.281 0.255 0.250 0.253Staten Island 0.065 0.063 0.064 0.059 0.059Regents math score 0.000 −0.068 0.044 −0.068 −0.044PSAT score 120 116 116 116 115High school graduation 0.587 0.763 0.789 0.610 0.624Attended college 0.463 0.588 0.616 0.478 0.478College quality $31,974 $33,934 $35,010 $31,454 $31,454

Observations 270,157 155,850 149,365 230,087 173,254

Notes: This table shows descriptive statistics for applicants to New York City public high schools between the 2003–2004 and 2006–2007 school years. Column 1 reports average characteristics and outcomes for all applicants with complete information on preferences, demographics, and eighth-grade test scores. Columns 2–5 display char-acteristics for the Regents math, PSAT, high school graduation, and college outcome samples. Outcome samples are restricted to students with data on the relevant outcome, enrolled in ninth grade at schools with at least 50 students for each outcome. Regents math scores are normalized to mean 0 and standard deviation 1 in the choice sample. High school graduation equals 1 for students who graduate from a New York City high school within five years of the end of their eighth grade year. College attendance equals 1 for students enrolled in any college within two years of projected high school graduation. College quality is the mean 2014 income for individuals in the 1980–1982 birth cohorts who attended a student’s college. This variable equals the mean income in the non-college population for students who did not attend college. The college outcome sample excludes students in the 2003–2004 cohort. Census tract median income is median household income measured in 2015 dollars using data from the 2011–2015 American Community Surveys. Regents math, PSAT, graduation, and college outcome statistics exclude students with missing values.

Table 2—Correlates of Preference Rankings for New York City High Schools

Fraction reporting Same borough Distance Regents math score(1) (2) (3) (4)

Choice 1 1.000 0.849 2.71 0.200Choice 2 0.929 0.844 2.94 0.149Choice 3 0.885 0.839 3.04 0.116Choice 4 0.825 0.828 3.12 0.085Choice 5 0.754 0.816 3.18 0.057Choice 6 0.676 0.803 3.23 0.030Choice 7 0.594 0.791 3.28 0.009Choice 8 0.523 0.780 3.29 −0.013Choice 9 0.458 0.775 3.31 −0.031Choice 10 0.402 0.773 3.32 −0.051Choice 11 0.345 0.774 3.26 −0.071Choice 12 0.278 0.787 3.04 −0.107

Notes: This table reports average characteristics of New York City high schools by student pref-erence rank. Column 1 displays fractions of student applications listing each choice. Column 2 reports the fraction of listed schools located in the same borough as a student’s home address. Column 3 reports the mean distance between a student’s home address and each ranked school, measured in miles. This column excludes schools outside the home borough. Column 4 shows average Regents math scores in standard deviation units relative to the New York City average.

1510 THE AMERICAN ECONOMIC REVIEW MAY 2020

to follow, therefore, treats schools in a student’s home borough as her choice set and aggregates schools in other boroughs into a single outside option. Column 3, which reports average distances (measured as great-circle distance in miles) for each choice restricted to schools in the home borough, shows that students rank nearby schools higher within boroughs as well.

Applicants also prefer schools with strong academic performance. The last col-umn of Table 2 reports the average Regents high school math score for schools at each position on the rank list. Regents scores are normalized to have mean 0 and standard deviation 1 in the New York City population. To earn a high school diploma in New York state, students must pass a Regents math exam. These results reveal that higher-ranked schools enroll students with better math scores. The average score at a first-choice school is 0.2 standard deviations ( σ ) above the city average, and aver-age scores monotonically decline with rank. PSAT, graduation, college enrollment, and college quality indicators also decline with rank. Students and parents clearly prefer schools with high achievement levels. Our objective in the remainder of this paper is to decompose this pattern into components due to preferences for school effectiveness and peer quality.

II. Conceptual Framework

Consider a population of students indexed by i , each of whom attends one of J schools. Let Y ij denote the potential value of some outcome of interest for student i if she attends school j . The projection of Y ij on a vector of observed characteris-tics, X i , is written as

(1) Y ij = α j + X i ′ β j + ϵ ij ,

where E [ ϵ ij ] = E [ X i ϵ ij ] = 0 by definition of α j and β j . The coefficient vector β j measures the returns to observed student characteristics at school j , while ϵ ij reflects variation in potential outcomes unexplained by these characteristics. We further normalize E [ X i ] = 0 , so α j = E [ Y ij ] is the population mean potential outcome at school j . The realized outcome for student i is Y i = ∑ j 1 S i = j Y ij , where S i ∈ 1, …, J denotes school attendance.

We decompose potential outcomes into components explained by student abil-ity, school effectiveness, and idiosyncratic factors. Let A i = (1/J) ∑ j Y ij denote student i ’s general ability, defined as the average of her potential outcomes across all schools. This variable describes how the student would perform at the average school. Adding and subtracting A i on the right-hand side of (1) yields

(2) Y ij = α – + X i ′ β – + ϵ – i A i

+ ( α j − α – ) ⏟

AT E j

+ X i ′ ( β j − β – ) + ( ϵ ij − ϵ – i )

M ij

,

where α – = (1/J) ∑ j α j , β – = (1/J) ∑ j β j , and ϵ – i = (1/J) ∑ j ϵ ij . Equation (2) shows

that student i ’s potential outcome at school j is the sum of three terms: the student’s general ability, A i ; the school’s average treatment effect, AT E j , defined as the causal effect of school j relative to an average school for an average student; and a match effect, M ij , which reflects student i ’s idiosyncratic suitability for school j . Match

1511ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

effects may arise either because of an interaction between student i ’s observed characteristics and the extra returns to characteristics at school j (captured by X i ′ ( β j − β –

) ) or because of unobserved factors that make student i more or less suit-able for school j (captured by ϵ ij − ϵ – i ).

This decomposition allows us to interpret variation in observed outcomes across schools using three terms. The average outcome at school j is given by

(3) E [ Y i | S i = j] = Q j + AT E j + E [ M ij | S i = j] .

Here Q j = E [ A i | S i = j] is the average ability of students enrolled at school j , a variable we label “peer quality.” The quantity E [ M ij | S i = j] is the average suitabil-ity of j ’s students for this particular school. In a Roy (1951)-style model in which students sort into schools on the basis of comparative advantage in the production of Y i , we would expect these average match effects to be positive. Parents and stu-dents may also choose schools on the basis of peer quality Q j , overall school effec-tiveness AT E j , or the idiosyncratic match M ij for various outcomes.

III. Empirical Methods

The goal of our empirical analysis is to assess the roles of peer quality, school effectiveness, and academic match quality in applicant preferences. Our analysis proceeds in three steps. We first use rank-ordered choice lists to estimate prefer-ences, thereby generating measures of each school’s popularity. Next, we estimate schools’ causal effects on test scores, high school graduation, college attendance, and college choice. Finally, we combine these two sets of estimates to characterize the relationships among school popularity, peer quality, and causal effectiveness.

A. Estimating Preferences

Let U ij denote student i ’s utility from enrolling in school j , and let = 1, …, J represent the set of available schools. We abstract from the fact that students rank programs rather than schools by ignoring repeat occurrences of any individual school on a student’s choice list. Therefore, U ij may be interpreted as the indirect utility associated with student i ’s favorite program at school j . The school ranked first on a student’s choice list is

R i1 = arg max j∈

U ij ,

while subsequent ranks satisfy

R ik = arg max j∈ \ R im :m<k

U ij , k > 1 .

Student i ’s rank-order list is then R i = ( R i1 , …, R iℓ (i) ) ′, where ℓ (i) is the length of the list submitted by this student.

1512 THE AMERICAN ECONOMIC REVIEW MAY 2020

We summarize these preference lists by fitting random utility models with param-eters that vary according to observed student characteristics. Student i ’s utility from enrolling in school j is modeled as

(4) U ij = δ c ( X i ) j − τ c ( X i ) D ij + η ij ,

where the function c ( X i ) assigns students to covariate cells based on the variables in the vector X i , and D ij records distance from student i ’s home address to school j . The parameter δ cj is the mean utility of school j for students in covariate cell c , and τ c is a cell-specific distance parameter or “cost.” We include distance in the model because a large body of evidence suggests it plays a central role in school choices (e.g., Hastings, Kane, and Staiger 2009 and Abdulkadiroglu et al. 2017). We model unobserved tastes η ij as following independent extreme value type I distributions conditional on X i and D i = ( D i1 , …, D iJ ) ′. Equation (4) is therefore a rank-ordered multinomial logit model (Hausman and Ruud 1987).

The logit model implies the conditional likelihood of the rank list R i is

( R i | X i , D i ) = ∏ k=1

ℓ (i)

exp ( δ c ( X i ) R ik − τ c ( X i ) D i R ik ) _________________________

∑ j∈ \ R im :m<k exp ( δ c ( X i ) j − τ c ( X i ) D ij )

.

We allow flexible heterogeneity in tastes by estimating preference models sep-arately for 360 covariate cells defined by the intersection of borough, sex, race/ethnicity (black, Hispanic, or other), subsidized lunch status, above-median cen-sus tract income, and terciles of the mean of eighth grade math and reading scores. This specification follows several recent studies that flexibly parametrize preference heterogeneity in terms of observable characteristics (e.g., Hastings, Hortaçsu, and Syverson 2017 and Langer 2016). Students rarely rank schools outside their home bor-oughs, so covariate cells often include zero students ranking any given out-of-borough school. We therefore restrict the choice set to schools located in the home borough and aggregate all other schools into an outside option with utility normalized to zero. Maximum likelihood estimation of the preference parameters produces a list of school mean utilities along with a distance coefficient for each covariate cell.

B. Estimating School Effectiveness

Our analysis of school effectiveness aims to recover the parameters of the poten-tial outcome equations defined in Section II. We take two approaches to estimating these parameters.

Approach 1: Selection on observables.

The first set of estimates is based on the assumption

(5) E [ Y ij | X i , S i ] = α j + X i ′ β j , j = 1, …, J.

This restriction, often labeled “selection on observables,” requires school enroll-ment to be as good as random conditional on the covariate vector X i , which includes

1513ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

sex, race, subsidized lunch status, the log of median census tract income, and eighth grade math and reading scores. Assumption (5) implies that an ordinary least squares (OLS) regression of Y i on school indicators interacted with X i recovers unbiased estimates of α j and β j for each school. This fully interacted specification is a multiple-treatment extension of the Oaxaca-Blinder (1973) treatment effects estimator (Kline 2011).3 By allowing school effectiveness to vary with student char-acteristics, we generalize the constant effects “ value-added” approach commonly used to estimate the contributions of teachers and schools to student achievement (Koedel, Mihaly, and Rockoff 2015).

The credibility of the selection on observables assumption underlying value-added estimators is a matter of continuing debate (Rothstein 2010, 2017; Kane et al. 2013; Bacher-Hicks, Kane, and Staiger 2014; Chetty, Friedman, and Rockoff 2014a, 2016, 2017; Guarino, Reckase, and Wooldridge 2015). Comparisons to results from admis-sion lotteries indicate that school value-added models accurately predict the impacts of random assignment but are not perfectly unbiased (Deming 2014; Angrist et al. 2016b, 2017). Selection on observables may also be more plausible for test scores than for longer-run outcomes, for which lagged measures of the dependent variable are not available (Chetty, Friedman, and Rockoff 2014a). We therefore report OLS estimates as a benchmark and compare these to estimates from a more general strat-egy that relaxes assumption (5).

Approach 2: Rank-ordered control functions.

Our second approach is motivated by the restriction

(6) E [ Y ij | X i , D i , η i1 , …, η iJ , S i ] = α j + X i ′ β j + g j ( D i , η i1 , …, η iJ ) , j = 1, …, J.

This restriction implies that any omitted variable bias afflicting OLS value-added estimates is due either to spatial heterogeneity captured by distances to each school ( D i ) or to the preferences underlying the rank-ordered lists submitted to the assign-ment mechanism ( η ij ). The function g j ( · ) allows potential outcomes to vary arbi-trarily across students with different preferences over schools. Factors that lead students with the same observed characteristics, spatial locations, and preferences to ultimately enroll in different schools, such as school priorities, random rationing due to oversubscription, or noncompliance with the assignment mechanism, are pre-sumed to be unrelated to potential outcomes.

Under assumption (6), comparisons of matched sets of students with the same covariates, values of distance, and rank-ordered choice lists recover causal effects of school attendance. This model is therefore similar to the “ self-revelation” model proposed by Dale and Krueger (2002, 2014) in the context of postsecondary enroll-ment. Dale and Krueger assume that students reveal their unobserved “types” via the selectivity of their college application portfolios, so college enrollment is as good as random among students who apply to the same schools. Similarly, (6) implies that

3 We also include main effects of borough so that the model includes the same variables used to define covariate cells in the preference model.

1514 THE AMERICAN ECONOMIC REVIEW MAY 2020

high school applicants reveal their types through the content of their rank-ordered preference lists.

Though intuitively appealing, full nonparametric matching on rank-ordered lists is not feasible in practice because few students share the exact same rankings. We therefore use the structure of the logit choice model in equation (4) to derive a para-metric approximation to this matching procedure. Specifically, we replace equation (6) with the assumption

(7) E [ Y ij | X i , D i , η i1 , …, η iJ , S i ]

= α j + X i ′ β j + D i ′ γ + ∑ k=1

J

ψ k × ( η ik − μ η ) + φ × ( η ij − μ η ) , j = 1, …, J,

where μ η = E [ η ij ] is Euler’s constant.4 As in the multinomial logit selection model of Dubin and McFadden (1984), equation (7) imposes a linear relationship between potential outcomes and the unobserved logit errors. Functional form assumptions of this sort are common in multinomial selection models with many alternatives, where requirements for nonparametric identification are very stringent (Lee 1983; Dahl 2002; Heckman, Urzua, and Vytlacil 2008).5

Equation (7) accommodates a variety of forms of selection on unobservables. The coefficient ψ k represents an effect of the preference for school k common to all potential outcomes. This permits students with strong preferences for particu-lar schools to have higher or lower general ability A i . The parameter φ captures an additional match effect of the preference for school j on the potential outcome at this specific school. The model therefore allows for “essential” heterogeneity linking preferences to unobserved match effects in student outcomes (Heckman, Urzua, and Vytlacil 2006). A Roy (1951)-style model of selection on gains would imply φ > 0 , but we do not impose this restriction.

By iterated expectations, equation (7) implies that mean observed outcomes at school j are

(8) E [ Y i | X i , D i , R i , S i = j]

= α j + X i ′ β j + D i ′ γ + ∑ k=1

J

ψ k λ k ( X i , D i , R i ) + φ λ j ( X i , D i , R i ) ,

where λ k ( X i , D i , R i ) = E [ η ik − μ η | X i , D i , R i ] gives the mean preference for school k conditional on a student’s characteristics, spatial location, and preference list. The λ k ( · ) s serve as “control functions” correcting for selection on unobservables (Heckman and Robb 1985, Blundell and Matzkin 2014, Wooldridge 2015). As shown in online Appendix Section B.1, these functions are generalizations of the formulas derived by Dubin and McFadden (1984), extended to account for the fact that we observe a list of several ranked alternatives rather than just the most preferred choice.

Note that equation (8) includes main effects of distance to each school; we do not impose an exclusion restriction for distance. Identification of the selection parame-ters ψ k and φ comes from variation in preference rankings for students who enroll at

4 The means of both X i and D i are normalized to zero to maintain the interpretation that α j = E [ Y ij ] .5 As discussed in Section V, we also estimate an alternative model that includes fixed effects for first choice schools.

1515ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

the same school conditional on covariates and distance. Intuitively, if students who rank school j highly do better than expected given their observed characteristics at all schools, we will infer that ψ j > 0 . If these students do better than expected at school j but not elsewhere, we will infer that φ > 0 .

We use the choice model parameters to build first-step estimates of the con-trol functions, then estimate equation (8) in a second-step OLS regression of Y i on school indicators and their interactions with X i , controlling for D i and the esti-mated λ k ( · ) functions.6 We adjust inference for estimation error in the control func-tions via a two-step extension of the score bootstrap procedure of Kline and Santos (2012). As detailed in online Appendix Section B.2, the score bootstrap avoids the need to recalculate the first-step logit estimates or the inverse variance matrix of the second-step regressors in the bootstrap iterations.

The Joint Distribution of Peer Quality and School Effectiveness.—Estimates of equations (5) and (7) may be used to calculate each school’s peer quality. A stu-dent’s predicted ability in the value-added model is

(9) A ˆ i = 1 _ J ∑ j=1

J

[ α ˆ j + X i ′ β ˆ j ] ,

where α ˆ j and β ˆ j are OLS value-added coefficients. Predicted ability in the control function model adds estimates of the distance and control function terms in equation (8). Estimated peer quality at school j is then Q ˆ j = ∑ i 1 S i = j A ˆ i / ∑ i 1 S i = j , the average predicted ability of enrolled students.

The end result of our school quality estimation procedure is a vector of estimates for each school, θ ˆ j = ( α ˆ j , β ˆ j ′ , Q ˆ j ) ′. The vector of parameters for the control function model also includes an estimate of the selection coefficient for school j , ψ ˆ j . These estimates are unbiased but noisy measures of the underlying school-specific param-eters θ j . We investigate the distribution of θ j using the following hierarchical model:

(10) θ ˆ j | θ j ∼ N ( θ j , Ω j ) ,

θ j ∼ N ( μ θ , Σ θ ) .

Here Ω j is the sampling variance of the estimator θ ˆ j , while μ θ and Σ θ govern the dis-tribution of latent parameters across schools. In a hierarchical Bayesian framework μ θ and Σ θ are hyperparameters describing a prior distribution for θ j . We estimate these hyperparameters by maximum likelihood applied to model (10), approximat-ing Ω j with an estimate of the asymptotic variance of θ ˆ j .7 The resulting estimates of μ θ and Σ θ characterize the joint distribution of peer quality and school treatment effect parameters, purged of the estimation error in θ ˆ j .

6 The choice model uses only preferences over schools in students’ home boroughs, so λ k ( · ) is undefined for students outside school k ’s borough. We therefore include dummies for missing values and code the control func-tions to 0 for these students. We similarly code D ik to 0 for students outside of school k ’s borough and include bor-ough indicators so that the distance coefficients are estimated using only within-borough variation. Our key results are not sensitive to dropping students attending out-of-borough schools from the sample.

7 The peer quality estimates Q ˆ j are typically very precise, so we treat peer quality as known rather than estimated when fitting the hierarchical model.

1516 THE AMERICAN ECONOMIC REVIEW MAY 2020

This hierarchical model can also be used to improve estimates of parameters for individual schools. An empirical Bayes (EB) posterior mean for θ j is given by

θ j ∗ = ( Ω ˆ j −1 + Σ ˆ θ −1 ) −1 ( Ω ˆ j −1 θ ˆ j + Σ ˆ θ −1 μ ˆ θ ) ,

where Ω ˆ j , μ ˆ θ , and Σ ˆ θ are estimates of Ω j , μ θ , and Σ θ . Relative to the unbiased but noisy estimate θ ˆ j , this EB shrinkage estimator uses the prior distribution to reduce sampling variance at the cost of increased bias, yielding a minimum mean squared error (MSE) prediction of θ j (Robbins 1956, Morris 1983). This approach paral-lels recent work applying shrinkage methods to estimate causal effects of teachers, schools, neighborhoods, and hospitals (Chetty, Friedman, and Rockoff 2014a; Hull 2016; Angrist et al. 2017; Chetty and Hendren 2017; Finkelstein et al. 2017). Online Appendix Section B.3 further describes our EB estimation strategy. In addition to reducing MSE, empirical Bayes shrinkage eliminates attenuation bias that would arise in models using elements of θ ˆ j as regressors (Jacob and Lefgren 2008). We exploit this property by regressing estimates of school popularity on EB posterior means in the final step of our empirical analysis.

C. Linking Preferences to School Effectiveness

We relate preferences to peer quality and causal effects with regressions of the form

(11) δ ˆ cj = κ c + ρ 1 Q j ∗ + ρ 2 AT E j ∗ + ρ 3 M cj ∗ + ξ cj ,

where δ ˆ cj is an estimate of the mean utility of school j for students in covariate cell c , κ c is a cell fixed effect, and Q j ∗ and AT E j ∗ are EB posterior mean predictions of peer quality and average treatment effects. The variable M cj ∗ is an EB prediction of the mean match effect of school j for students in cell c . Observations in equa-tion (11) are weighted by the inverse sampling variance of δ ˆ cj . We use the vari-ance estimator proposed by Cameron, Gelbach, and Miller (2011) to double-cluster inference by cell and school. Two-way clustering accounts for correlated estimation errors in δ ˆ cj across schools within a cell as well as unobserved determinants of pop-ularity common to a given school across cells. We estimate equation (11) separately for Regents test scores, PSAT scores, high school graduation, college attendance, and college quality. The parameters ρ 1 , ρ 2 , and ρ 3 measure how preferences relate to peer quality, school effectiveness, and match quality.8

8 The control function version of our estimation procedure is closely related to classic selection-correction methods from studies of labor supply decisions. In their review of identification of labor supply models, French and Taber (2011) details a procedure that estimates labor market participation probabilities in a first step, uses these probabilities to selection-correct a wage equation in a second step, then relates participation to the unselected wage equation parameters in a third “structural probit” step. Similarly, we use preference estimates to selection-correct equations for student outcomes, then link the selection-corrected outcome estimates back to preferences to under-stand relationships between choices and treatment effects.

1517ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

IV. Parameter Estimates

A. Preference Parameters

Table 3 summarizes the distribution of household preference parameters across the 316 high schools and 360 covariate cells in the choice sample. The first row reports estimated standard deviations of the mean utility δ cj across schools and cells, while the second row displays the mean and standard deviation of the cell-specific distance cost τ c . School mean utilities are deviations from cell averages to account for differences in the reference category across boroughs, and calculations are weighted by cell size. We adjust these standard deviations for sampling error in the estimated preference parameters by subtracting the average squared standard error from the sample variance of mean utilities.

Consistent with the descriptive statistics in Table 1, the preference estimates indi-cate that households dislike more distant schools. The mean distance cost is 0.33. This implies that increasing the distance to a particular school by one mile reduces the odds that a household prefers this school to another in the same borough by 33 percent. The standard deviation of the distance cost across covariate cells is 0.12. While there is significant heterogeneity in distastes for distance, all of the estimated distance costs are positive, suggesting that all subgroups prefer schools closer to home.

The estimates in Table 3 reveal significant heterogeneity in tastes for schools both within and between subgroups. The within-cell standard deviation of school mean utilities, which measures the variation in δ cj across schools j for a fixed cell c , equals 1.12. This is equivalent to roughly 3.4 (1.12/0.33) miles of distance, implying that households are willing to travel substantial distances to attend more popular schools. The between-cell standard deviation, which measures variation in δ cj across c for a

Table 3—Variation in Student Preference Parameters

Standard deviations

Mean Within cells Between cells Total(1) (2) (3) (4)

School mean utility — 1.117 0.500 1.223(0.045) (0.003) (0.018)

Distance cost 0.330 — 0.120 0.120(0.006) (0.005) (0.005)

Number of students 270,157Number of schools 316Number of covariate cells 360

Notes: This table summarizes variation in school value-added and utility parameters across schools and covariate cells. Utility estimates come from rank-ordered logit models fit to stu-dent preference rankings. These models include school indicators and distance to school and are estimated separately in covariate cells defined by borough, gender, race, subsidized lunch status, an indicator for above or below the median of census tract median income, and tercile of the average of eighth grade math and reading scores. Column 1 shows the mean of the dis-tance coefficient across cells weighted by cell size. Column 2 shows the standard deviation of school mean utilities across schools within a cell, and column 3 shows the standard deviation of a given school’s mean utility across cells. School mean utilities are deviated from cell aver-ages to account for differences in the reference category across cells. Estimated standard devi-ations are adjusted for sampling error by subtracting the average squared standard error of the parameter estimates from the total variance.

1518 THE AMERICAN ECONOMIC REVIEW MAY 2020

fixed j , is 0.50, equivalent to about 1.5 (0.5/0.33) miles of distance. The larger within-cell standard deviation indicates that students in different subgroups tend to prefer the same schools.

B. School Effectiveness and Peer Quality

Our estimates of school treatment effects imply substantial variation in both causal effects and sorting across schools. Table 4 reports estimated means and standard deviations of peer quality Q j , average treatment effects AT E j , and slope coefficients β j . We normalize the means of Q j and AT E j to 0 and quantify the vari-ation in these parameters relative to the average school. As shown in column 2, the value-added model produces standard deviations of Q j and AT E j for Regents math scores equal to 0.29σ . This is somewhat larger than corresponding estimates of variation in school value-added from previous studies (usually around 0.15–0.2σ ; see, e.g., Angrist et al. 2017). One possible reason for this difference is that most

Table 4—Distributions of Peer Quality and Treatment Effect Parameters for Regents Math Scores

Value-added model Control function model

Mean SD Mean SD(1) (2) (3) (4)

Peer quality 0 0.288 0 0.305— (0.012) — (0.012)

ATE 0 0.290 0 0.233— (0.012) — (0.014)

Female −0.048 0.062 −0.029 0.062(0.005) (0.006) (0.005) (0.006)

Black −0.112 0.130 −0.108 0.120(0.011) (0.011) (0.010) (0.011)

Hispanic −0.097 0.114 −0.085 0.105(0.010) (0.011) (0.010) (0.012)

Subsidized lunch 0.001 0.052 0.026 0.054(0.005) (0.006) (0.005) (0.006)

log census tract median income 0.020 0.037 0.013 0.045(0.005) (0.007) (0.005) (0.006)

Eighth grade math score 0.622 0.105 0.599 0.105(0.007) (0.006) (0.007) (0.006)

Eighth grade reading score 0.159 0.048 0.143 0.052(0.004) (0.004) (0.004) (0.004)

Preference coefficient ( ψ j ) — — −0.001 0.007(0.001) (0.000)

Match coefficient (φ) — — 0.006 —(0.001)

Notes: This table reports estimated means and standard deviations of peer quality and school treatment effect parameters for Regents math scores. Peer quality is a school’s average predicted test score given the characteristics of its students. The ATE is a school’s average treatment effect, and other treatment effect parameters are school-spe-cific interactions with student characteristics. Estimates come from maximum likelihood models fit to school-spe-cific regression coefficients. Columns 1 and 2 report estimates from an OLS regression that includes interactions of school indicators with sex, race, subsidized lunch, the log of the median income in a student’s census tract, and eighth grade reading and math scores. This model also includes main effects of borough. Columns 3 and 4 show estimates from a control function model that adds distance to each school and predicted unobserved preferences from the choice model. Control functions and distance variables are set to 0 for out-of-borough schools and indica-tors for missing values are included.

1519ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

students in our sample attend high school for two years before taking Regents math exams, while previous studies look at impacts after one year.

As shown in columns 3 and 4 of Table 4, the control function model attributes some of the variation in Regents math value-added parameters to selection bias. Adding controls for unobserved preferences and distance increases the estimated standard deviation of Q j to 0.31σ and reduces the estimated standard deviation of AT E j to 0.23σ . Figure 1, which compares value-added and control function esti-mates for all five outcomes, demonstrates that this pattern holds for other outcomes as well: adjusting for selection on unobservables compresses the estimated distri-butions of treatment effects. This compression is more severe for high school grad-uation, college attendance, and college quality than for Regents math and PSAT scores. Our findings are therefore consistent with previous evidence that bias in OLS value-added models is more important for longer-run and non-test score outcomes (see, e.g., Chetty, Friedman, and Rockoff 2014b).

The bottom rows of Table 4 show evidence of substantial treatment effect hetero-geneity across students. For example, the standard deviation of the slope coefficient on a black indicator equals 0.12σ in the control function model. This implies that holding the average treatment effect AT E j fixed, a one standard deviation improve-ment in a school’s match quality for black students boosts scores for these students by about a tenth of a standard deviation relative to whites. We also find signifi-cant variation in slope coefficients for gender ( 0.06σ ), Hispanic ( 0.11σ ), subsidized lunch status ( 0.05σ ), the log of median census tract income ( 0.05σ ), and eighth grade math and reading scores ( 0.11σ and 0.05σ ). The final row of column 3 reports a control function estimate of φ , the parameter capturing matching between unob-served preferences and Regents scores. This estimate indicates a positive relation-ship between preferences and the unobserved component of student-specific test score gains, but the magnitude of the coefficient is very small.9

Our estimates imply that high-ability students tend to enroll in more effec-tive schools. Table 5 reports correlations between Q j and key school treatment effect parameters based on control function estimates for Regents math scores. Corresponding value-added estimates appear in online Appendix Table A2. The estimated correlation between peer quality and average treatment effects is 0.59. This may reflect either positive peer effects or higher-achieving students’ tendency to enroll in schools with better inputs. Our finding that schools with high-ability peers are more effective contrasts with recent studies of exam schools in New York City and Boston, which show limited treatment effects for highly selective pub-lic schools (Abdulkadiroglu, Angrist, and Pathak 2014; Dobbie and Fryer 2014). Within the broader New York public high school system, we find a strong positive association between school effectiveness and average student ability.

Table 5 also reports estimated correlations of Q j and AT E j with the slope coeffi-cients β j . Schools with larger average treatment effects tend to be especially good for girls: the correlation between AT E j and the female slope coefficient is positive and statistically significant. This is consistent with evidence from Deming et al.

9 The average predicted value of ( η ij − μ η ) for a student’s enrolled school in our sample is 2.0. Our estimate of φ therefore implies that unobserved match effects increase average test scores by about 1 percent of a standard deviation ( 0.006σ × 2.0 = 0.012σ) .

1520 THE AMERICAN ECONOMIC REVIEW MAY 2020

(2014) showing that girls’ outcomes are more responsive to school value-added. We estimate a very high positive correlation between black and Hispanic coefficients, suggesting that match effects tend to be similar for these two groups.

The slope coefficient on eighth grade reading scores is negatively correlated with peer quality and the average treatment effect. Both of these estimated correlations are below −0.4 and statistically significant. In other words, schools that enroll higher-ability students and produce larger achievement gains are especially effec-tive at teaching low-achievers. In contrast to our estimate of the parameter φ , this suggests negative selection on the observed component of match effects in student achievement. A similar selection pattern is documented by Walters (2018), which

Panel A. Regents math scores Panel B. PSAT scores

Panel D. College attendance

Panel E. log college quality

−1

−0.1

0

0.1

0.2

0.3

−0.2

−0.5

0

0.5

1

Con

trol

func

tion

estim

ate

−0.5−1 0 0.5 1

Value-added estimate

Value-added estimate

Value-added estimate

Value-added estimate

−10

0

10

20

30

Con

trol

func

tion

estim

ate

−10 0 10 20 30

Con

trol

func

tion

estim

ate

0 0.1−0.1−0.2 0.2 0.3

−0.2

−0.10

0.1

0.2

0.3

Con

trol

func

tion

estim

ate

0 0.1−0.1−0.2 0.2 0.3

Panel C. High school graduation

Value-added estimateCon

trol

func

tion

estim

ate

−0.1−0.2−0.3−0.3

0 0.1 0.2

−0.2

−0.1

0

0.1

0.2

Figure 1. Comparison of Value-Added and Control Function Estimates of School Average Treatment Effects

Notes: This figure plots school average treatment effect (ATE) estimates from value-added models against corre-sponding estimates from models including control functions that adjust for selection on unobservables. Value-added estimates come from regressions of outcomes on school indicators interacted with gender, race, subsidized lunch status, the log of census tract median income, and eighth grade math and reading scores. Control function mod-els add distance to school and predicted unobserved tastes from the choice model. Points in the figure are empiri-cal Bayes posterior means from models fit to the distribution of school-specific estimates. Dashed lines show the 45-degree line.

1521ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

shows that lower-scoring students in Boston are less likely to apply to charter schools despite receiving larger achievement benefits. Section V presents a more systematic investigation of relationships between preferences and match effects.

Patterns of estimates for PSAT scores, high school graduation, college atten-dance, and college quality are generally similar to results for Regents math scores. Online Appendix Tables A3–A6 present estimated distributions of peer quality and school effectiveness for these longer-run outcomes. For all five outcomes, we find substantial variation in peer quality and average treatment effects, a strong positive correlation between these variables, and significant effect heterogeneity with respect to student characteristics. Overall, causal effects for the longer-run outcomes are highly correlated with effects on Regents math scores. This is evident in Figure 2, which plots EB posterior mean predictions of average treatment effects on Regents scores against corresponding predictions for the other four outcomes. These results are consistent with recent evidence that short-run test score impacts reliably pre-dict effects on longer-run outcomes (Chetty et al. 2011; Dynarski, Hyman, and Schanzenbach 2013; Angrist et al. 2016a).

C. Decomposition of School Average Outcomes

We summarize the joint distribution of peer quality and school effectiveness by implementing the decomposition introduced in Section II. Table 6 uses the control function estimates to decompose variation in school averages for each outcome into

Table 5—Correlations of Peer Quality and Treatment Effect Parameters for Regents Math Scores

Control function parameters

Peerquality ATE Female Black Hispanic

Sub. lunch

Math score

Reading score

(1) (2) (3) (4) (5) (6) (7) (8)

ATE 0.588(0.052)

Female 0.078 0.299(0.078) (0.101)

Black 0.006 0.107 −0.177(0.077) (0.106) (0.142)

Hispanic −0.013 0.115 −0.235 0.922(0.080) (0.112) (0.150) (0.028)

Subsidized lunch 0.045 −0.168 0.066 −0.038 0.004(0.086) (0.117) (0.140) (0.153) (0.159)

Eighth grade math score −0.075 0.037 −0.074 −0.005 −0.007 0.060(0.064) (0.083) (0.099) (0.102) (0.109) (0.113)

Eighth grade reading score −0.418 −0.452 −0.193 −0.090 −0.078 0.004 0.256(0.068) (0.094) (0.117) (0.130) (0.138) (0.135) (0.099)

Preference coefficient ( ψ j ) 0.429 0.247 0.212 −0.083 −0.058 −0.127 −0.241 −0.281(0.063) (0.092) (0.104) (0.106) (0.111) (0.116) (0.083) (0.099)

Notes: This table reports estimated correlations between peer quality and school treatment effect parameters for Regents math scores. The ATE is a school’s average treatment effect, and other treatment effect parameters are school-specific interactions with student characteristics. Estimates come from maximum likelihood models fit to school-specific regression coefficients from a control function model controlling for observed characteristics, dis-tance to school, and unobserved tastes from the choice model.

1522 THE AMERICAN ECONOMIC REVIEW MAY 2020

components explained by peer quality, school effectiveness, average match effects, and covariances of these components.

Consistent with the estimates in Table 4, both peer quality and school effective-ness play roles in generating variation in school average outcomes, but peer quality is generally more important. Peer quality explains 47 percent of the variance in average Regents scores (0.093/0.191), while average treatment effects explain 28 percent (0.054/0.191). The explanatory power of peer quality for other outcomes ranges from 49 percent (PSAT scores) to 83 percent (high school graduation), while the importance of average treatment effects ranges from 10 percent (PSAT scores) to 19 percent (log college quality).

Despite the significant variation in slope coefficients documented in Table 4, match effects are unimportant in explaining dispersion in school average outcomes. The vari-ance of match effects accounts for only 5 percent of the variation in average Regents scores, and corresponding estimates for the other outcomes are also small. Although school treatment effects vary substantially across subgroups, there is not much sorting of students to schools on this basis, so the existence of potential match effects is of little consequence for realized variation in outcomes across schools.

Panel A. Regents math scores and PSAT scores

Panel B. Regents math scores andhigh school graduation

Panel C. Regents math scores and college attendance

Panel D. Regents math scores and log college quality

−10

−5

0

5

10P

SA

T e

ffect

−0.5 0 0.5 1

Regents math effect

−0.1

−0.05

0

0.05

0.1

0.15

Hig

h sc

hool

gra

duat

ion

effe

ct

−0.5 0 0.5 1

Regents math effect

−0.2

−0.1

0

0.1

0.2

Col

lege

atte

ndan

ce e

ffect

−0.5 0 0.5 1

Regents math effect

−0.1

−0.05

0

0.05

0.1

log

colle

ge q

ualit

y ef

fect

−0.5 0 0.5 1

Regents math effect

Figure 2. Relationships between Effects on Test Scores and Effects on Long-Run Outcomes

Notes: This figure plots estimates of causal effects on Regents math scores against estimates of effects on lon-ger-run outcomes. Treatment effects are empirical Bayes posterior mean estimates of school average treatment effects from control function models. Panel A plots the relationship between Regents math effects and effects on PSAT scores. Panels B, C, and D show corresponding results for high school graduation, college attendance, and log college quality.

1523ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

The final three rows of Table 6 quantify the contributions of covariances among peer quality, treatment effects, and match effects. As a result of the positive rela-tionship between peer quality and school effectiveness, the covariance between Q j and AT E j substantially increases cross-school dispersion in mean outcomes. The covariances between match effects and the other variance components are negative. This indicates that students at highly effective schools and schools with higher-ability students are less appropriately matched on the heterogeneous component of treat-ment effects, slightly reducing variation in school average outcomes.

D. Testing for Bias in Estimates of School Effectiveness

As described in Section I, New York City’s centralized assignment mecha-nism breaks ties at random for students with the same preferences and priorities. Abdulkadiroglu et al. (2017, 2019) derive methods for using the random assign-ment implicit in such systems for impact evaluation. The core of this approach uses student preferences and priorities along with the structure of the assignment mechanism to derive a probability of assignment (propensity score) for each student to each school where random tie-breaking occurs. Conditioning on the propensity score is sufficient to isolate the random component of school assignment, generating valid instruments for school enrollment (Rosenbaum and Rubin 1983). The priority information necessary to produce lottery propensity scores is only available for the 2003 New York City high school applicant cohort, and random assignment occurs for only a minority of schools. While these data constraints prevent us from produc-ing a lottery-based estimate of effectiveness for every school, we can use the shifts in school attendance resulting from these lottery assignments to test the accuracy of our value-added and control function estimates.

To implement these tests we first construct DA mechanism-based propensity scores and then apply the lottery-based test for bias in non-experimental estimators

Table 6—Decomposition of School Average Outcomes

Regents math

PSAT score/10

High school graduation

College attendance

log college quality

(1) (2) (3) (4) (5)

Total variance of average outcome 0.191 1.586 0.012 0.016 0.021Variance of peer quality 0.093 0.781 0.010 0.010 0.009Variance of ATE 0.054 0.160 0.002 0.003 0.004Variance of match 0.008 0.027 0.002 0.002 0.0012cov(peer quality, ATE) 0.081 0.745 0.005 0.008 0.0112cov(peer quality, match) −0.023 −0.061 −0.003 −0.003 −0.0022cov(ATE, match) −0.022 −0.068 −0.004 −0.005 −0.003

Notes: This table decomposes variation in average outcomes across schools into components explained by student characteristics, school average treatment effects (ATE), and the match between student characteristics and school effects. Estimates come from control function models adjusting for selection on unobservables. Column 1 shows results for Regents math scores in standard deviation units, column 2 reports estimates for PSAT scores, column 3 displays estimates for high school graduation, column 4 reports results for college attendance, and column 5 shows results for log college quality. The first row reports the total variance of average outcomes across schools. The sec-ond row reports the variance of peer quality, defined as the average predicted outcome as a function of student characteristics and unobserved tastes. The third row reports the variance of ATE, and the fourth row displays the variance of the match effect. The remaining rows show covariances of these components.

1524 THE AMERICAN ECONOMIC REVIEW MAY 2020

of school effectiveness proposed by Angrist et al. (2016b, 2017). For a set of L lot-teries, this test is implemented by estimating the following two-stage least squares (2SLS) system:

(12) Y i = κ 0 + ϕ Y ˆ i + ∑ ℓ=1

L

κ ℓ p iℓ + e i ,

(13) Y ˆ i = π 0 + ∑ ℓ=1

L

[ π ℓ Z iℓ + ω ℓ p iℓ ] + υ i ,

where Y ˆ i is the fitted value generated by a non-experimental estimation proce-dure (either value-added or control function), Z iℓ is an indicator equal to 1 if stu-dent i is assigned to school ℓ , and p iℓ is the propensity score measuring student i ’s probability of assignment to school ℓ . The first-stage coefficients π ℓ describe the non-experimental estimator’s predicted effects of assignment in each lottery, which are nonzero because lottery offers shift students across schools. Angrist et al. (2017) shows that the “forecast coefficient” ϕ should equal 1 if the estimator used to gen-erate Y ˆ i correctly predicts the effects of random lottery assignments on average, while the overidentification test for the system defined by (12) and (13) measures whether the estimator has the same predictive validity in every lottery. As discussed in Angrist et al. (2016b), the combination of these restrictions can be viewed as a Hausman (1983)-style test comparing lottery-based instrumental variables (IV) and OLS value-added estimates. If the restrictions underlying the value-added model are true, the value-added estimates should match lottery-based estimates of the same causal parameters.

As shown in Table 7, this lottery-based test suggests that our value-added and control function estimates accurately capture the causal effects of schools on stu-dent outcomes.10 Column 1 reports tests of an “uncontrolled” model that measures school effectiveness as the unadjusted mean outcome at the school. This model gen-erates forecast coefficients far from one and decisive rejections of the overiden-tification test for all three outcomes available for the 2003 cohort (Regents math, PSAT scores, and high school graduation), indicating that the available lotteries have power to detect bias in the most naïve nonexperimental estimators. Columns 2 and 3 show that the addition of controls for observed student characteristics gener-ates forecast coefficients much closer to 1 and overidentification tests that generally do not reject at conventional levels. Unfortunately, we cannot use lotteries to vali-date our estimates for postsecondary outcomes since college attendance data are not available for the 2003 applicant cohort, so estimates for these outcomes should be viewed more cautiously.

While the estimates in Table 7 are encouraging, it’s worth noting that the lotteries available in New York may have weak power to detect bias in our value-added and control function models. Specifically, the first stage F-statistics for equation (13) are below the rule-of-thumb value of 10 commonly used to diagnose weak instru-ments (Staiger and Stock 1997), implying that the lotteries tend to shift students

10 We validate our approach to reconstructing school lotteries by reporting relationships between lottery offers and students characteristics in Appendix Table A7. Without controls for propensity scores, school offers are strongly correlated with baseline test scores and other observables, but we cannot reject that offers at all schools are independent of observed characteristics after controlling for the propensity scores. This indicates that our strategy successfully isolates randomized lottery assignments.

1525ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

across schools with similar estimated effectiveness. Columns 4–6 demonstrate that we obtain similar results based on the Unbiased Jackknife Instrumental Variables (UJIVE) estimator proposed by Kolesár (2013). The UJIVE estimator performs well with weak instruments and (unlike other common approaches such as limited information maximum likelihood) it is robust to the presence of treatment effect heterogeneity. The UJIVE estimates suggest that our non-experimental estimators are accurate, particularly the control function estimator, for which we cannot reject forecast unbiasedness or the overidentifying restrictions for any outcome (we reject forecast unbiasedness for the value-added estimator for PSAT scores). The similar-ity of 2SLS and UJIVE also eases concerns about weak instrument bias in 2SLS, though the weak first stage also indicates that tests based on both estimators are

Table 7—Lottery-Based Tests for Bias in Estimates of School Effectiveness

2SLS UJIVE

Uncontrolled Value-addedControl function Uncontrolled Value-added

Control function

(1) (2) (3) (4) (5) (6)Panel A. MathForecast coefficient 0.599 0.965 0.967 0.598 0.961 0.963

(0.040) (0.038) (0.038) (0.040) (0.046) (0.045)p-value 0.000 0.354 0.388 0.000 0.343 0.376

Overid. χ 2 (123) stat. 174.1 84.88 86.50 174.1 84.88 86.50p-value 0.000 0.996 0.995 0.002 0.996 0.995

First-stage F-stat. 91.1 6.1 5.9 91.1 6.1 5.9Number of lotteries 124 124Number of students 22,515 22,515

Panel B. PSATForecast coefficient 0.306 0.879 0.912 0.296 0.815 0.862

(0.049) (0.048) (0.048) (0.050) (0.083) (0.084)p-value 0.000 0.012 0.066 0.000 0.025 0.101

Overid. χ 2 (123) stat. 145.8 112.9 106.8 145.7 111.7 106.1p-value 0.079 0.732 0.851 0.0792 0.759 0.861

First-stage F-stat. 62.9 2.3 2.1 62.9 2.3 2.1Number of lotteries 124 124Number of students 16,554 16,554

Panel C. High school graduationForecast coefficient 0.333 0.905 0.914 0.329 0.893 0.901

(0.063) (0.076) (0.076) (0.064) (0.093) (0.094)p-value 0.000 0.214 0.262 0.000 0.245 0.295

Overid. χ 2 (123) stat. 205.1 145.6 147.7 205.1 145.6 147.7p-value 0.000 0.080 0.064 0.000 0.080 0.064

First-stage F-stat. 92.1 5.4 5.0 92.1 5.4 5.0Number of lotteries 124 124Number of students 32,131 32,131

Notes: This table reports the results of lottery-based tests for bias in estimates of school effectiveness. The sample is restricted to students who have nondegenerate risk for at least one school and lotteries with 100 or more students at risk. Students are considered to have risk at a given school if their propensity score is strictly between 0 and 1 and they are in a score cell with variation in school offers. Columns 1 and 4 measure school effectiveness as the school mean outcome, columns 2 and 5 use value-added estimates, and columns 3 and 6 use control function estimates. Forecast coefficients and overidentification tests in columns 1–3 come from two-stage least squares regressions of test scores on the fitted values from the non-lottery estimation procedure, instrumenting with school-specific lot-tery offer indicators and controlling for school-specific propensity scores. Columns 4–6 use the Unbiased Jackknife Instrumental Variables (UJIVE) estimator of Kolesár (2013) instead of 2SLS.

1526 THE AMERICAN ECONOMIC REVIEW MAY 2020

likely to have low power. Taken together, the results in Table 7 suggest that our esti-mation strategies generate reliable measures of causal effects, though the available lottery variation may be insufficient to detect modest violations.

V. Preferences, Peer Quality, and School Effectiveness

A. Productivity versus Peers

The last step of our analysis compares the relative strength of peer quality and school effectiveness as predictors of parent preferences. Table 8 reports estimates of equation (11) for Regents math scores, first including Q j ∗ and AT E j ∗ one at a time and then including both variables simultaneously. Mean utilities, peer quality, and treatment effects are scaled in standard deviations of their respective school-level distributions, so the estimates can be interpreted as the standard deviation change in mean utility associated with a 1 standard deviation increase in Q j or AT E j .

Bivariate regressions show that school popularity is positively correlated with both peer quality and school effectiveness. Results based on the OLS value-added model, reported in columns 1 and 2 of panel A, imply that a 1 standard deviation increase in Q j is associated with a 0.42 standard deviation increase in mean utility, while a 1 standard deviation increase in AT E j is associated with a 0.24 standard devi-ation increase in mean utility. The latter result contrasts with studies reporting no average test score impact of attending preferred schools (Cullen, Jacob, and Levitt 2006; Hastings, Kane, and Staiger 2009). These studies rely on admission lotteries that shift relatively small numbers of students across a limited range of schools. Our results show that looking across all high schools in New York City, more popular schools tend to be more effective on average.

While preferences are positively correlated with school effectiveness, however, this relationship is entirely explained by peer quality. Column 3 of panel A shows that when both variables are included together, the coefficient on peer quality is essentially unchanged, while the coefficient on the average treatment effect is ren-dered small and statistically insignificant. The AT E j coefficient also remains precise: we can rule out increases in mean utility on the order of 0.06 standard deviations associated with a 1 standard deviation change in school value-added at conventional significance levels. The control function estimates in columns 5–7 are similar to the value-added estimates, showing no association between school effectiveness and popularity after controlling for peer quality.

Columns 4 and 8 of Table 8 explore the role of treatment effect heterogeneity by adding posterior mean predictions of match quality to equation (11), also scaled in standard deviation units of the distribution of match effects across schools and cells.11 The match coefficient is negative for both the value-added and control func-tion models, and the control function estimate is statistically significant. This reflects the negative correlation between baseline test score slope coefficients and peer

11 Equation (11) captures the projection of cell mean utility on peer quality and cell mean treatment effects. In the control function model the projection of student-specific utility on student-specific treatment effects also includes the idiosyncratic match component φ η ij . Our small estimate of the matching coefficient φ implies this term is negligible, so we focus on relationships at the cell mean level for both the value-added and control function models.

1527ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

quality reported in Table 5: schools that are especially effective for low-achieving students tend to be more popular among high-achievers and therefore enroll more of these students despite their lower match quality. This is consistent with recent studies of selection into early-childhood programs and charter schools, which also find negative selection on test score match effects (Cornelissen et al. 2016, Kline and Walters 2016, Walters 2018).

Correlations between popularity and effectiveness may fail to capture the causal effects of school effectiveness on parent demand if other determinants of par-ent preferences are correlated with school effectiveness.12 It’s worth noting that for omitted variables bias to explain our finding that parents do not place positive weight on effectiveness conditional on peer quality, an omitted amenity that parents value would need to be negatively correlated with effectiveness after controlling for Q j . Since we might expect any omitted variables to be positively correlated with both effectiveness and demand (as is the case with peer quality itself), this sort of selection bias seems implausible. Nevertheless, Panel B of Table 8 investigates

12 An analogous problem arises in studies of teacher and school value-added, which often analyze relationships between teacher or school effects and observed characteristics without quasi-experimental variation in the charac-teristics (see, e.g., Kane, Rockoff, and Staiger 2008 and Angrist, Pathak, and Walters 2013).

Table 8—Preferences for Peer Quality and Regents Math Effects

Value-added models Control function models

(1) (2) (3) (4) (5) (6) (7) (8)Panel A. No controls for school characteristicsPeer quality 0.416 0.438 0.406 0.407 0.439 0.437

(0.061) (0.063) (0.067) (0.057) (0.059) (0.059)ATE 0.244 −0.033 −0.022 0.219 −0.051 −0.047

(0.047) (0.046) (0.047) (0.046) (0.043) (0.043)Match effect −0.072 −0.172

(0.047) (0.054)

Observations 21,684

Panel B. With controls for school characteristicsPeer quality 0.310 0.314 0.286 0.299 0.303 0.308

(0.060) (0.059) (0.060) (0.056) (0.056) (0.056)ATE 0.157 −0.005 0.005 0.144 −0.008 −0.003

(0.042) (0.039) (0.040) (0.040) (0.035) (0.035)Match effect −0.068 −0.142

(0.039) (0.044)

Observations 20,200

Notes: This table reports estimates from regressions of school popularity on peer quality and school effectiveness. School popularity is measured as the estimated mean utility for each school and covariate cell in the choice model from Table 4. Covariate cells are defined by borough, gender, race, subsidized lunch status, an indicator for stu-dents above the median of census tract median income, and tercile of the average of eighth grade math and reading scores. Peer quality is constructed as the average predicted Regents math score for enrolled students. Treatment effect estimates are empirical Bayes posterior mean predictions of Regents math effects. Mean utilities, peer quality, and treatment effects are scaled in standard deviation units. Columns 1–4 report results from value-added models, while columns 5–8 report results from control function models. All regressions include cell indicators and weight by the inverse of the squared standard error of the mean utility estimates. Panel A includes no additional controls, while panel B controls for the school environment score, violent and disruptive incidents per student, and percent of teachers with master’s degrees. Standard errors are double-clustered by school and covariate cell.

1528 THE AMERICAN ECONOMIC REVIEW MAY 2020

the potential for such omitted variable bias by adding controls for other important school attributes to equation (11). The sensitivity of regression coefficients to con-trols for observables is a common diagnostic for assessing the scope for selection on unobservables (Altonji, Elder, and Taber 2005; Oster 2019).

We explore the impact of controlling for three school covariates. The first is a measure of the quality of the school environment derived from New York City’s Learning Environments Survey (New York City Department of Education 2008). This survey is taken each year by New York City students in grades 6 through 12 as well as parents and teachers. We construct an overall school environment score by taking the first principle component of the Safety and Respect, Communication, Engagement, and Academic Expectations summary measures from the school sur-vey. Second, we add a measure of violent and disruptive incidents (VADI) per stu-dent reported by the New York State Department of Education (New York State Department of Education 2007b). Finally, we control for the fraction of teachers with master’s degrees as reported on New York school report cards distributed between 2005 and 2007 (New York State Department of Education 2007a). The results in panel B of Table 8 show that our main conclusions are unaffected by the addition of these control variables: the coefficient on Q j remains large and statisti-cally insignificant, while the coefficient on AT E j remains close to zero.13 While we cannot control for all unobserved factors that influence preferences for schools, the robustness of our results to controls for observed characteristics suggests that our key findings are unlikely to be driven by omitted variable bias.

Figure 3 presents a graphical summary of the links among preferences, peer quality, and treatment effects by plotting bivariate and multivariate relationships between mean utility (averaged across covariate cells) and posterior predictions of Q j and AT E j from the control function model. Panel A shows strong positive bivariate correlations for both variables. Panel B plots mean utilities against residuals from a regression of Q j ∗ on AT E j ∗ ( left-hand panel) and residuals from a regression of AT E j ∗ on Q j ∗ ( right-hand panel). Adjusting for school effectiveness has little effect on the relationship between preferences and peer quality. In contrast, partialing out peer quality eliminates the positive association between popularity and effectiveness.

B. Preferences and Effects on Longer-Run Outcomes

Parents may care about treatment effects on outcomes other than short-run stan-dardized test scores. We explore this by estimating equation (11) for PSAT scores, high school graduation, college attendance, and log college quality.

Results for these outcomes are similar to the findings for Regents math scores: preferences are positively correlated with average treatment effects in a bivariate sense but are uncorrelated with treatment effects conditional on peer quality. Table 9 reports results based on control function estimates of treatment effects. The magni-tudes of all treatment effect coefficients are small, and the overall pattern of results suggests no systematic relationship between preferences and school effectiveness

13 Online Appendix Table A8 reports the coefficients on the control variables. These estimates show that parents prefer schools with fewer violent incidents and those with more teachers with master’s degrees, while the school environment score is uncorrelated with demand.

1529ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

conditional on peer composition. We find a modest positive relationship between preferences and match effects for log college quality, but corresponding estimates for PSAT scores, high school graduation, and college attendance are small and sta-tistically insignificant. This pattern contrasts with results for the Norwegian higher education system, reported by Kirkeboen, Leuven, and Mogstad (2016), which show sorting into fields of study based on heterogeneous earnings gains. Unlike Norwegian college students, New York City’s high school students do not prefer schools with higher academic match quality.

C. Heterogeneity in Preferences for Peer and School Quality

Previous evidence suggests that parents of higher-income, higher-achieving stu-dents place more weight on academic performance levels when choosing schools (Hastings, Kane, and Staiger 2009). This pattern may reflect either greater respon-siveness to peer quality or more sensitivity to causal school effectiveness. If parents of high-achievers value school effectiveness, choice may indirectly create incentives

Panel A. Bivariate relationships

Panel B. Multivariate relationships

Mea

n ut

ility

0 0.25 0.5 0.75 1 1.25

Peer quality

−4

−2

0

2

4

Mea

n ut

ility

−0.25−0.75−0.5 0 0.25 0.5 0.75 1

Regents math effect

−2

−4

0

2

4

Mea

n ut

ility

Peer quality

Mea

n ut

ility

Regents math effect

−4

−2

0

2

4

−4

−2

0

2

4

−0.25−0.5

−0.25−0.5 0 0.25 0.5 0.75 1 −0.25−0.75 −0.5 0 0.25 0.5 0.75 1

−2

−4

0

2

4

Figure 3. Relationships among Preferences, Peer Quality, and Regents Math Effects

Notes: This figure plots school mean utility estimates against estimates of peer quality and Regents math average treatment effects. Mean utilities are school average residuals from a regression of school-by-covariate cell mean utility estimates on cell indicators. Peer quality is defined as the average predicted Regents math score for enrolled students. Regents math effects are empirical Bayes posterior mean estimates of school average treatment effects from control function models. The left plot in panel A displays the bivariate relationship between mean utility and per quality, while the right plot shows the bivariate relationship between mean utility and Regents math effects. The left plot in panel B displays the relationship between mean utility and residuals from a regression of peer quality on Regents math effects, while the right plot shows the relationship between mean utility and residuals from a regres-sion of Regents math effects on peer quality. Dashed lines are ordinary least squares regression lines.

1530 THE AMERICAN ECONOMIC REVIEW MAY 2020

for schools to improve because better instruction will attract high-ability students, raising peer quality and therefore demand from other households. In Table 10 we investigate this issue by estimating equation (11) separately by race, subsidized lunch status, and baseline test score category.

We find that no subgroup of households responds to causal school effectiveness. Consistent with previous work, we find larger coefficients on peer quality among nonminority students, richer students (those ineligible for subsidized lunches), and students with high baseline achievement. We do not interpret this as direct evidence of stronger preferences for peer ability among higher-ability students; since stu-dents are more likely to enroll at schools they rank highly, any group component to preferences will lead to a positive association between students’ rankings and the enrollment share of others in the same group.14 The key pattern in Table 10 is that, among schools with similar peer quality, no group prefers schools with greater causal impacts on academic achievement.

D. Changes in Demand over Time

If parents do not have perfect information about school quality we might expect changes in demand over time as parents learn more about which schools are effec-tive. The evolution of choice behavior is of particular interest in our sample since New York City changed from an uncoordinated assignment process to a coordinated single-offer system in 2003, the first year of our data (Abdulkadiroglu, Agarwal, and Pathak 2017). Online Appendix Table A9 assesses whether parents systemati-cally select more effective schools over time by reporting estimates of equation (11) based on preference models fit separately for each of the four applicant cohorts in

14 This is a version of the “reflection problem” that plagues econometric investigations of peer effects (Manski 1993).

Table 9—Preferences for Peer Quality and School Effectiveness by Outcome

PSAT score High school graduation College attendance log college quality

(1) (2) (3) (4) (5) (6) (7) (8)Peer quality 0.467 0.430 0.235 0.322

(0.070) (0.070) (0.054) (0.065)ATE 0.325 −0.092 0.103 −0.174 0.273 0.132 0.199 0.029

(0.056) (0.074) (0.045) (0.054) (0.048) (0.054) (0.059) (0.080)Match effect −0.049 −0.065 −0.017 0.053

(0.047) (0.044) (0.050) (0.061)

Observations 21,684

Notes: This table reports estimates from regressions of school popularity on peer quality and school effectiveness separately by outcome. School popularity is measured as the estimated mean utility for each school and covariate cell in the choice model from Table 4. Covariate cells are defined by borough, gender, race, subsidized lunch sta-tus, an indicator for students above the median of census tract median income, and tercile of the average of eighth grade math and reading scores. Peer quality is constructed as the average predicted outcome for enrolled students. Treatment effect estimates are empirical Bayes posterior mean predictions from control function models. Mean utilities, peer quality, and treatment effects are scaled in standard deviation units. All regressions include cell indi-cators and weight by the inverse of the squared standard error of the mean utility estimates. Standard errors are dou-ble-clustered by school and covariate cell.

1531ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

our sample. The results reveal remarkably stable patterns of choice: in each year the coefficient on peer quality is large and positive, the coefficient on the average treatment effect is a precise zero, and the match effect coefficient is zero or negative. Evidently, more experience with the centralized matching process did not lead to a stronger relationship between preferences and effectiveness for parents in New York City. This suggests that either parents do not learn much about school effectiveness over time, or the patterns we identify reflect preferences for other school attributes rather than a lack of information about effectiveness.

E. Alternative Specifications

We investigate the robustness of our key results by estimating a variety of alterna-tive specifications, reported in online Appendix Tables A10 and A11. To assess the sensitivity of our estimates to reasonable changes in our measure of school popular-ity, columns 1–4 of online Appendix Table A10 display results from models replac-ing δ ˆ cj in equation (11) with the log share of students in a cell ranking a school first or minus the log sum of ranks in the cell (treating unranked schools as tied). These alternative measures of demand produce very similar results to the rank-ordered logit results in Table 8.

Estimates based on students’ submitted rankings may not accurately describe demand if students strategically misreport their preferences in response to the 12-choice constraint on list length. As noted in Section I, truthful reporting is a dominant strategy for the 72 percent of students who list fewer than 12 choices. Columns 5 and 6 of online Appendix Table A10 report results based on rank-ordered logit models estimated in the subsample of unconstrained students. Results here are again similar to the full sample estimates, suggesting that strategic misreporting is not an important concern in our setting.

Table 10—Heterogeneity in Preferences for Peer Quality and Regents Math Effects

By race By subsidized lunch By eighth grade test score tercile

Black Hispanic Other Eligible Ineligible Lowest Middle Highest(1) (2) (3) (4) (5) (6) (7) (8)

Peer quality 0.396 0.370 0.705 0.410 0.501 0.251 0.395 0.686(0.060) (0.063) (0.128) (0.057) (0.077) (0.055) (0.062) (0.092)

ATE −0.047 −0.011 −0.192 −0.036 −0.076 −0.015 −0.029 −0.117(0.045) (0.044) (0.094) (0.042) (0.050) (0.042) (0.042) (0.059)

Match effect −0.200 −0.144 −0.149 −0.180 −0.155 −0.166 −0.169 −0.125(0.056) (0.066) (0.061) (0.054) (0.054) (0.061) (0.058) (0.055)

Observations 7,467 7,433 6,784 11,043 10,641 7,264 7,286 7,134

Notes: This table reports estimates from regressions of school popularity on peer quality and school effectiveness separately by student subgroup. School popularity is measured as the estimated mean utility for each school and covariate cell in the choice model from Table 4. Peer quality is constructed as the average predicted Regents math score for enrolled students. Treatment effect estimates are empirical Bayes posterior mean predictions of Regents math effects from control function models. Mean utilities, peer quality, and treatment effects are scaled in standard deviation units. Peer quality is constructed as the average predicted Regents math score for enrolled students. All regressions include cell indicators and weight by the inverse of the squared standard error of the mean utility esti-mates. Standard errors are double-clustered by school and covariate cell.

1532 THE AMERICAN ECONOMIC REVIEW MAY 2020

Our preference estimation approach models students’ choices among all schools in their home boroughs. Students may be unaware of some schools and therefore consider only a subset of the available alternatives. A conservative approach to defin-ing consideration sets is to assume students are only aware of the schools ranked on their preference lists. Columns 7 and 8 of online Appendix Table A10 show results based on preference estimates that omit all unranked alternatives from the choice set. This approach produces similar estimates as well.

Equation (8) parameterizes the relationship between potential outcomes and preference rankings through the control functions λ k ( · ) . Columns 1–4 of online Appendix Table A11 present an alternative parameterization that replaces the con-trol functions with fixed effects for first choice schools. This approach ignores infor-mation on lower-ranked schools but more closely parallels the application portfolio matching approach in Dale and Krueger (2002, 2014). As a second alternative spec-ification, columns 5–8 report estimates from a control function model that drops the distance control variables from equation (8). This model relies on an exclusion restriction for distance, a common identification strategy in the literature on educa-tional choice (Card 1995, Neal 1997, Booker et al. 2011, Mountjoy 2017, Walters 2018). These alternative approaches to estimating school effectiveness produce no meaningful changes in the results.

VI. Discussion

The findings reported here inform models of school choice commonly considered in the literature. Theoretical analyses often assume parents know students’ poten-tial achievement outcomes and choose between schools on this basis. For example, Epple, Figlio, and Romano (2004) and Epple and Romano (2008) study models in which parents value academic achievement and consumption of other goods, and care about peer quality only insofar as it produces higher achievement through peer effects. Hoxby (2000) argues that school choice may increase achievement by allowing students to sort on match quality. Such models imply that demand should be positively correlated with both average treatment effects and match effects con-ditional on peer quality, a prediction that is inconsistent with the pattern in Table 8.

Parents may choose between schools based on test score levels rather than treat-ment effects. Cullen, Jacob, and Levitt (2006) suggests confusion between levels and gains may explain limited effects of admission to preferred schools in Chicago. Since our setting has substantial variation in both levels and value-added, we can more thoroughly investigate this model of parent decision-making. If parents choose between schools based on average outcomes, increases in these outcomes due to selection and causal effectiveness should produce equal effects on popularity. In contrast, we find that demand only responds to the component of average outcomes that is due to enrollment of higher-ability students. That is, we can reject the view that parental demand is driven by performance levels: demand places no weight on the part of performance levels explained by value-added but significant weight on the part explained by peer quality.

It is important to note that our findings do not imply parents are uninterested in school effectiveness. Without direct information about treatment effects, for exam-ple, parents may use peer characteristics as a proxy for school quality, as in MacLeod

1533ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

and Urquiola (2015). In view of the positive correlation between peer quality and school effectiveness, this is a reasonable strategy for parents that cannot observe treatment effects and wish to choose effective schools. Effectiveness varies widely conditional on peer quality, however, so parents make substantial sacrifices in aca-demic quality by not ranking schools based on effectiveness. Table 11 compares Regents math effects for observed preference rankings versus hypothetical rankings in which parents order schools according to their effectiveness. The average treat-ment effect of first-choice schools would improve from 0.07σ to 0.43σ if parents ranked schools based on effectiveness, and the average match effect would increase from − 0.04σ to 0.16σ . This implies that the average student loses more than one-half of a standard deviation in math achievement by enrolling in her first-choice school rather than the most effective option.

The statistics in Table 11 suggest that if information frictions prevent parents from ranking schools based on effectiveness, providing information about school effec-tiveness could alter school choices considerably. These changes may be particularly valuable for disadvantaged students. As shown in online Appendix Table A12, gaps in effectiveness between observed first-choice schools and achievement-maximizing choices are larger for students with lower baseline achievement. This is driven by the stronger relationship between peer quality and preferences for more-advantaged parents documented in Table 10. These results suggest reducing information barri-ers could lead to differential increases in school quality for disadvantaged students and reduce inequality in student achievement. On the other hand, the patterns doc-umented here may also reflect parents’ valuation of school amenities other than academic effectiveness rather than a lack of information about treatment effects. It is also important to note that our estimates capture average impacts of changing an individual student’s rankings, holding fixed the behavior of other students; we might expect schools’ treatment effect parameters to change if all students changed behav-ior simultaneously due to changes in peer effects or other inputs.

Regardless of why parents respond to peer quality rather than school effective-ness, our results have important implications for the incentive effects of school choice programs. Since parents only respond to the component of school average outcomes that can be predicted by the ability of enrolled students, our estimates imply a school wishing to boost its popularity must recruit better students; improv-ing outcomes by increasing causal effectiveness for a fixed set of students will have no impact on parent demand. Our results therefore suggest that choice may create incentives for schools to invest in screening and selection.

The evolution of admissions criteria used at New York City’s high schools is consistent with the implication that schools have an increased incentive to screen applicants due to parents’ demand for high-ability peers. After the first year of the new assignment mechanism, several school programs eliminated all lottery-based admissions procedures and became entirely screened. In the 2003–2004 high school brochure, 36.8 percent of programs are screened, and this fraction jumps to 40.3 percent two years later. The Beacon High School in Manhattan, for exam-ple, switched from a school where one-half of the seats were assigned via random lottery in 2003–2004 to a screened school the following year, where admissions is based on test performance, an interview, and a portfolio of essays. Leon Goldstein High School for Sciences in Brooklyn underwent a similar transition. Both high

1534 THE AMERICAN ECONOMIC REVIEW MAY 2020

schools frequent lists of New York City’s best public high schools.15 Compared to the first years of the new system, there has also been growth in the number of limited unscreened programs, which use a lottery but also give priority to students who attend an open house or high school fair. Compared to unscreened programs, prioritizing applicants who attend an information session provides an ordeal that favors applicants with time and resources thus resulting in positive selection.16 The number of limited unscreened programs nearly doubled from 106 to 210 from 2005 to 2012 (Nathanson, Corcoran, and Baker-Smith 2013).

VII. Conclusion

A central motivation for school choice programs is that parents’ choices gen-erate demand-side pressure for improved school productivity. We investigate this possibility by comparing estimates of school popularity and treatment effects based on rank-ordered preference data for applicants to public high schools in New York City. Parents prefer schools that enroll higher-achieving peers. Conditional on peer quality, however, parents’ choices are unrelated to causal school effectiveness. Moreover, no subgroup of parents systematically responds to causal school effec-tiveness. We also find no relationship between preferences for schools and estimated match quality. This indicates that choice does not lead students to sort into schools on the basis of comparative advantage in academic achievement.

15 Mary Kay Linge and Joshua Tanzer, “The Top 40 Public High Schools in NYC,” New York Post, September 17, 2016, https://nypost.com/2016/09/17/the-top-40-public-high-schools-in-nyc/.

16 Monica Disare, “City to Eliminate High School Admissions Method That Favored Families with Time and Resources,” Chalkbeat, June 6, 2017, https://www.chalkbeat.org/posts/ny/2017/06/06/city-to-eliminate-high-school-admissions-method-that-favored-families-with-time-and-resources/ (accessed December 2017).

Table 11—Potential Achievement Gains from Ranking Schools by Effectiveness

Observed rankings Rankings based on effectiveness

Peer quality ATE Match

Peer quality ATE Match

(1) (2) (3) (4) (5) (6)

Choice 1 0.112 0.071 −0.037 0.286 0.427 0.162Choice 2 0.057 0.055 −0.020 0.182 0.352 0.108Choice 3 0.021 0.045 −0.012 0.087 0.275 0.113Choice 4 −0.013 0.036 −0.006 0.105 0.247 0.103Choice 5 −0.046 0.027 −0.002 0.124 0.228 0.092Choice 6 −0.074 0.019 −0.001 0.103 0.209 0.085Choice 7 −0.097 0.014 0.001 0.118 0.197 0.075Choice 8 −0.114 0.012 0.001 0.099 0.169 0.066Choice 9 −0.127 0.007 0.001 0.064 0.333 0.111Choice 10 −0.139 0.004 0.003 0.046 0.165 0.063Choice 11 −0.146 0.003 0.003 0.028 0.157 0.056Choice 12 −0.156 −0.002 0.002 0.013 0.146 0.053

Notes: This table summarizes Regents math score gains that parents could achieve by rank-ing schools based on effectiveness. Columns 1–3 report average peer quality, average treat-ment effects, and average match quality for students’ observed preference rankings. Columns 4–6 display corresponding statistics for hypothetical rankings that list schools in order of their treatment effects. Treatment effect estimates come from control function models. All calcula-tions are restricted to ranked schools within the home borough.

1535ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

This pattern of findings has important implications for the expected effects of school choice programs. Our results on match quality suggest choice is unlikely to increase allocative efficiency. Our findings regarding peer quality and average treatment effects suggest choice may create incentives for increased screening rather than academic effectiveness. If parents respond to peer quality but not causal effects, a school’s easiest path to boosting its popularity is to improve the ability of its student population. Since peer quality is a fixed resource, this creates the potential for socially costly zero-sum competition as schools invest in mechanisms to attract the best students. MacLeod and Urquiola (2015) argues that restricting a school’s ability to select pupils may promote efficiency when student choices are based on school reputation. The impact of school choice on effort devoted to screening is an important empirical question for future research.

While we have shown that parents do not choose schools based on causal effects for a variety of educational outcomes, we cannot rule out the possibility that pref-erences are determined by effects on unmeasured outcomes. Our analysis also does not address why parents put more weight on peer quality than on treatment effects. If parents rely on student composition as a proxy for effectiveness, coupling school choice with credible information on causal effects may strengthen incentives for improved productivity and weaken the association between preferences and peer ability. Distinguishing between true tastes for peer quality and information frictions is another challenge for future work.

REFERENCES

Abdulkadiroglu, Atila, Nikhil Agarwal, and Parag A. Pathak. 2017. “The Welfare Effects of Coordi-nated Assignment: Evidence from the New York City High School Match.” American Economic Review 107 (12): 3635–89.

Abdulkadiroglu, Atila, Joshua D. Angrist, Yusuke Narita, and Parag A. Pathak. 2017. “Research Design Meets Market Design: Using Centralized Assignment for Impact Evaluation.” Economet-rica 85 (5): 1373–1432.

Abdulkadiroglu, Atila, Joshua D. Angrist, Yusuke Narita, and Parag A. Pathak. 2019. “Breaking Ties: Regression Discontinuity Design Meets Market Design.” Cowles Foundation Discussion Paper 2170.

Abdulkadiroglu, Atila, Joshua D. Angrist, and Parag A. Pathak. 2014. “The Elite Illusion: Achieve-ment Effects at Boston and New York Exam Schools.” Econometrica 82 (1): 137–96.

Abdulkadiroglu, Atila, Parag A. Pathak, and Alvin E. Roth. 2005. “The New York City High School Match.” American Economic Review 95 (2): 364–67.

Abdulkadiroglu, Atila, Parag A. Pathak, and Alvin E. Roth. 2009. “Strategy-Proofness versus Effi-ciency in Matching with Indifferences: Redesigning the NYC High School Match.” American Eco-nomic Review 99 (5): 1954–78.

Abdulkadiroglu, Atila, Parag A. Pathak, Jonathan Schellenberg, and Christopher R. Walters. 2020. “Replication Data for: Do Parents Value School Effectiveness?” American Economic Association [publisher], Inter-university Consortium for Political and Social Research [distributor]. https://doi.org/10.3886/E112266V1.

Abdulkadiroglu, Atila, Parag A. Pathak, and Christopher R. Walters. 2018. “Free to Choose: Can School Choice Reduce Student Achievement?” American Economic Journal: Applied Economics 10 (1): 175–206.

Agarwal, Nikhil, and Paulo Somaini. 2018. “Demand Analysis Using Strategic Reports: An Applica-tion to a School Choice Mechanism.” Econometrica 86 (2): 391–444.

Altonji, Joseph G., Todd E. Elder, and Christopher R. Taber. 2005. “Selection on Observed and Unob-served Variables: Assessing the Effectiveness of Catholic Schools.” Journal of Political Economy 113 (1): 151–84.

Altonji, Joseph G., Ching-I Huang, and Christopher R. Taber. 2015. “Estimating the Cream Skimming Effect of School Choice.” Journal of Political Economy 123 (2): 266–324.

1536 THE AMERICAN ECONOMIC REVIEW MAY 2020

Angrist, Joshua D., Sarah R. Cohodes, Susan M. Dynarski, Parag A. Pathak, and Christopher R. Wal-ters. 2016a. “Stand and Deliver: Effects of Boston’s Charter High Schools on College Preparation, Entry, and Choice.” Journal of Labor Economics 34 (2): 275–318.

Angrist, Joshua D., Peter D. Hull, Parag A. Pathak, and Christopher R. Walters. 2016b. “Interpreting Tests of School VAM Validity.” American Economic Review 106 (5): 388–92.

Angrist, Joshua D., Peter D. Hull, Parag A. Pathak, and Christopher R. Walters. 2017. “Leverag-ing Lotteries for School Value-Added: Testing and Estimation.” Quarterly Journal of Economics 132 (2): 871–919.

Angrist, Joshua D., Parag A. Pathak., and Christopher R. Walters. 2013. “Explaining Charter School Effectiveness.” American Economic Journal: Applied Economics 5 (4): 1–27.

Avery, Christopher, and Parag A. Pathak. 2015. “The Distributional Consequences of Public School Choice.” NBER Working Paper 21525.

Bacher-Hicks, Andrew, Thomas J. Kane, and Douglas O. Staiger. 2014. “Validating Teacher Effect Estimates Using Changes in Teacher Assignments in Los Angeles.” Unpublished.

Barseghyan, Levon, Damon Clark, and Stephen Coate. 2014. “Public School Choice: An Economic Analysis.” NBER Working Paper 20701.

Bayer, Patrick, Fernando Ferreira, and Robert McMillan. 2007. “A Unified Framework for Measur-ing Preferences for Schools and Neighborhoods.” Journal of Political Economy 115 (4): 588–638.

Beuermann, Diether, C. Kirabo Jackson, Laia Navarro-Sola, and Francisco Pardo. 2018. “What Is a Good School, and Can Parents Tell? Evidence on the Multidimensionality of School Output.” NBER Working Paper 25342.

Black, Sandra E. 1999. “Do Better Schools Matter? Parental Valuation of Elementary Education.” Quarterly Journal of Economics 114 (2): 577–99.

Blinder, Alan S. 1973. “Wage Discrimination: Reduced Form and Structural Estimates.” Journal of Human Resources 8 (4): 436–55.

Blundell, Richard, and Rosa L. Matzkin. 2014. “Control Functions in Nonseparable Simultaneous Equations Models.” Quantitative Economics 5 (2): 271–95.

Booker, Kevin, Tim R. Sass, Brian Gill, and Ron Zimmer. 2011. “The Effects of Charter High Schools on Educational Attainment.” Journal of Labor Economics 29 (2): 377–415.

Burgess, Simon, Ellen Greaves, Anna Vignoles, and Deborah Wilson. 2015. “What Parents Want: School Preferences and School Choice.” Economic Journal 125 (587): 1262–89.

Cameron, A. Colin, Jonah B. Gelbach, and Douglas L. Miller. 2011. “Robust Inference with Multiway Clustering.” Journal of Business and Economic Statistics 29 (2): 238–49.

Card, David. 1995. “Using Geographic Variation in College Proximity to Estimate the Return to School-ing.” In Aspects of Labour Market Behaviour: Essays in Honour of John Vanderkamp, edited by Louis N. Christofides, E. Kenneth Grant, and Robert Swidinsky. Toronto: University of Toronto Press.

Chetty, Raj, John N. Friedman, Nathaniel Hilger, Emmanuel Saez, Diane Whitmore Schanzenbach, and Danny Yagan. 2011. “How Does Your Kindergarten Classroom Affect Your Earnings? Evi-dence from Project STAR.” Quarterly Journal of Economics 126 (4): 1593–1660.

Chetty, Raj, John N. Friedman, and Jonah E. Rockoff. 2014a. “Measuring the Impact of Teachers I: Eval-uating Bias in Teacher Value-Added Estimates.” American Economic Review 104 (9): 2593–2632.

Chetty, Raj, John N. Friedman, and Jonah E. Rockoff. 2014b. “Measuring the Impact of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood.” American Economic Review 104 (9): 2633–79.

Chetty, Raj, John N. Friedman, and Jonah Rockoff. 2016. “Using Lagged Outcomes to Evaluate Bias in Value-Added Models.” American Economic Review 106 (5): 393–99.

Chetty, Raj, John N. Friedman, and Jonah E. Rockoff. 2017. “Measuring the Impacts of Teachers: Reply.” American Economic Review 107 (6): 1685–1717.

Chetty, Raj, John N. Friedman, Emmanuel Saez, Nicholas Turner, and Danny Yagan. 2017. “Mobil-ity Report Cards: The Role of Colleges in Intergenerational Mobility.” The Equality of Opportunity Project, January.

Chetty, Raj, and Nathaniel Hendren. 2017. “The Impacts of Neighborhoods on Intergenerational Mobility II: County-Level Estimates.” NBER Working Paper 23002.

Chubb, John E., and Terry M. Moe. 1990. Politics, Markets, and America’s Schools. Washington, DC: Brookings Institution Press.

Cornelissen, Thomas, Christian Dustmann, Anna Raute, and Uta Schönberg. 2016. “Who Benefits from Universal Child Care? Estimating Marginal Returns to Early Child Care Attendance.” Unpublished.

Cullen, Julie Berry, Brian A. Jacob, and Steven Levitt. 2006. “The Effect of School Choice on Partic-ipants: Evidence from Randomized Lotteries.” Econometrica 74 (5): 1191–1230.

Dahl, Gordon B. 2002. “Mobility and the Return to Education: Testing a Roy Model with Multiple Markets.” Econometrica 70 (6): 2367–2420.

1537ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

Dale, Stacy Berg, and Alan B. Krueger. 2002. “Estimating the Payoff to Attending a More Selective College: An Application of Selection on Observables and Unobservables.” Quarterly Journal of Economics 117 (4): 1491–1527.

Dale, Stacy Berg, and Alan B. Krueger. 2014. “Estimating the Effects of College Characteristics over the Career Using Administrative Earnings Data.” Journal of Human Resources 49 (2): 323–58.

Deming, David J. 2014. “Using School Choice Lotteries to Test Measures of School Effectiveness.” American Economic Review 104 (5): 406–11.

Deming, David J., Justine S. Hastings, Thomas J. Kane, and Douglas O. Staiger. 2014. “School Choice, School Quality, and Postsecondary Attainment.” American Economic Review 104 (3): 991–1013.

DeVos, Betsy. 2017. Comments at Brookings Institution event on the 2016 Education Choice and Competition Index, March 29. https://www.brookings.edu/events/the-2016-education-choice-and-competition-index.

Dobbie, Will, and Roland G. Fryer. 2014. “The Impact of Attending a School with High-Achieving Peers: Evidence from the New York City Exam Schools.” American Economic Journal: Applied Economics 6 (3): 58–75.

Dubin, Jeffrey A., and Daniel L. McFadden. 1984. “An Econometric Analysis of Residential Electric Appliance Holdings and Consumption.” Econometrica 52 (2): 345–62.

Dubins, L. E., and D. A. Freedman. 1981. “Machiavelli and the Gale-Shapley Algorithm.” American Mathematical Monthly 88 (7): 485–94.

Dynarski, Susan, Joshua Hyman, and Diane Whitmore Schanzenbach. 2013. “Experimental Evidence on the Effect of Childhood Investments on Postsecondary Attainment and Degree Completion.” Journal of Policy Analysis and Management 32 (4): 692–717.

Epple, Dennis, David N. Figlio, and Richard Romano. 2004. “Competition between Private and Pub-lic Schools: Testing Stratification and Pricing Predictions.” Journal of Public Economics 88 (7–8): 1215–45.

Epple, Dennis, and Richard Romano. 1998. “Competition between Private and Public Schools, Vouch-ers, and Peer-Group Effects.” American Economic Review 88 (1): 33–62.

Epple, Dennis, and Richard Romano. 2008. “Educational Vouchers and Cream Skimming.” Interna-tional Economic Review 49 (4): 1395–1435.

Fack, Gabrielle, Julien Grenet, and Yinghua He. 2015. “Beyond Truth-Telling: Preference Estimation with Centralized School Choice.” Paris School of Economics Working Paper.

Figlio, David N., and Maurice E. Lucas. 2004. “What’s in a Grade? School Report Cards and the Hous-ing Market.” American Economic Review 94 (3): 591–604.

Finkelstein, Amy, Matthew Gentzkow, Peter Hull, and Heidi Williams. 2017. “Adjusting Risk Adjust-ment: Accounting for Variation in Diagnostic Intensity.” New England Journal of Medicine 376: 608–10.

French, Eric, and Christopher Taber. 2011. “Identification of Models of the Labor Market.” In Hand-book of Labor Economics, Vol. 4A, edited by Orley Ashenfelter and David Card, 537–617. Amster-dam: Elsevier.

Friedman, Milton. 1962. Capitalism and Freedom. Chicago: University of Chicago Press.Gale, D., and L. S. Shapley. 1962. “College Admissions and the Stability of Marriage.” American

Mathematical Monthly 69 (1): 9–15.Glazerman, Steven, and Dallas Dotter. 2016. “Market Signals: Evidence on the Determinants and

Consequences of School Choice from a Citywide Lottery.” Mathematica Policy Research Work-ing Paper 45.

Guarino, Cassandra M., Mark D. Reckase, and Jeffrey M. Wooldridge. 2015. “Can Value-Added Mea-sures of Teacher Performance Be Trusted?” Education Finance and Policy 10 (1): 117–56.

Haeringer, Guillaume, and Flip Klijn. 2009. “Constrained School Choice.” Journal of Economic The-ory 144 (5): 1921–47.

Hanushek, Eric A. 1981. “Throwing Money at Schools.” Journal of Policy Analysis and Manage-ment 1 (1): 19–41.

Harris, Douglas N., and Matthew Larsen. 2014. “What Schools Do Families Want (And Why)?” Tech-nical report, Education Research Alliance for New Orleans.

Hastings, Justine, Ali Hortaçsu, and Chad Syverson. 2017. “Sales Force and Competition in Finan-cial Product Markets: The Case of Mexico’s Social Security Privatization.” Econometrica 85 (6): 1723–61.

Hastings, Justine S., Thomas J. Kane, and Douglas O. Staiger. 2009. “Heterogeneous Preferences and the Efficacy of Public School Choice.” Unpublished.

Hastings, Justine S., and Jeffrey M. Weinstein. 2008. “Information, School Choice, and Academic Achievement: Evidence from Two Experiments.” Quarterly Journal of Economics 123 (4): 1373–1414.

1538 THE AMERICAN ECONOMIC REVIEW MAY 2020

Hausman, Jerry A. 1983. “Specification and Estimation of Simultaneous Equation Models.” In Hand-book of Econometrics, Vol. 1, edited by Zvi Griliches and Michael D. Intriligator, 391–448. New York: Elsevier.

Hausman, Jerry A., and Paul A. Ruud. 1987. “Specifying and Testing Econometric Models for Rank-Ordered Data.” Journal of Econometrics 34 (1–2): 83–104.

Heckman, James J., and Richard Robb. 1985. “Alternative Methods for Evaluating the Impact of Inter-ventions: An Overview.” Journal of Econometrics 30 (1–2): 239–67.

Heckman, James J., Sergio Urzua, and Edward Vytlacil. 2006. “Understanding Instrumental Variables in Models with Essential Heterogeneity.” Review of Economics and Statistics 88 (3): 389–432.

Heckman, James J., Sergio Urzua, and Edward Vytlacil. 2008. “Instrumental Variables in Models with Multiple Outcomes: The General Unordered Case.” Annales d’Économie et de Statistique (91–92): 151–74.

Hoxby, Caroline M. 2000. “Does Competition among Public Schools Benefit Students and Taxpay-ers?” American Economic Review 90 (5): 1209–38.

Hoxby, Caroline M. 2003. “School Choice and School Productivity: Could School Choice Be a Tide That Lifts All Boats?” In The Economics of School Choice, edited by Caroline M. Hoxby. Chicago: University of Chicago Press.

Hsieh, Chang-Tai, and Miguel Urquiola. 2006. “The Effects of Generalized School Choice on Achieve-ment and Stratification: Evidence from Chile’s Voucher Program.” Journal of Public Economics 90 (8–9): 1477–1503.

Hull, Peter. 2016. “Estimating Hospital Quality with Quasi-Experimental Data.” Unpublished.Imberman, Scott A., and Michael F. Lovenheim. 2016. “Does the Market Value Value-Added? Evi-

dence from Housing Prices after a Public Release of School and Teacher Value-Added.” Journal of Urban Economics 91: 104–21.

Jacob, Brian A., and Lars Lefgren. 2007. “What Do Parents Value in Education? An Empirical Inves-tigation of Parents’ Revealed Preferences for Teachers.” Quarterly Journal of Economics 122 (4): 1603–37.

Jacob, Brian A., and Lars Lefgren. 2008. “Can Principals Identify Effective Teachers? Evidence on Subjective Performance Evaluation in Education.” Journal of Labor Economics 26 (1): 101–36.

Kane, Thomas J., Daniel F. McCaffrey, Trey Miller, and Douglas O. Staiger. 2013. “Have We Identi-fied Effective Teachers? Validating Measures of Effective Teaching Using Random Assignment.” Gates Foundation Report.

Kane, Thomas J., Jonah E. Rockoff, and Douglas O. Staiger. 2008. “What Does Certification Tell Us about Teacher Effectiveness? Evidence from New York City.” Economics of Education Review 27 (6): 615–31.

Kane, Thomas J., and Douglas O. Staiger. 2002. “The Promise and Pitfalls of Using Imprecise School Accountability Measures.” Journal of Economic Perspectives 16 (4): 91–114.

Kapor, Adam, Christopher A. Neilson, and Seth D. Zimmerman. 2017. “Heterogeneous Beliefs and School Choice Mechanisms.” Unpublished.

Kirkeboen, Lars J., Edwin Leuven, and Magne Mogstad. 2016. “Field of Study, Earnings, and Self-Se-lection.” Quarterly Journal of Economics 131 (3): 1057–1111.

Kline, Patrick. 2011. “Oaxaca-Blinder as a Reweighting Estimator.” American Economic Review 101 (3): 532–37.

Kline, Patrick, and Andres Santos. 2012. “A Score Based Approach to Wild Bootstrap Inference.” Journal of Econometric Methods 1 (1): 23–41.

Kline, Patrick, and Christopher R. Walters. 2016. “Evaluating Public Programs with Close Substi-tutes: The Case of Head Start.” Quarterly Journal of Economics 131 (4): 1795–1848.

Koedel, Cory, Kata Mihaly, and Jonah E. Rockoff. 2015. “Value-Added Modeling: A Review.” Eco-nomics of Education Review 47: 180–95.

Kolesár, Michal. 2013. “Estimation in an Instrumental Variables Model with Treatment Effect Hetero-geneity.” Unpublished.

Ladd, Helen F. 2002. “Market-Based Reforms in Urban Education.” Unpublished.Langer, Ashley. 2016. “(Dis)Incentives for Demographic Price Discrimination in the New Vehicle

Market.” Unpublished.Lee, Lung-Fei. 1983. “Generalized Econometric Models with Selectivity.” Econometrica 51 (2): 507–

12. MacLeod, W. Bentley, Evan Riehl, Juan E. Saavedra, and Miguel Urquiola. 2017. “The Big Sort: Col-

lege Reputation and Labor Market Outcomes.” American Economic Journal: Applied Economics 9 (3): 223–61.

MacLeod, W. Bentley, and Miguel Urquiola. 2015. “Reputation and School Competition.” American Economic Review 105 (11): 3471–88.

1539ABDULKADIROGLU ET AL.: DO PARENTS VALUE SCHOOL EFFECTIVENESS?VOL. 110 NO. 5

Manski, Charles F. 1993. “Identification of Endogenous Social Effects: The Reflection Problem.” Review of Economic Studies 60 (3): 531–42.

Morris, Carl N. 1983. “Parametric Empirical Bayes Inference: Theory and Applications.” Journal of the American Statistical Association 78 (381): 47–55.

Mountjoy, Jack. 2017. “Community Colleges and Upward Mobility.” Unpublished.Nathanson, Lori, Sean P. Corcoran, and Christine Baker-Smith. 2013. “High School Choice in New

York City: A Report on the School Choices and Placements of Low-Achieving Students.” Research Alliance for New York City Schools, April.

Neal, Derek. 1997. “The Effects of Catholic Secondary Schooling on Educational Achievement.” Jour-nal of Labor Economics 15 (1): 98–123.

New York City Department of Education. 2003. “Directory of the New York City Public High Schools, 2003–2004.” New York.

New York City Department of Education. 2004. “2003–2004 Annual School Reports.” New York.New York City Department of Education. 2008. “NYC School Survey.” https://opendata.cityofnewyork.

us.New York City Department of Education. 2013. “Student Administrative Data System.” New York.New York City Department of Education. 2017. “2016–2017 School Quality Reports.” http://schools.

nyc.gov/accountability/tools/report/default.htm.New York State Department of Education. 2007a. “School Report Card Database.” https://data.nysed.

gov/downloads.php.New York State Department of Education. 2007b. “School Safety and Educational Climate Reports.”

http://www.p12.nysed.gov/irs/school_safety/vadir_archive.html.Oaxaca, Ronald. 1973. “Male-Female Wage Differentials in Urban Labor Markets.” International Eco-

nomic Review 14 (3): 693–709.Oster, Emily. 2019. “Unobservable Selection and Coefficient Stability: Theory and Evidence.” Journal

of Business and Economic Statistics 37 (2): 187–204.Pathak, Parag A., and Tayfun Sönmez. 2013. “School Admissions Reform in Chicago and England:

Comparing Mechanisms by Their Vulnerability to Manipulation.” American Economic Review 103 (1): 80–106.

Peterson, Paul E., and David E. Campbell. 2001. Charters, Vouchers, and Public Education. Washing-ton, DC: Brookings Institution Press.

Robbins, Herbert. 1956. “An Empirical Bayes Approach to Statistics.” In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1, edited by Jerzy Neyman, 157–63. Berkeley: University of California Press.

Rosenbaum, Paul R., and Donald B. Rubin. 1983. “The Central Role of the Propensity Score in Obser-vational Studies for Causal Effects.” Biometrika 70 (1): 41–55.

Roth, Alvin E. 1982. “The Economics of Matching: Stability and Incentives.” Mathematics of Opera-tions Research 7 (4): 617–28.

Rothstein, Jesse M. 2006. “Good Principals or Good Peers? Parental Valuation of School Characteris-tics, Tiebout Equilibrium, and the Incentive Effects of Competition among Jurisdictions.” American Economic Review 96 (4): 1333–50.

Rothstein, Jesse M. 2010. “Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement.” Quarterly Journal of Economics 125 (1): 175–214.

Rothstein, Jesse M. 2017. “Measuring the Impacts of Teachers: Comment.” American Economic Review 107 (6): 1656–84.

Roy, A. D. 1951. “Some Thoughts on the Distribution of Earnings.” Oxford Economic Papers 3 (2): 135–46.

Staiger, Douglas, and James H. Stock. 1997. “Instrumental Variables Regression with Weak Instru-ments.” Econometrica 65 (3): 557–86.

Todd, Petra E., and Kenneth I. Wolpin. 2003. “On the Specification and Estimation of the Production Function for Cognitive Achievement.” Economic Journal 113 (485): F3–F33.

Walters, Christopher R. 2018. “The Demand for Effective Charter Schools.” Journal of Political Econ-omy 126 (6): 2179–2223.

Wooldridge, Jeffrey M. 2015. “Control Function Methods in Applied Econometrics.” Journal of Human Resources 50 (2): 420–45.

Online Appendix to “Do Parents Value School Effectiveness?”

Atila Abdulkadiroglu, Parag Pathak, Jonathan Schellenberg, and Christopher Walters

November 2019

Appendix A: Data

The data used for this project were provided by the NYC Department of Education (DOE). This Appendix

describes the DOE data files and explains the process used to construct our working extract from these

files.

A.1 Application Data

Data on NYC high school applications are controlled by the Student Enrollment Office. We received all

applications for the 2003-2004 through 2006-2007 school years. Application records include students’ rank-

ordered lists of academic programs submitted in each round of the application process, along with school

priorities and student attributes such as special education status, race, gender, and address. The raw

application files contained all applications, including private school students and first-time ninth graders

who wished to change schools as well as new high school applicants. From these records we selected the

set of eighth graders who were enrolled as NYC public school students in the previous school year.

A.2 Enrollment Data

We received registration and enrollment files from the Office of School Performance and Accountability

(OSPA). These data include every student’s grade and building code, or school ID, as of October of

each school year. A separate OSPA file contains biographical information, including many of the same

demographic variables from the application data. We measure demographics from the application records

for variables that appeared in both files and use the OSPA file to gather additional background information

such as subsidized lunch status.

OSPA also provided an attendance file with days attended and absent for each student at every school

he or she attended in a given year. We use these attendance records to assign students to ninth-grade

schools. If a student was enrolled in multiple schools, we use the school with the greatest number of days

attended in the year following their final application to high school. A final OSPA file included scores on

1

New York State Education Department eighth grade achievement tests. We use these test scores to assign

baseline math and English Language Arts (reading) scores. Baseline scores are normalized to have mean

zero and standard deviation one in our applicant sample.

A.3 Outcome Data

Our analysis studies five outcomes: Regents math scores, PSAT scores, high school graduation, college

attendance, and college quality. We next describe the construction of each of these outcomes.

The Regents math test is one of five tests NYC students must pass to receive a Regents high school

diploma from the state of New York. We received records of scores on all Regents tests taken between

2004 and 2008. We measured Regents math scores based on the lowest level math test offered in each year,

which changed over the course of our sample. For the first three cohorts the lowest level math test offered

was the Math A (Elementary Algebra and Planar Geometry) test. In 2007, the Board of Regents began

administering the Math E (Integrated Algebra I) exam in addition to the Math A exam; the latter was

phased out completely by 2009. We assign the earliest high school score on either of these two exams as

the Regents math outcome for students in our sample. The majority of students took Math A in tenth

grade, while most students taking Math E did so in ninth grade.

PSAT scores were provided to the NYC DOE by the College Board for 2003-2012. We retain PSAT

scores that include all three test sections: math, reading, and writing (some subtests are missing for some

observations, particularly in earlier years of our sample). If students took the PSAT multiple times, we

use the score from the first attempt.

High school graduation is measured from graduation files reporting discharge status for all public school

students between 2005 and 2012. These files indicate the last school attended by each student and the

reason for discharge, including graduation, equivalent achievement (e.g. receiving a general equivalency

diploma), or dropout. Discharge status is reported in years 4, 5, and 6 from expected graduation based on

a student’s year of ninth grade enrollment; our data window ends in 2012, so we only observe 4-year and

5-year high school discharge outcomes for students enrolled in eighth grade for the 2006-2007 year. We

therefore focus on 5-year graduation for all four cohorts. Our graduation outcome equals one if a student

received either a local diploma, a Regents diploma, or an Advanced Regents diploma within 5 years of her

expected graduation date. Students not present in the graduation files are coded as not graduating.

College outcomes are measured from National Student Clearinghouse (NSC) files. The NSC records

enrollment for the vast majority of post-secondary institutions, though a few important New York City-area

institutions, including Rutgers and Columbia University, were not included in the NSC during our sample

period.1 The NYC DOE submitted identifying information for all NYC students graduating between 2009

and 2012 for matching to the NSC. Since many students in the 2003-04 eighth grade cohort graduated in

2008, NSC data are missing for a large fraction of this cohort. Our college outcomes are therefore defined1In addition, about 100 parents opted out of the NSC in 2011 and 2012.

2

only for the last three cohorts in the sample. For these years we code a student as attending college if she

enrolled in a post-secondary institution within five years of applying to high school. This captures students

who graduated from high school on time and enrolled in college the following fall, as well as students that

delayed high school graduation or college enrollment by one year.

We measure college quality based on the mean 2014 incomes of students enrolled in each institution

among those born between 1980 and 1982. These average incomes are reported by Chetty et al. (2017b).

Fewer than 100 observations in the NSC sample failed to match to institutions in the Chetty et al. (2017b)

sample. For students who enrolled in multiple postsecondary institutions, we assign the quality of the first

institution attended. If a student enrolled in multiple schools simultaneously, we use the institution with

the highest mean earnings.

A.4 School Characteristic Data

We include three school characteristic measures in our analysis: a school environment score, violent and

disruptive incidents (VADI) per student, and percent of teachers with masters degrees.

We construct the school environment score using the 2008 NYC Learning Environments Survey. This

survey surveys parents, teachers, and students with questions that fall into four main areas: Safety and

Respect, Communication, Academic Expectations, and Engagement. The survey provides an aggregate

score for each of these categories, and we use principal components analysis on these four scores to construct

a single school environment score. School environment scores are in standard deviation units across schools

in the survey sample.

VADI data are available on the New York State Department of Education (NYSED) website as a part

of the School Safety and Educational Climate data collection. We construct VADI per student by dividing

the total number of VADI by the total enrollment of the school for the 2005-2006 school year. There are

five schools in our sample missing VADI data. VADI per student is in standard deviation units across the

NYC schools reporting VADI.

Percent of teachers with masters degrees comes from the New York State School Report Card 2006-

2007 database and is available on the NYSED website. The database includes data from the 2004-2005,

2005-2006, and 2006-2007 school years. We use data from 2006-2007 school year when possible, and fill in

missing data with earlier years when necessary. Twenty-six schools are missing data on percent of teachers

with masters degrees. Percent of teachers with masters degrees is in standard deviation units across the

NYC schools reporting percent of teachers with masters degrees.

3

A.5 Matching Data Files

To construct our final analysis sample, we begin with the set of high school applications submitted by

students enrolled in eighth grade between the 2003-2004 and 2006-2007 school years. We match these

applications to the student enrollment file using a unique student identifier known as the OSISID and

retain individuals that appear as eighth graders in both data sets. If a student submits multiple high

school applications as an eighth grader, we select the final application for which data is available. We then

use the OSISID to match applicant records to the OSPA attendance and test scores files (used to assign

ninth grade enrollment and baseline test scores), and the Regents, PSAT, graduation, and NSC outcome

files.

This merged sample is used to construct the set of 316 high schools that enrolled at least 50 students

with observations for each of the five outcomes, excluding selective schools that do not participate in the

main DA round. The final choice sample includes the set of high school applicants reporting at least one of

these 316 schools on their preference lists. The five outcome samples are subsets of the choice sample with

observed data on the relevant outcome and enrolled in one of our sample high schools for ninth grade. Table

A1 displays the impact of each restriction on sample size for the four cohorts in our analysis sample. We use

the school DBN indentifier to match the school environment scores from the NYC Learning Environments

Survey to our final analysis sample. We construct the school DBN number from the borough, district, and

BEDS code in the VADI data and school report card data from NYSED in order to match these variables

to our final analysis file.

4

Appendix B: Econometric Methods

B.1 Rank-Ordered Control Functions

This section provides formulas for the rank-ordered control functions in equation (8). The choice model is

Uij = δc(Xi)j − τc(Xi)Dij + ηij = Vij + ηij ,

where Vij ≡ δc(Xi)j − τc(Xi)Dij represents the observed component of student i’s utility for school j and

ηij is the unobserved component. The control functions are given by λij = E[ηij − µη|Xi, Di, Ri] =

E[ηij |Ri, Vi] − µη, where Vi = (Vi1, ..., ViJ)′. To compute the conditional mean of ηij , it will be useful to

define the following functions for any set of mean utilities S and subset S′ ⊆ S:

P (S′|S) =

∑v∈S′

exp(v)∑v∈S

exp(v),

I(S) = µη + log(∑

v∈S exp(v)).

P (S′|S) gives the probability that an individual chooses an option in S′ from the set S when the value

of each option is the sum of its mean utility and an extreme value type I error term, while I(S) gives

the expected maximum utility of choosing an option in S, also known as the inclusive value. We provide

expressions for the control functions for two cases: (1) when a student ranks all available alternatives, and

(2) when the student leaves some alternatives unranked.

B.1.1 All alternatives ranked

Control function for the highest-ranked alternative

Without loss of generality label alternatives in decreasing order of student i’s preferences, so that Rij = j

for j = 1...J . The control function associated with the highest ranked alternative is

λi1 = −(Vi1 + µη) + E [Ui1|Ri, Vi]

= −(Vi1 + µη) +

∫∞−∞

∫ u1−∞

∫ u2−∞ ....

∫ uJ−1−∞

[u1∏Jj=1 f(uj |Vij)

]duJ ...du2du1∏J−1

j=1 P (Vij |Vij ....ViJ),

where f(u|V ) = exp (V − u− exp(V − u)) is the density function of a Gumbel random variable with

location parameter V . This simplifies to

λi1 = −(Vi1 + µη) +∏Jj=1 P (Vij |Vij ....ViJ)× I(Vi1...ViJ)∏J−1

j=1 P (Vij |Vij ....ViJ).

= −Vi1 + (I(Vi1...ViJ)− µη)

5

= − logP (Vi1|Vi1...ViJ),

which coincides with the control function for the best alternative in the multinomial logit model of Dubin

and McFadden (1984). This shows that knowledge of the rankings of less-preferred alternatives does not

affect the expected utility associated with the best choice.

Control functions for lower-ranked alternatives

To work out λij for j > 1, define the following functions:

Gi0(u) = 1,

Gik(u) =∫ ∞u

f(x|Vik)Gi(k−1)(x)dx, k = 1...J .

It can be shown that

Gik(u) =k∑j=1

Bjik [1− F (u|I(Vj ...Vk)− µη)],

where F (u|V ) = exp(− exp(V − u)) is the Gumbel CDF with location V , and the coefficients Bjik are:

B1i1 = 1,

Bjik = −Bji(k−1) × P (Vik|Vij ....Vik) , k > 1, j 6= k,

Bkik =k−1∑j=1

Bji(k−1), k > 1.

Then for j > 1, we have

λij = −(Vij + µη) +

∫∞−∞

∫∞uj

∫∞uj−1

...∫∞u2

∫ uj−∞

∫ uj+1−∞ ...

∫ uJ−1−∞

[uj∏Jk=1 f(uk|Vik)

]duJ ...duj+1du1...duj∏J−1

k=1 P (Vik|Vik....ViJ)

= −(Vij + µη) +∫∞−∞ ujf(uj |I(Vij ...ViJ)− µη)Gi(j−1)(uj)duj∏j−1

k=1 P (Vik|Vik....ViJ)

= −(Vij + µη) +∑j−1m=1B

mi(j−1) [I(Vij ...ViJ)− P (Vij ...ViJ |Vim...ViJ)I(Vim...ViJ)]∏j−1

k=1 P (Vik|Vik....ViJ).

B.1.2 Unranked alternatives

To derive the control functions for a case in which some alternatives are unranked, assign arbitrary labels

`(i) + 1....J to unranked schools. The control functions for all ranked alternatives can be obtained by

defining a composite unranked alternative with observed utility Viu = I (Vik : k > `(i))− µη and treating

this as the lowest-ranked option in the calculations in section B.1.1. The control function for an unranked

alternative j > `(i) is defined by the expression

λij + (Vij + µη) = E[Uij |Ui1 > ... > Ui`(i), Ui`(i) > Uik ∀k > `(i), Vi

]6

=

∫∞−∞

∫∞uj

∫∞u`(i)

∫∞u`(i)−1

...∫∞u2

∫ u`(i)−∞ ..

∫ u`(i)−∞ uj

∏Jk=1 f(uk|Vik)du`(i)+1duj−1duj+1...duJdu1...du`(i)duj∏`(i)

k=1 P (Vik|Vik...ViJ)

=

∫∞−∞ ujf(uj |Vij)

[∫∞ujf(u`(i)|I(S−ji (`(i)))− µη

)Gi(`(i)−1)(u`(i))du`(i)

]duj

P (Vi`(i)|S−ji (`(i)))−1 ×∏`(i)k=1 P (Vik|Vik...ViJ)

,

where S−ji (m) = Vik : k ≥ m\Vij is the set of i’s mean utilities for alternatives m and higher excluding

alternative j. When `(i) = 1, we have Gi(`(i)−1)(u`) = 1 and this expression collapses to

λij = P (Vij |Vi1...ViJ)1− P (Vij |Vi1...ViJ) logP (Vij |Vi1...ViJ),

which is the expression derived by Dubin and McFadden (1984) for the expected errors of alternatives that

are not selected in the multinomial logit model. For `(i) > 1, we have

λij = −(Vij + µη)

+

`(i)−1∑m=1

Bmi(`(i)−1)

[(1− P (S−j

i (`(i))|S−ji (m)))I(Vij)− P (Vij |Vi`(i)..ViJ )I(Vi`(i)..ViJ ) + P (S−j

i (`(i))|S−ji (m))P (Vij |Vim..ViJ )I(Vim..ViJ )

]P (Vi`(i)|S

−ji (`(i)))−1 ×

`(i)∏k=1

P (Vik|Vik...ViJ )

.

B.2 Two-Step Score Bootstrap

We use a two-step modification of the score bootstrap of Kline and Santos (2012) to conduct inference

for the control function models. Let ∆ = (δ11...δ1J , τ1...δC1...δCJ , τC)′ denote the vector of choice model

parameters for all covariate cells. Maximum likelihood estimates of these parameters are given by:

∆ = arg max∆

∑i

logL(Ri|Xi, Di; ∆),

where L(Ri|Xi, Di; ∆) is the likelihood function defined in Section III.A, now explicitly written as a function

of the choice model parameters.

Let Γ = (α1, β′1, ψ1...αJ , β

′J , ψJ , γ

′, ϕ)′ denote the vector of outcome equation parameters. Second-step

estimates of these parameters are

Γ =[∑

i

Wi(∆)Wi(∆)′]−1

×∑i

Wi(∆)Yi,

where Wi(∆) is the vector of regressors in equation (8). This vector depends on ∆ through the control

functions λj(Xi, Di, Ri; ∆), which in turn depend on the choice model parameters as described in Appendix

B.1.

The two-step score bootstrap adjusts inference for the extra uncertainty introduced by the first-step

estimates while avoiding the need to recalculate ∆ or to analytically derive the influence of ∆ on Γ. The

7

first step directly applies the approach in Kline and Santos (2012) to the choice model estimates. This

approach generates a bootstrap distribution for ∆ by taking repeated Newton-Raphson steps from the

full-sample estimates, randomly reweighting each observation’s score contribution. The bootstrap estimate

of ∆ in trial b ∈ 1...B is:

∆b = ∆−[∑

i

(∂2 logL(Ri|Xi,Di;∆)

∂∆∂∆′

)]−1

×∑i

ζbi

(∂ logL(Ri|Xi,Di;∆)

∂∆

),

where the ζbi are iid random weights satisfying E[ζbi]

= 0 and E[(ζbi )2] = 1. We draw these weights from

a standard normal distribution.

Next, we use an additional set of Newton-Raphson steps to generate a bootstrap distribution for Γ.

The second-step bootstrap estimates are:

Γb = Γ−[∑

i

Wi(∆)Wi(∆)′]−1

×∑i

[−ζbiWi(∆)(Yi −Wi(∆)′Γ)−Wi(∆b)(Yi −Wi(∆b)′Γ)

].

The second term in the last sum accounts for the additional variability in the second-step score due to the

first-step estimate ∆. We construct standard errors and conduct hypothesis tests involving Γ using the

distribution of Γb across bootstrap trials.

B.3 Empirical Bayes Shrinkage

We next describe the empirical Bayes shrinkage procecure summarized in Section III.B. Value-added or

control function estimation produces a set of school-specific parameter estimates,θj

Jj=1

. Under the

hierarchical model (10), the likelihood of the estimates for school j conditional on the latent parameters

θj and the sampling variance matrix Ωj is:

L(θj |θj ,Ωj

)= (2π)−T/2 |Ωj |−1/2 exp

(− 1

2 (θj − θj)′Ω−1j (θj − θj)

),

where T = dim(θj). We estimate Ωj using conventional asymptotics for the value-added models and the

bootstrap procedure described in Section B.2 for the control function models. Our approach therefore

requires school-specific samples to be large enough for these asymptotic approximations to be accurate.

An integrated likelihood function that conditions only on the hyperparameters is:

LI(θj |µθ,Σθ,Ωj) =∫L(θj |θj ,Ωj)dF (θj |µθ,Σθ)

= (2π)−T/2 |Ωj + Σθ|−1/2 exp(− 1

2 (θj − µθ)′ (Ωj + Σθ)−1 (θj − µθ)).

EB estimates of the hyperparameters are then(µθ, Σθ

)= arg max

µθ,Σθ

∑j

logLI(θj |µθ,Σθ, Ωj

),

where Ωj estimates Ωj .

By standard arguments, the posterior distribution for θj given the estimate θj is

8

θj |θj ∼ N(θ∗j ,Ω∗j

),

where

θ∗j =(Ω−1j + Σ−1

θ

)−1 (Ω−1j θj + Σ−1

j µθ

),

Ω∗j =(Ω−1j + Σ−1

θ

)−1.

We form EB posteriors by plugging Ωj , µθ and Σθ into these formulas.

9

All cohorts 2003-2004 2004-2005 2005-2006 2006-2007(1) (2) (3) (4) (5)

All NYC eighth graders 368,603 89,671 93,399 94,015 91,518

In public school 327,948 78,904 83,112 84,067 81,865

With baseline demographics 276,797 68,507 67,555 68,279 72,456

With address data 275,405 67,644 67,377 68,108 72,276

In preference sample 270,157 66,125 66,004 67,163 70,865

In Regents math sample 155,850 40,994 41,022 39,177 34,657

In PSAT sample 149,365 31,563 37,502 39,480 40,820

In high school graduation sample 230,087 56,833 56,979 57,803 58,472

In college sample 173,254 0 56,979 57,803 58,472Notes: This table displays the selection criteria for inclusion in the final analysis samples. Preference models are estimated using the sample in the fourth row, and school effects are estimated using the samples in the remaining rows.

Table A1. Sample restrictions

Peerquality ATE Female Black Hispanic Sub. lunch Math score

(1) (2) (3) (4) (5) (6) (7)ATE 0.531

(0.042)

Female 0.133 0.232(0.077) (0.082)

Black -0.033 -0.007 -0.287(0.074) (0.082) (0.133)

Hispanic -0.002 -0.028 -0.414 0.939(0.077) (0.086) (0.135) (0.022)

Subsidized lunch 0.093 -0.133 0.098 -0.027 0.065(0.088) (0.097) (0.145) (0.151) (0.155)

Eighth grade math score -0.108 0.033 -0.104 -0.005 0.054 0.012(0.064) (0.069) (0.098) (0.100) (0.105) (0.118)

Eighth grade reading score -0.564 -0.425 -0.036 -0.065 -0.064 0.071 0.244(0.065) (0.079) (0.124) (0.123) (0.130) (0.134) (0.103)

Value-added parametersTable A2. Correlations of peer quality and treatment effect parameters for Regents math scores, value-added model

Notes: This table reports estimated correlations between peer quality and school treatment effect parameters for Regents math scores. The ATE is a school's average treatment effect, and other treatment effect parameters are school-specific interactions with student characteristics. Estimates come from maximum likelihood models fit to school-specific regression coefficients from a value-added model controlling for observed characteristics.

Peerquality ATE Female Black Hispanic Sub. lunch Math score Reading score Pref. coef.

(1) (2) (3) (4) (5) (6) (7) (8) (9)Mean 0 0 -0.033 -0.284 -0.259 -0.006 0.963 1.032 -0.003

- - (0.010) (0.026) (0.027) (0.011) (0.016) (0.011) (0.001)

Standard deviation 0.884 0.401 0.111 0.333 0.352 0.111 0.240 0.152 0.017(0.056) (0.048) (0.012) (0.023) (0.026) (0.017) (0.016) (0.073) (0.011)

Correlations: ATE 0.979(0.086)

Female -0.251 -0.315(0.094) (0.068)

Black -0.130 -0.253 0.020(0.124) (0.090) (0.160)

Hispanic -0.168 -0.274 0.112 0.932(0.094) (0.079) (0.150) (0.123)

Subsidized lunch -0.197 -0.211 0.252 -0.131 -0.120(0.144) (0.101) (0.117) (0.135) (0.124)

Eighth grade math score 0.709 0.701 -0.117 -0.005 -0.090 -0.099(0.123) (0.102) (0.093) (0.125) (0.108) (0.135)

Eighth grade reading score 0.164 0.249 -0.219 0.011 -0.084 0.108 0.246(0.230) (0.121) (0.074) (0.067) (0.065) (0.072) (0.287)

Preference coefficient (𝜓j) 0.377 0.291 -0.159 -0.114 -0.062 -0.157 0.100 -0.109(0.280) (0.145) (0.039) (0.038) (0.055) (0.066) (0.074) (0.105)

Table A3. Joint distribution of peer quality and treatment effect parameters for PSAT scores/10Control function parameters

Notes: This table shows the estimated joint distribution of peer quality and school treatment effect parameters for PSAT scores divded by 10. The ATE is a school's average treatment effect, and other treatment effect parameters are school-specific interactions with student characteristics. Estimates come from maximum likelihood models fit to school-specific regression coefficients from a control function model controlling for observed characteristics, distance to school and unobserved tastes from the choice model.

Peerquality ATE Female Black Hispanic Sub. lunch Math score Reading score Pref. coef.

(1) (2) (3) (4) (5) (6) (7) (8) (9)Mean 0 0 0.063 -0.006 -0.013 -0.013 0.132 0.062 0.000

- - (0.004) (0.007) (0.008) (0.003) (0.003) (0.002) (0.000)

Standard deviation 0.100 0.043 0.047 0.090 0.103 0.024 0.034 0.027 0.006(0.004) (0.008) (0.004) (0.007) (0.007) (0.003) (0.002) (0.002) (0.000)

Correlations: ATE 0.590(0.106)

Female -0.072 -0.549(0.070) (0.170)

Black -0.226 -0.296 -0.069(0.069) (0.195) (0.142)

Hispanic -0.174 -0.237 -0.078 0.956(0.067) (0.196) (0.135) (0.013)

Subsidized lunch 0.169 -0.120 0.119 0.171 0.264(0.096) (0.238) (0.169) (0.180) (0.176)

Eighth grade math score -0.396 -0.619 0.075 -0.168 -0.114 0.051(0.060) (0.166) (0.098) (0.109) (0.107) (0.128)

Eighth grade reading score -0.571 -0.570 -0.125 0.188 0.094 -0.194 0.475(0.059) (0.180) (0.112) (0.136) (0.134) (0.153) (0.103)

Preference coefficient (𝜓j) 0.625 0.437 0.123 -0.110 -0.049 0.021 -0.246 -0.470(0.044) (0.180) (0.084) (0.089) (0.086) (0.120) (0.078) (0.078)

Table A4. Joint distribution of peer quality and treatment effect parameters for high school graduationControl function parameters

Notes: This table shows the estimated joint distribution of peer quality and school treatment effect parameters for high school graduation. The ATE is a school's average treatment effect, and other treatment effect parameters are school-specific interactions with student characteristics. Estimates come from maximum likelihood models fit to school-specific regression coefficients from a control function model controlling for observed characteristics, distance to school and unobserved tastes from the choice model.

Peerquality ATE Female Black Hispanic Sub. lunch Math score Reading score Pref. coef.

(1) (2) (3) (4) (5) (6) (7) (8) (9)Mean 0 0 0.075 -0.010 -0.011 -0.008 0.118 0.064 -0.002

- - (0.003) (0.009) (0.009) (0.003) (0.002) (0.002) (0.000)

Standard deviation 0.099 0.053 0.035 0.122 0.120 0.031 0.030 0.024 0.005(0.118) (0.022) (0.004) (0.009) (0.009) (0.005) (0.013) (0.009) (0.002)

Correlations: ATE 0.862(0.158)

Female -0.074 -0.307(0.017) (0.031)

Black -0.035 -0.455 0.040(0.021) (0.066) (0.160)

Hispanic -0.135 -0.471 -0.024 0.947(0.019) (0.031) (0.043) (0.019)

Subsidized lunch 0.110 0.235 -0.005 -0.390 -0.339(0.027) (0.078) (0.139) (0.119) (0.117)

Eighth grade math score -0.204 -0.188 0.265 -0.067 -0.028 0.073(0.073) (0.179) (0.074) (0.073) (0.056) (0.110)

Eighth grade reading score -0.290 -0.121 -0.131 -0.346 -0.364 -0.198 0.304(0.112) (0.197) (0.078) (0.083) (0.082) (0.105) (0.171)

Preference coefficient (𝜓j) 0.770 0.524 0.144 0.106 0.059 0.003 -0.072 -0.314(0.119) (0.130) (0.068) (0.056) (0.057) (0.129) (0.238) (0.183)

Table A5. Joint distribution of peer quality and treatment effect parameters for college attendanceControl function parameters

Notes: This table shows the estimated joint distribution of peer quality and school treatment effect parameters for college attendance. The ATE is a school's average treatment effect, and other treatment effect parameters are school-specific interactions with student characteristics. Estimates come from maximum likelihood models fit to school-specific regression coefficients from a control function model controlling for observed characteristics, distance to school and unobserved tastes from the choice model.

Peerquality ATE Female Black Hispanic Sub. lunch Math score Reading score Pref. coef.

(1) (2) (3) (4) (5) (6) (7) (8) (9)Mean 0 0 0.048 -0.037 -0.035 -0.006 0.103 0.058 -0.002

- - (0.002) (0.006) (0.006) (0.002) (0.002) (0.002) (0.000)

Standard deviation 0.097 0.063 0.027 0.081 0.084 0.022 0.031 0.019 0.004(0.078) (0.017) (0.003) (0.006) (0.006) (0.004) (0.010) (0.006) (0.004)

Correlations: ATE 0.931(0.051)

Female 0.114 0.084(0.018) (0.021)

Black -0.065 -0.258 -0.023(0.019) (0.029) (0.157)

Hispanic -0.239 -0.354 -0.127 0.946(0.018) (0.021) (0.059) (0.048)

Subsidized lunch -0.063 0.060 0.253 -0.334 -0.208(0.035) (0.038) (0.082) (0.085) (0.071)

Eighth grade math score 0.533 0.728 0.381 -0.143 -0.151 0.146(0.078) (0.063) (0.054) (0.072) (0.040) (0.066)

Eighth grade reading score 0.296 0.479 -0.027 -0.266 -0.275 -0.355 0.466(0.064) (0.033) (0.018) (0.019) (0.020) (0.046) (0.070)

Preference coefficient (𝜓j) 0.750 0.623 0.135 0.033 -0.061 -0.086 0.310 0.161(0.076) (0.041) (0.008) (0.019) (0.009) (0.021) (0.059) (0.033)

Table A6. Joint distribution of peer quality and treatment effect parameters for log college qualityControl function parameters

Notes: This table shows the estimated joint distribution of peer quality and school treatment effect parameters for college quality. The ATE is a school's average treatment effect, and other treatment effect parameters are school-specific interactions with student characteristics. Estimates come from maximum likelihood models fit to school-specific regression coefficients from a control function model controlling for observed characteristics, distance to school and unobserved tastes from the choice model.

Uncontrolled Controlled(1) (2)

Black 74.53 0.873(0.000) (0.842)

Hispanic 54.86 0.897(0.000) (0.788)

Female 15.71 1.115(0.000) (0.181)

Log census tract median income 116.4 1.134(0.000) (0.147)

Subsidized lunch 30.46 1.184(0.000) (0.080)

Eighth grade math score 27.96 1.059(0.000) (0.311)

Eighth grade reading score 30.06 1.034(0.000) (0.380)

Schools 124 124Students 53,327 32,131Notes: This table reports F-statistics from school-specific tests for balance, computed by regressing covariates on dummies indicating offers at each school in the sample and testing that the coefficients on all offer dummies are jointly zero. P -values reported in parentheses. Column (2) controls for linear school-specific propensity score controls and school-specific dummies for degenerate p-score values. The sample is restricted to students who have non-degenerate risk for at least one school and lotteries with 100 or more students at risk. Students are considered to have risk at a given school if their propensity score is strictly between zero and one and they are in a score cell with variation in school offers.

Table A7. Tests for covariate balance in admission lotteries

(1) (2) (3) (4) (5) (6) (7) (8) (9)Peer quality 0.310 0.314 0.286 0.299 0.303 0.308

(0.060) (0.059) (0.060) (0.056) (0.056) (0.056)

ATE 0.157 -0.005 0.005 0.144 -0.008 -0.003(0.042) (0.039) (0.040) (0.040) (0.035) (0.035)

Match effect -0.068 -0.142(0.039) (0.044)

School Environment Score 0.015 -0.123 -0.037 -0.123 -0.128 -0.122 -0.034 -0.121 -0.118(0.042) (0.044) (0.044) (0.044) (0.045) (0.045) (0.044) (0.045) (0.046)

VADI Per Student -0.331 -0.222 -0.284 -0.223 -0.223 -0.226 -0.288 -0.227 -0.213(0.042) (0.038) (0.039) (0.039) (0.039) (0.038) (0.039) (0.038) (0.038)

% Teachers with Masters + 0.418 0.347 0.400 0.347 0.344 0.347 0.406 0.346 0.336(0.040) (0.036) (0.039) (0.036) (0.038) (0.037) (0.039) (0.037) (0.042)

N

Table A8. Preferences for peer quality and Regents math effects with controls for additional school characteristicsValue-added models Control function models

20200Notes: This table reports estimates from regressions of school popularity on peer quality and school effectiveness along with controls for other school attributes. School popularity is measured as the estimated mean utility for each school and covariate cell in the choice model from Table 4. Covariate cells are defined by borough, gender, race, subsidized lunch status, an indicator for students above the median of census tract median income, and tercile of the average of eighth grade math and reading scores. Peer quality is constructed as the average predicted Regents math score for enrolled students. Treatment effect estimates are empirical Bayes posterior mean predictions of Regents math effects. Mean utilities, peer quality, and treatment effects are scaled in standard deviation units. Columns (2)-(5) report results from value-added models, while columns (6)-(9) report results from control function models. All regressions include cell indicators and include the following school controls: school environment score, VADI per student, and percent of teachers with master’s degrees plus. The school environment score is a measure that combines the following school survey summary scores from the NYC Learning Environments Survey: Safety and Respect, Communication, Engagement and Academic Expectations using principle components analysis. VADI (Violent and Disruptive incidents) per student is constructed using Violent and Disruptive Incident Reporting data from the NYS department of education. Percent of teachers with master’s degrees plus comes from the NY school report cards from 2005-2007. We use the latest year of data available. School environment scores, VADI per student, and percent of teachers with masters degrees are scaled in standard deviation units. All regressions weight by the inverse of the squared standard error of the mean utility estimates. Standard errors are double-clustered by school and covariate cell.

2003-2004 2004-2005 2005-2006 2006-2007 2003-2004 2004-2005 2005-2006 2006-2007(1) (2) (3) (4) (5) (6) (7) (8)

Peer Quality 0.384 0.455 0.538 0.465 0.412 0.478 0.570 0.505(0.072) (0.077) (0.076) (0.074) (0.065) (0.067) (0.066) (0.064)

ATE -0.044 -0.029 -0.031 -0.009 -0.061 -0.061 -0.071 -0.050(0.056) (0.055) (0.054) (0.050) (0.053) (0.052) (0.050) (0.047)

Match Effect -0.090 -0.054 -0.057 -0.062 -0.175 -0.181 -0.165 -0.160(0.053) (0.056) (0.051) (0.048) (0.058) (0.062) (0.056) (0.053)

N 17141 18493 18787 19286 17141 18493 18787 19286

Table A9. Yearly preferences for peer quality and Regents math effectsValue-added models Control function models

Notes: This table reports estimates from regressions of school popularity on peer quality and school effectiveness. School popularity is measured as the estimated mean utility for each school and covariate cell in the choice model from Table 4. We estimate this model on four subsamples: one from each school year. Covariate cells are defined by borough, gender, race, subsidized lunch status, an indicator for students above the median of census tract median income, and tercile of the average of eighth grade math and reading scores. Peer quality is constructed as the average predicted Regents math score for enrolled students across all years. Treatment effect estimates are empirical Bayes posterior mean predictions of Regents math effects across all years. Mean utilities, peer quality, and treatment effects are scaled in standard deviation units. Columns (1)-(4) report results from value-added models, while columns (5)-(8) report results from control function models. All regressions include cell indicators and weight by the inverse of the squared standard error of the mean utility estimates. Standard errors are double-clustered by school and covariate cell.

Control Control Control ControlValue-added function Value-added function Value-added function Value-added function

(1) (2) (3) (4) (5) (6) (7) (8)Peer quality 0.487 0.542 0.036 0.038 0.445 0.485 0.373 0.386

(0.071) (0.062) (0.005) (0.005) (0.077) (0.068) (0.026) (0.029)

ATE -0.009 -0.034 -0.001 -0.002 -0.040 -0.073 -0.021 -0.027(0.045) (0.040) (0.003) (0.003) (0.054) (0.049) (0.026) (0.024)

Match effect -0.091 -0.219 -0.004 -0.012 -0.092 -0.184 -0.049 -0.106(0.043) (0.047) (0.003) (0.004) (0.050) (0.055) (0.023) (0.027)

N

Table A10. Preferences, peer quality, and math effects, alternative measures of popularity

Notes: This table reports estimates from regressions of alternative measures of school popularity on peer quality and school effectiveness. The dependent variable in columns (1) and (2) is the log of the share of students in a covariate cell ranking each school first, and the dependent variable in columns (3) and (4) is minus the log of the sum of ranks for students in the cell. Unranked schools are assigned one rank below the least-preferred ranked school. Columns (5) and (6) restrict preference estimation to students that ranked fewer than 12 choices. Columns (7) and (8) estimate preferences using only the schools on a student's choice list, omitting unranked alternatives from the likelihood. Covariate cells are defined by borough, gender, race, subsidized lunch status, an indicator for students above the median of census tract median income, and tercile of the average of eighth grade math and reading scores. Peer quality is constructed as the average predicted Regents math score for enrolled students. Treatment effect estimates are empirical Bayes posterior mean predictions of Regents math effects. Columns (1), (3), (5), and (7) report results from value-added models, while columns (2), (4), (6), and (8) report results from control function models. All regressions include cell indicators. Standard errors are double-clustered by school and covariate cell.

Log first-choice share Omitting unranked schools

15892 21684

Minus log sum of ranks Fewer than 12 choices

20898 19842

(1) (2) (3) (4) (5) (6) (7) (8)Peer quality 0.367 0.400 0.406 0.397 0.402 0.408

(0.053) (0.054) (0.067) (0.058) (0.060) (0.060)

ATE 0.209 -0.058 -0.036 0.236 -0.009 -0.027(0.045) (0.043) (0.045) (0.046) (0.044) (0.045)

Match effect -0.092 -0.129(0.049) (0.041)

N

Table A11. Preferences, peer quality, and math effects, alternative treatment effect models

21684Notes: This table reports estimates from regressions of school popularity on peer quality and alternative measures of school effectiveness. Estimates in columns (1)-(4) come from an OLS regression of Regents math scores on school indicators interacted with covariates, with controls for distance and fixed effects for first choice schools. Estimates in columns (5)-(8) come from a regression of Regents math scores on school indicators interacted with covariates and control functions measuring mean preferences for each school, excluding distance controls. School popularity is measured as the estimated mean utility for each school and covariate cell in the choice model from Table 4. Covariate cells are defined by borough, gender, race, subsidized lunch status, an indicator for students above the median of census tract median income, and tercile of the average of eighth grade math and reading scores. Peer quality is constructed as the average predicted Regents math score for enrolled students. Treatment effect estimates are empirical Bayes posterior mean predictions of Regents math effects. Mean utilities, peer quality, and treatment effects are scaled in standard deviation units. All regressions include cell indicators and weight by the inverse of the squared standard error of the mean utility estimates. Standard errors are double-clustered by school and covariate cell.

Matched first choice model Distance instrument model

Peer quality ATE Match Peer quality ATE MatchBaseline quartile (1) (2) (3) (4) (5) (6) (7)

Lowest -0.084 0.015 0.015 0.312 0.452 0.356 0.779

Second 0.011 0.042 0.005 0.395 0.469 0.122 0.545

Third 0.127 0.074 -0.011 0.329 0.464 0.018 0.419

Highest 0.399 0.155 -0.157 0.106 0.324 0.149 0.475

Table A12. Potential achievement gains from ranking schools by effectiveness, by baseline test score quartileObserved rankings Rankings based on effectiveness Increase in

effectiveness

Notes: This table summarizes Regents math score gains that parents could achieve by ranking schools based on effectiveness., separately by baseline math score quartile. Columns (1)-(3) report average peer quality, average treatment effects, and average match effects for schools ranked first by students in each quartile. Columns (4)-(6) display corresponding statistics for hypothetical rankings that list schools in order of their treatment effects. Column (7) reports the difference in treatment effects (ATE+match) between the top-ranked school when rankings are based on effectiveness and the observed top-ranked school. Treatment effect estimates come from control function models. All calculations are restricted to ranked schools within the home borough.