## Abstract

Although it is well known that certain charter schools dramatically increase students' standardized test scores, there is considerably less evidence that these human capital gains persist into adulthood. To address this matter, we match three years of lottery data from a high-performing charter high school to administrative college enrollment records and estimate the effect of winning an admissions lottery on college matriculation, quality, and persistence. Seven to nine years after the lottery, we find that lottery winners are 10.0 percentage points more likely to attend college and 9.5 percentage points more likely to enroll for at least four semesters. Unlike previous studies, our estimates are powerful enough to uncover improvements on the extensive margin of college attendance (enrolling in any college), the intensive margin (persistence of attendance), and the quality margin (enrollment at selective, four-year institutions). We conclude by providing nonexperimental evidence that more recent cohorts at other campuses in the network increased enrollment at a similar rate.

## 1.  Introduction

As a college education becomes increasingly necessary to succeed in the modern American labor market and achieve upward socioeconomic mobility (Goldin and Katz 2007; Autor, Katz, and Kearney 2008; Dynarski 2008), a college degree remains one of the best investments available to young Americans (Barrow and Rouse 2005; Heckman, Lochner, and Todd 2008; Acemoglu and Autor 2010; Greenstone and Looney 2011; Oreopoulos and Petronijevic 2013). It is therefore unsurprising that many high-performing urban schools have focused on making college the default option for low-income students. In the contentious debate over school accountability and high-stakes testing, both sides can agree that preparing students to reach their potential as productive citizens generally means preparing them to enroll and succeed in postsecondary studies.

The consensus on how to achieve this goal is far less clear. Many in the education reform movement advocate an interventionist approach, arguing that schools must take an intensive, hands-on role in the lives of poor children (Thernstrom and Thernstrom 2004; Whitman 2008). As evidence, these advocates point to the growing number of studies showing that high-performing charter schools dramatically raise the test scores of low-income students.1 On the other hand, some critics argue that the relentless focus on test scores detracts from real learning and is harmful to students in the long run (Ravitch 2010). The root of the controversy is essentially an empirical question about the education production function. Even though there is a robust positive correlation between test performance and college-going,2 it is not obvious that short-run achievement effects necessarily translate into college success.

To advance this debate, we exploit the outcomes of randomized admissions lotteries for three cohorts of students who applied to enter the Noble Street Charter School (hereafter Noble), a high-performing charter high school in Chicago. We match these records to administrative college enrollment data to estimate the effect of being selected to attend Noble on college matriculation and persistence. We find that lottery winners are 10.0 percentage points more likely to enroll in college and 9.5 percentage points more likely to stay enrolled for at least four semesters. Effects on college graduation are not statistically different from zero, but this at least partially reflects the fact the largest cohort in our sample would not have been old enough to graduate from a four-year college at the time of our data pull.

Importantly, the increase in quantity did not come at the expense of quality. We estimate nearly identical effects if we restrict attention to attendance at four-year colleges or schools with an average entering SAT score of at least 1000 (out of 1600). In addition, the effect on enrollment at two-year colleges is statistically zero. We interpret these findings as strong evidence that Noble is measurably increasing students’ human capital as opposed to pushing marginal students into low-quality institutions.

Our work contributes to a broader literature that uses large administrative datasets to understand the causal relationship between school characteristics and college enrollment. Dobbie and Fryer (2015) find that sixth graders who are offered a spot in the Harlem Children's Zone Promise Academy are more likely to enroll in college immediately after graduating high school, though students in the control group eventually “catch up” and close the gap. In a similar vein, Angrist et al. (2016) show that attending a Boston-area charter school causes many students to enroll in four-year colleges instead of two-year colleges. Other work focuses on the long-term impacts of educational quality rather than school governance. Linking data from a large urban district to tax records, Chetty, Friedman, and Rockoff (2014) use a quasi-experimental design to show that a one-standard-deviation increase in teacher value added for a single year increases a student's probability of college attendance at age 20 years by 0.82 percentage points. Deming et al. (2014) examine school choice lottery results from Charlotte-Mecklenburg (North Carolina) and conclude that attending a first-choice high school leads to small but noteworthy increases in enrollment and graduation rates from competitive colleges.

We make three key contributions to the literature. First, we provide robust evidence that a school-based intervention can simultaneously improve overall enrollment, persistence, and college quality. In particular, we provide the first experimental evidence that a high-quality charter high school can increase overall college enrollment for more than one year.3 In part, this is a consequence of improved statistical power. Noble's admissions lottery generates a very powerful first-stage effect on enrollment that allows us to estimate a variety of treatment effects more precisely than similar papers in the literature (Dobbie and Fryer 2015; Angrist et al. 2016).

Second, we demonstrate the medium-term effectiveness of a relatively late intervention. Noble conducts lotteries for incoming high school freshmen, many of whom have spent their formative years in low-performing, traditional Chicago public schools. Because other economists have emphasized the importance of early interventions for improving the economic outcomes of poor children (see Heckman 2012), this aspect of our results is noteworthy.

Finally, the Noble Network's rapid expansion since 2006 allows us to examine whether the intervention we study is scalable. Over the ensuing eight years, Noble added fifteen high schools, and the network currently enrolls over 10,000 students each year. The expansion not only shows that the network merits study in its own right because of the sheer number of students enrolled; it also provides an interesting test case for whether the intervention we study is scalable. An important question is whether the administrative and instructional practices that drove the flagship campus's impact on college outcomes will have similar impacts in new environments across the diverse communities of Chicago. More generally, Noble's rapid expansion offers a unique opportunity to assess how No Excuses practices scale and replicate in new settings and populations, albeit within the same city.

Therefore, the final portion of our analysis focuses on results at Noble's broader network. Although we cannot exploit lottery-based randomization for this portion of our analysis, we make use of a college enrollment study conducted by Chicago Public Schools (CPS) to nonexperimentally estimate treatment effects for two later cohorts attending seven separate Noble campuses. These results reaffirm our lottery-based findings for Noble's flagship campus. Conditional on observables, the college enrollment rate of students in Noble's expanded network exceeds that of both traditional public school and charter school students in Chicago.

The remainder of the paper is organized as follows. Section 2 describes the history and academic practices of the Noble Network of Charter Schools. Section 3 summarizes our data and empirical strategies. Section 4 reviews sample characteristics and tests the balance of our lotteries. Section 5 presents our main results and our nonexperimental analysis, and section 6 concludes by interpreting our results in light of other recent research.

## 2.  The Noble Network of Charter Schools

The Noble Network of Charter Schools began in 1999 as a single high school, Noble Street Charter School (now Noble Street College Prep). In the fall of 2006, the operator opened two new campuses to serve the growing demand for admissions, and the network has since grown to include sixteen high schools serving roughly 10,000 students. College-readiness is an explicit part of Noble's stated mission “to prepare low-income students with the scholarship, dedication, and honor necessary to succeed in college and lead exemplary lives, and be a catalyst for education reform in Chicago.”4

Noble's campuses are spread across the west and south sides of Chicago. As figure 1 shows, the expansion has been concentrated in high-poverty neighborhoods. Accordingly, Noble attracts a predominantly poor, minority student body. Ninety-five percent of Noble students are black or Hispanic, and nearly 90 percent are eligible for federal lunch subsidies based on family income.

Figure 1.

Noble Network Schools, Lottery Applicants, and Neighborhood Poverty Rates

Notes: This figure plots the locations of the Noble Network of Charter Schools and the addresses of lottery applicants over a grid of Chicago-area census tracts. The largest star marks the location of the original Noble Street Charter School, the subject of our lottery results. The background color indicates the tract poverty rate averaged across the 2007—12 American Community Surveys. Five-year periods are the shortest moving averages available at the tract level.

Figure 1.

Noble Network Schools, Lottery Applicants, and Neighborhood Poverty Rates

Notes: This figure plots the locations of the Noble Network of Charter Schools and the addresses of lottery applicants over a grid of Chicago-area census tracts. The largest star marks the location of the original Noble Street Charter School, the subject of our lottery results. The background color indicates the tract poverty rate averaged across the 2007—12 American Community Surveys. Five-year periods are the shortest moving averages available at the tract level.

Figure 2a.

College Enrollment Over Time: Contemporaneous Enrollment

Figure 2a.

College Enrollment Over Time: Contemporaneous Enrollment

Figure 2b.

College Enrollment Over Time: Cumulative Enrollment

Notes: In both panels, the horizontal axis displays the number of school years after each cohort starts high school. Hence, 4.5 corresponds to the first semester of college for a student making normal and uninterrupted progress through school. Panel A records the percentage of lottery winners and losers who either (1) are enrolled in college in that period or (2) have graduated prior to that period. Panel B plots the percentage of students who have enrolled in college at any point up to the given time period. See section 3 for more details on how we construct these variables.

Figure 2b.

College Enrollment Over Time: Cumulative Enrollment

Notes: In both panels, the horizontal axis displays the number of school years after each cohort starts high school. Hence, 4.5 corresponds to the first semester of college for a student making normal and uninterrupted progress through school. Panel A records the percentage of lottery winners and losers who either (1) are enrolled in college in that period or (2) have graduated prior to that period. Panel B plots the percentage of students who have enrolled in college at any point up to the given time period. See section 3 for more details on how we construct these variables.

Noble's educational philosophy is quite similar to that of other No Excuses charter schools, such as the Knowledge is Power Program and the Harlem Children's Zone Promise Academy.5 Although no two charter management organizations follow precisely the same principles and practices, the philosophy largely aligns with those highlighted by Dobbie and Fryer (2013), who identify five within-school factors that explain roughly 50 percent of the variance in charter school performance in their sample. Noble network schools’ academic programs can be understood within the context of these five practices.

While exact school calendars vary slightly by campus, Noble campuses offer longer class periods, a longer school day, and a longer school year than traditional CPS high schools. On average, Noble students spend 7.5 hours per day and 185 days per year in school. For comparison, during the period covered by our analysis, typical CPS high school students spent only 6.9 hours per day and 170 days per year in school, on average.6 This implies that Noble students spent 18.3 percent more time in school, amounting to 858 hours over four years, or nearly three-quarters of a year of additional instruction.

Noble campuses structure their day to ensure that all students receive differentiated instruction in a smaller-group setting. Students attend daily 35-minute differentiated academic labs, where they get smaller-group instruction specific to their needs. Typical academic labs include reading groups organized by Lexile level, math groups assigned by performance on interim assessments, advanced placement course labs, and socioemotional support seminars. Teachers are required to hold daily office hours that students can attend for academic support from 4:00 to 4:30 pm. Office hour attendance becomes mandatory for students whose academic performance falls below certain thresholds. Most campuses also offer some form of after-school tutoring provided by outside organizations.

To help teachers use data to drive instruction, Noble schools administer quarterly interim assessments in all core content areas (in addition to traditional classroom summative and formative assessments). After students take interim assessments, teachers from all campuses meet in content area teams to analyze the data and collaboratively plan how to use the assessment data to drive their instruction. Morning and afternoon advisories track students’ academic progress, mark behavioral infractions, and hold students accountable as a group for maintaining academic and behavioral standards.

To develop human capital, teachers receive regular feedback on their performance. Teachers attend campus-wide professional development sessions every Friday, in addition to network-wide collaborative planning as outlined here. Noble aggressively recruits teachers with a demonstrated track record of success and rewards teachers with performance bonuses whose students demonstrate above-average academic growth.

Although school culture is notoriously hard to measure, Noble takes several explicit approaches to incubating a culture of high expectations for all. All students are expected to take college entrance exams and gain acceptance to college, regardless of their post-graduation plans. College acceptances are celebrated publicly, and counselors assist students in applying for grants and scholarships. Top students are invited to participate in the “Right Angle” program, which sends students to academic enrichment programs at highly-selective college campuses nationwide.7

## 3.  Data and Research Design

### Raw Data

To generate our empirical estimates, we combine lottery data and student demographic information from Noble administrative records with college matriculation and degree attainment data from the National Student Clearinghouse (NSC). The lottery records span high school lotteries from 1999 to 2012 for all campuses in the Noble Network of Charter Schools, though we can only make use of lottery data for three early cohorts at the original Noble campus. We describe our data sources and the relevant institutional details in the following subsections, and the online Appendix8 provides more details about the raw lottery files and variable construction.

#### Lottery Data and Sample Construction

Our analysis sample includes all students who entered a Noble admissions lottery between 2003 and 2005. Noble was not oversubscribed in 1999, 2000, or 2001, hence there was no lottery. Although Noble did have a binding lottery in 2002, the records are less complete than in subsequent years. We made several attempts to construct the 2002 lottery sample from the files we were provided, but after consulting with Noble staff we were not fully confident in their accuracy. We therefore drop the entire cohort in the interest of caution, though adding them to our sample does not change any of our findings (these results are available from the authors upon request).

To meet increased demand, the Noble network opened two new schools every year between 2006 to 2009, and a single school in 2010. As a result, almost every applicant was offered a spot in at least one of the network's schools. Students entering subsequent lotteries would not complete high school in time to enter our analysis. Therefore, our analysis focuses on the three cohorts of students who entered a lottery between 2003 and 2005.

The raw lottery files include a student's name, gender, date of birth, address, eighth-grade school, sibling indicators, lottery result (accepted or waitlist), and waitlist position. Noble also provided us with enrollment records that include all students who enroll in a network school. We merge these files with the lottery data to identify which students eventually enrolled at Noble.

We provide a detailed accounting of our sample and lottery results in online Appendix table A.1. Pooling the 2003–05 cohorts together results in an initial pool of 1,089 lottery entrants. We exclude students from our analysis sample for one of two reasons. First, 118 applicants (10.8 percent of the initial sample) are the younger siblings of a student enrolled at Noble. We drop these students because they are not subject to the random variation required for our research design. Second, we do not observe dates of birth for a small group of students (thirteen in total, representing 1.2 percent of the original pool). These appear to be due to recording errors. Because students' precise dates of birth are required to match them to entries in the NSC database, we are forced to drop these students from our analysis. Although lottery winners are slightly more likely to suffer from missing data than losers (the raw means are 1.4 and 0.9 percentage points), the difference is not statistically significant and the frequency of missing data is so small that even the most severe bounding methods do not threaten our findings.10

The remaining 958 students compose our experimental sample. We define lottery winners as students who were immediately offered a position at Noble or who received one of the first ten waitlist positions.11 This categorization results in a final tally of 412 lottery winners and 546 lottery losers. The admissions offer is a very powerful predictor of enrollment—81.5 percent of students who are immediately offered admission attend Noble for at least one year, as well as half of the students who receive one of the top ten waitlist positions. Only fourteen applicants (2.6 percent) further down the waitlist ever receive and accept an admissions offer. As we discuss in our Results section, this powerful first-stage effect grants us a degree of statistical precision unavailable to prior studies in this literature.

Finally, for additional summary statistics and our nonexperimental analysis, we combine a variety of publicly available datasets provided by CPS.12 From these, we observe a variety of demographic and performance indicators averaged to the school-grade level. These data sources vary in their coverage. Test scores, free-lunch eligibility, bilingual status, and special education counts are only available after 2009. However, we observe racial composition, graduation rates, and dropout rates as early as 2000.

We supplement our microdata by linking these records to data on middle schools attended by students in our sample. We use three middle school characteristics from these data: (1) the percentage of eighth graders who score proficient or better on the math section of the Illinois Standard Achievement Test, (2) the percentage proficient on the reading section, and (3) the percentage of the student body that is black or Hispanic. In total, we are able to match 71.7 percent of our sample to their eighth-grade school's CPS summary records. The remaining students either had missing data, ambiguous data, or attended a private or parochial school for which CPS does not track demographics or test performance.

These aggregated data also provide our outcome measures and control variables for our nonexperimental analysis. For clarity, we describe this portion of the data alongside our analysis strategy in section 5.

#### College Enrollment Data

To track college-going, we submitted the name, date of birth, and expected high school graduation date of every student in the 2002 through 2005 lotteries to the NSC, a nonprofit organization that maintains a database of college enrollment and degree attainment. NSC returned this information with data on enrollment spells and graduation for every school attended by matched students. By necessity, we assume that students who were not matched to any records in the NSC database never enroll in college. In principle, both students and schools have the option of preventing researchers from accessing their records. This is not a significant concern for our analysis, however, as only 1.3 percent of the records we submitted were blocked by either the student or the school.

At the time of our submission, the NSC database included enrollment records at over 90 percent of colleges and universities in the United States. It is natural to ask, however, whether coverage is similarly thorough for the colleges targeted by students in our sample. Fortunately, Noble provided us with a list of college enrollment histories of their graduating seniors, based on their own NSC submission combined with input from college counselors. Although we do not use this information in any other part of our analysis, it allows us to observe the prevalence of false negatives in our treatment group. Out of 232 students graduating from Noble between 2007 and 2009, our 211 are flagged in our NSC data—a match rate of 90.9 percent. This suggests that NSC coverage is not likely to be a major issue for our analysis.13

NSC also provides information on the timing of enrollment that we use to construct measures of persistence. Each row in the NSC match corresponds to an enrollment period, with a fixed beginning and end date. The majority of schools in our matched sample report semester-long enrollments, spanning August/September to December or January/February to May/June. However, some schools use a quarter system or report continuous enrollment durations of over a year. To combine all of these different formats into a consistent structure, we generate a series of indicators for enrollment in the fall and spring semesters between 2006 and 2012. We consider students to have been enrolled in a given fall semester if they are enrolled for at least thirty days between 1 August and 31 December of that year. For spring semesters, we use endpoints of 1 January and 1 June. All of our results are robust to other plausible definitions of persistence, including the total number of days enrolled or the total time between the first and last enrollment.

### Empirical Approach

To measure the effect of Noble attendance on college outcomes, we estimate both intent to treat (ITT) and local average treatment effect (LATE) specifications. Let Zi be an indicator for receiving an offer to attend Noble. We estimate the following model via least-squares regression to recover ITT estimates:
$Yic=βXi+τZi+γc+φc1[siblingi]+ɛic.$
(1)

The ITT effect τ measures the difference between lottery winners and losers, adjusted for chance imbalances in the vector of student-level controls Xi. This can be interpreted as the causal impact of receiving an offer to attend Noble. To ensure that lottery status is not correlated with unobservable student characteristics, we drop students who are automatically accepted because they have an older sibling enrolled in a Noble school. All regressions also include lottery-cohort fixed effect (γc) and interactions with an indicator for having a sibling entering the same lottery (1[siblingi]), since siblings in the same cohort are admitted if either draws a winning number.14 When students enter multiple lotteries over a series of years, we only include the first entry.

Identification of τ requires only that lottery status Zi be uncorrelated with the error term εi. If we also assume monotonicity in the first stage (i.e., the lottery offer does not discourage anyone from enrolling) and that the exclusion restriction holds (i.e., the offer only affects college outcomes by changing Noble enrollment status), the causal effect of attending Noble is also identified for a certain subpopulation. In regression terms, we use lottery status Zi to instrument for enrollment at Noble to identify the LATE parameter. Letting Noblei denote an indicator for whether a student has ever enrolled at a Noble charter school, the second-stage equation can be written as follows:
$Yic=αXi+ρNoblei+δc+θc1[siblingi]+ηic.$
(2)

Estimating equation 2 by two-stage least squares produces an estimate of the LATE parameter, defined as the average effect of attending Noble for “compliers”—students who are induced to attend Noble by winning the lottery (Angrist and Imbens 1994). We report ITT estimates throughout the text except where noted. (The results of equation 2 are reported in table 2, alongside the corresponding ITT estimates.)

## 4.  Summary Statistics and Balance

Table 1 reports summary statistics for Noble students, other charter school students, and CPS students. The source for all of the statistics in this table is the publicly available, school-level data provided by CPS and described in the previous section. We show statistics for two separate time periods to provide a picture of Noble's early years (when the students in our experimental sample were in high school) as well as the later period after Noble's expansion had begun.

Table 1.
Characteristics of Chicago Public High Schools
Experimental Sample Period 2003—08Nonexperimental Sample Period 2009—13
Original Noble Campus (1)Other Charter (2)Traditional Public School (3)Noble Network (4)Other Charte (5)Traditional Public School (6)
Panel A: Student Body Characteristics
Black 0.115 0.712 0.493 0.349 0.619 0.449
Hispanic 0.825 0.214 0.361 0.602 0.324 0.413
Bilingual 0.053 0.045 0.068
Special education 0.116 0.138 0.135
Free-lunch eligible 0.885 0.888 0.849
Average cohort size 103.6 71.7 242.7 138.6 102.2 216.7
Panel B: Performance Measures
11th-grade math (ACT) 0.313 −0.277 0.004 1.028 −0.318 −0.003
11th-grade English (ACT) 0.446 0.059 −0.006 0.911 −0.215 −0.005
Five-year graduation rate 0.816 0.713 0.616
Five-year dropout rate 0.161 0.256 0.349
Number of schools 23 116 12 33 128
Experimental Sample Period 2003—08Nonexperimental Sample Period 2009—13
Original Noble Campus (1)Other Charter (2)Traditional Public School (3)Noble Network (4)Other Charte (5)Traditional Public School (6)
Panel A: Student Body Characteristics
Black 0.115 0.712 0.493 0.349 0.619 0.449
Hispanic 0.825 0.214 0.361 0.602 0.324 0.413
Bilingual 0.053 0.045 0.068
Special education 0.116 0.138 0.135
Free-lunch eligible 0.885 0.888 0.849
Average cohort size 103.6 71.7 242.7 138.6 102.2 216.7
Panel B: Performance Measures
11th-grade math (ACT) 0.313 −0.277 0.004 1.028 −0.318 −0.003
11th-grade English (ACT) 0.446 0.059 −0.006 0.911 −0.215 −0.005
Five-year graduation rate 0.816 0.713 0.616
Five-year dropout rate 0.161 0.256 0.349
Number of schools 23 116 12 33 128

Notes: This table reports sample averages for a variety of variables and populations. The full sample includes all Chicago Public Schools (CPS) high schools open at any point between 2003 and 2013. All results are calculated from school-level summary data published by CPS. We weight each observation by the number of enrolled students at each time period. The full sample includes all CPS high schools open at any point between 2003 and 2013. Bilingual status, special education statistics, free-lunch eligibility, and ninth- and tenth-grade test scores are not available before 2009. Five-year graduation (dropout) rates are the percentage of original freshmen who graduate (drop out) within five years. We adopt the convention that the graduation (dropout) rate at year t reflects the outcomes of students entering high school in year t − 5. All test scores are normalized to have mean zero and standard deviation 1 in each year. Because these variables are measured at the school level, one standard deviation in our sample is smaller than the true student-level standard deviation.

Like most Chicago schools, Noble serves students who are roughly 90 percent minority, though Noble enrolls a much larger share of Hispanic students than the average charter or traditional public school in Chicago. Other demographic data are not available prior to 2009. In more recent years, the network's collective student body is quite similar to the rest of the district on nonracial dimensions. Noble students are marginally less likely than traditional public school students to qualify for special education considerations (11.6 percent vs. 13.5 percent) but slightly more likely to be eligible for free or subsidized lunch (88.5 percent vs. 84.9 percent).

Even though Noble students are demographically similar to students at other schools, the differences in test performance are striking. Throughout our sample period, CPS students took the Explore test during the fall of their freshman year.15 Hence, ninth-grade test scores are a good measure of the incoming ability of each school's students. Based on these measures, Noble students in later cohorts enter high school with slightly lower test performance than the average traditional public school student (though significantly higher than the average charter student). However, by eleventh grade, Noble students overtake their peers and score markedly higher than the CPS average and the charter average on all sections of the ACT. In the 2013–14 school year, all ten Noble campuses with enrolled eleventh graders ranked in the top thirty-three CPS high schools in overall ACT performance (out of 156 schools with reported scores). Looking beyond test scores, students entering ninth grade at a Noble campus are 20.0 percentage points more likely to graduate within five years of enrollment than their peers in traditional public schools, and 18.8 percentage points less likely to drop out within five years of enrollment.

These summary statistics naturally give rise to questions about what sort of high schools our control group attends. Put differently, we would like to better understand the counterfactual outcome for our treatment group. Unfortunately, we do not observe what high school untreated students attend; our microdata only tell us whether students ever attended Noble. However, it is worth noting that the Chicago charter sector was extremely small at this time. The 2005 ninth-grade cohort had only eight charter high school options, and those eight schools educated a population 5 percent of the size of the traditional public school population.16 We also directly observe the middle schools attended by most applicants. Of those, less than 1 percent attended a charter middle school for eighth grade, largely reflecting the low supply of charter middle schools at the time (only fourteen were active, relative to 484 traditional public middle schools). Of course, the lottery population has revealed their preference for a charter school education, indicating they are likely not representative of the district at large. Nevertheless, given the dearth of other charter options, we believe it is likely that most lottery losers attended traditional public schools.

Although detailed school-level data are useful for comparing Noble's student body to the broader CPS population, we require student microdata to assess the balance of the admissions lottery. This is an important check—if lottery status is not determined randomly, neither of our empirical specifications is valid.

Our demographic data are somewhat limited because we do not have access to detailed, district-provided administrative data used in other studies exploiting charter school lotteries (e.g., Dobbie and Fryer 2011; Abdulkadiroglu et al. 2011; and Curto and Fryer 2014, among many others). We address this issue by constructing several indirect proxies for key socioeconomic variables. First, though we lack direct information on race and ethnicity, we can glean some useful information from students’ last names given the prevalence of Hispanic students in our sample. The Census Bureau provides a table containing surname-by-race frequencies for the 150,000 most common surnames, to which we matched 95 percent of the students in our sample.17 We code a student as Hispanic if 80 percent of the individuals with his/her last name are Hispanic (though our results are not sensitive to using other reasonable thresholds).18 Second, although we do not observe students’ lagged test scores, we can use average test scores at their middle school as a proxy.

Table 2 displays summary statistics and balance test results for the short list of student-level covariates that we construct from the lottery files. Columns 1 and 2 present means and standard deviations for lottery winners and losers, respectively. Column 3 reports the p-value from a regression of each characteristic on lottery status, controlling only for cohort fixed effects and a cohort-by-sibling-status effect.

Table 2.
Lottery Summary Statistics and Randomization Check
Lottery Winners (1)Lottery Losers (2)p-value (1) = (2) (3)
Panel A: Student Characteristics
Hispanic (inferred) 0.705 0.672 0.190
(0.457) (0.470)
Female 0.621 0.615 0.994
(0.486) (0.487)
Age at high school entry 14.52 14.556 0.810
(1.124) (1.177)
Average math proficiency 0.341 0.358 0.114
(0.152) (0.163)
Average reading proficiency 0.559 0.571 0.227
(0.138) (0.148)
Percent minority 0.904 0.903 0.749
(0.118) (0.118)
Missing baseline school data 0.269 0.289 0.554
(0.444) (0.454)
Observations 420 551 971
p-value from joint F-test   0.751
Lottery Winners (1)Lottery Losers (2)p-value (1) = (2) (3)
Panel A: Student Characteristics
Hispanic (inferred) 0.705 0.672 0.190
(0.457) (0.470)
Female 0.621 0.615 0.994
(0.486) (0.487)
Age at high school entry 14.52 14.556 0.810
(1.124) (1.177)
Average math proficiency 0.341 0.358 0.114
(0.152) (0.163)
Average reading proficiency 0.559 0.571 0.227
(0.138) (0.148)
Percent minority 0.904 0.903 0.749
(0.118) (0.118)
Missing baseline school data 0.269 0.289 0.554
(0.444) (0.454)
Observations 420 551 971
p-value from joint F-test   0.751

Notes: This table presents summary statistics and balance tests for students who entered the ninth-grade admissions lottery at the Noble Network of Charter Schools between 2003 and 2005. Students with an older sibling enrolled at Noble are dropped. All balance regressions include lottery fixed effects interacted with an indicator for having a sibling enter the same lottery. Hispanic ethnicity is inferred if 80 percent of Census respondents with the student's last name identify as Hispanic. Math and reading proficiency represent the percentage of eighth graders at each student's middle school who scored proficient or higher on the Illinois Standard Achievement Test one year before the lottery. Heteroskedasticity-robust standard errors are reported in parentheses.

The lotteries are well balanced. Noble attracts roughly three female applicants for every two male applicants, and this ratio is balanced across treatment and control. Among lottery winners, 70.5 percent have distinctly Hispanic last names, compared with a statistically indistinguishable 67.2 percent of lottery losers. Lottery losers attend middle schools with slightly higher levels of math and reading proficiency, though once again differences are small and statistically insignificant. Both lottery winners and losers went to middle schools where 90 percent of students were black or Hispanic. None of the differences in baseline variables is statistically significant, and a joint test of all six coefficients in table 2 yields a p-value of 0.751. Taken together, these results provide a strong case for the internal validity of our evaluation, suggesting that lottery imbalance is unlikely to be an issue for our analysis.

## 5.  Results

### Experimental Estimates

Although aggregated statistics show that students at Noble significantly outperform their CPS peers on standardized tests, it is not clear ex ante that these results would translate into improvements on medium-term outcomes. For instance, the differences may be driven by selection into Noble rather than any actual treatment effect. What's more, even if the difference is causal, the short-term treatment effect on test scores may not translate into long-term benefits if the effects are driven by test-specific preparation rather than true learning. Our lottery design does not allow us to directly measure underlying mechanisms, but we can provide clear evidence on whether attending Noble leads to increased college enrollment, quality, and persistence.

Panel A of table 3 shows the effect of Noble attendance on college enrollment outcomes. We report estimates of equations 1 and 2, controlling for student's gender, average test scores at the student's middle school, and student age to increase precision (our results are almost identical when we drop all demographic controls). Lottery winners are 10.0 (standard error [SE] = 3.1) percentage points more likely to enroll in college than lottery losers. For compliers, we estimate a LATE parameter of 13.5 (4.0) percentage points. Relative to the control mean of 58.8 percent, this is a 23 percent increase. If we restrict attention to “on-time” enrollments that begin the fall after a student's expected high school graduation dates, the ITT effect falls to 6.7 (3.3) percentage points but remains statistically significant.

Table 3.
The Effect of Attending a High-Quality Charter School on College Enrollment, Quality, and Persistence
Control Mean (1)ITT (2)LATE (3)
Panel A: Enrollment
Enrolled in college 0.588 0.100*** 0.131***
(0.031) (0.040)
Enrolled on time 0.43 0.064* 0.083**
(0.033) (0.042)
Panel B: Quality
Enrolled in 2-year college 0.427 0.037 0.048
(0.033 (0.042)
Enrolled in 4-year college 0.322 0.114*** 0.149***
(0.032) (0.042)
Median SAT > 1000 0.209 0.105*** 0.137***
(0.029) (0.038)
Panel C: Persistence
Two semesters or more 0.493 0.126*** 0.165***
(0.032) (0.042)
Four semesters or more 0.374 0.095*** 0.124***
(0.033) (0.042)
Graduated from college 0.149 0.007 0.010
(0.030) (0.042)
Control Mean (1)ITT (2)LATE (3)
Panel A: Enrollment
Enrolled in college 0.588 0.100*** 0.131***
(0.031) (0.040)
Enrolled on time 0.43 0.064* 0.083**
(0.033) (0.042)
Panel B: Quality
Enrolled in 2-year college 0.427 0.037 0.048
(0.033 (0.042)
Enrolled in 4-year college 0.322 0.114*** 0.149***
(0.032) (0.042)
Median SAT > 1000 0.209 0.105*** 0.137***
(0.029) (0.038)
Panel C: Persistence
Two semesters or more 0.493 0.126*** 0.165***
(0.032) (0.042)
Four semesters or more 0.374 0.095*** 0.124***
(0.033) (0.042)
Graduated from college 0.149 0.007 0.010
(0.030) (0.042)

Notes: This table reports the effect of attending a school in the Noble Network of Charter Schools on college enrollment, quality, and persistence. Column 1 reports the control mean for each outcome. Column 2 reports intent-to-treat (ITT) estimates following the specification described in the text. Column 3 estimates local average treatment effects (LATE), using lottery status to instrument for a binary variable indicating whether a student enrolled at Noble. All regressions control for gender; math and reading achievement at the student's middle school; the student's age; cohort fixed effects; and cohort effects interacted with an indicator for having a sibling entering the same lottery. Students who are automatically admitted because an older sibling is already enrolled at Noble are dropped. Our sample comprises 412 lottery winners and 546 lottery losers. On-time enrollment is defined as enrolling in the semester following each cohort's expected high school graduation date. Graduation effects exclude the 2005 lottery cohort, resulting in a sample size of 276 winners and 289 losers. Heteroskedasticity-robust standard errors are reported in parentheses.

*p < 0.1; **p < 0.05; ***p < 0.01.

To put these estimates in perspective, it is worth comparing our results with those of several recent papers studying the impact of improved high school quality on college outcomes. Deming et al. (2014) use school choice lotteries in Charlotte-Mecklenburg to instrument for attending a first-choice high school. Their estimates are not statistically significant when aggregated across all students and colleges, but their LATE estimate for attending a competitive college is 4.2 percentage points. Students who win the Harlem Children's Zone admissions lottery to Promise Academy are 17 percentage points more likely to enroll in college the fall after graduation, although lottery losers close the enrollment gap fairly quickly (Dobbie and Fryer 2015). Furthermore, this “treatment” includes middle school education, high school education, and a wide range of community investments. Results for Boston-area charters do not show a significant effect on overall college enrollment rates despite large increases in test scores (Angrist et al. 2016).19 In other words, the best evidence to date has shown improvements that are either transitory or much less robust than our estimates for Noble.

A natural concern is that the increase in college enrollment might come at the expense of quality. If Noble merely pushes marginal students into lower quality schools, then we would expect to see smaller effects on attending more rigorous and more competitive schools. The results in panel B of table 3 show that this is not the case. The treatment effect on attending a two-year college is statistically zero—an increase of 3.7 (3.3) percentage points—and the effect on four-year college attendance is 11.4 (3.2) percentage points, slightly larger than the ITT estimate for all colleges. We obtain similar results using the test scores of incoming college students to proxy for quality. Winning the lottery increases the probability that a student enrolls in a school where the median two-subject SAT score is above 1000 by 10.5 (2.9) percentage points.20 The LATE estimate (13.7 [3.8] percentage points) represents an increase of 65 percent over the control mean of 20.9 percent. Taken together, the results in panel B suggest that attending Noble increases both the likelihood and quality of college enrollment.

Finally, we turn to measures of college persistence in panel C. Lottery winners are 12.6 (3.2) percentage points more likely to enroll in at least two semesters and 9.5 (3.3) percentage points more likely to enroll in at least four semesters. Relative to the average control student, these effects represent increases of 26.0 percent and 25.9 percent, respectively. To calculate graduation effects, we drop students from the 2005 cohort because we would not observe graduation if they were making normal progress through high school and a four-year college.21 Using students from the 2003 and 2004 cohort only, we calculate a small and statistically insignificant effect on degree completion: 0.7 (3.0) percentage points, with a control mean of 14.9 percent.

For completeness, we also calculate and present persistence and graduation effects separately for two-year and four-year colleges. The results in online Appendix table A.2 confirm that the patterns we observe are driven by enrollment at four-year institutions. Treatment effects for two-year college outcomes are negligible; the only marginally significant estimate is a 5.5 (3.0) percentage point increase in the likelihood of enrolling for at least two semesters. The estimates for four-year college outcomes are quite similar to the main estimates in table 3.

To provide a fuller picture of enrollment behavior, figure 2 depicts how enrollment rates evolve over time. In each panel we normalize the time scale on the horizontal axis such that the date of the lottery corresponds to year 0. In other words, 4.5 on our axis corresponds to the first semester of college for a student making normal progress through school, 5 corresponds to the second semester, and so forth.

Panel A of figure 2 displays the percentage of students currently enrolled or graduated, separately for lottery winners and losers. Consistent with the on-time enrollment results, lottery winners are more likely to enroll in college immediately after graduating. They are also substantially more likely to be enrolled at every subsequent time period.22 Aside from a small increase in treatment-group enrollment in the second fall after expected graduation, the lines also slope down at a similar rate, indicating that persistence, conditional on starting college, is similar in the two groups. Panel B of figure 2 plots the percentage of students who have ever been enrolled in college. Both the treatment and control lines rise steeply in the first two years after expected high school graduation, indicating that substantial quantities of new students are enrolling after their predicted freshman fall. Importantly, however, the treatment line rises at roughly the same rate as the line for the control group. This indicates that Noble is truly increasing rates of college attendance rather than inducing students to enroll sooner than they otherwise would. Appendix figures 1 and 2 show these graphical results separately for two- and four-year college enrollment outcomes.

### Heterogeneity

To test for potential heterogeneity in our treatment effects, table 4 reports ITT results estimated separately by gender, neighborhood poverty rate, and middle school quality. We also report p-values on the null hypothesis that treatment effects are equal for each pair of groups.

Table 4.
The Impact of Attending a High-Performing Charter High School for Various Subgroups
GenderNeighborhood PovertyMiddle School QualityInferred Ethnicity
Female (1)Male (2)p-value (3)High (4)Low (5)p-value (6)High (7)Low (8)p-value (9)Hispanic (10)Non-Hispanic (11)p-value (12)
Enrolled in college 0.094** 0.112**  0.090** 0.113**  0.160*** 0.074  0.080** 0.110**
(0.039) (0.051) 0.785 (0.043) (0.044) 0.715 (0.051) (0.053) 0.245 (0.039) (0.056) 0.656
[0.620] [0.536]  [0.614] [0.559]  [0.580] [0.537]  [0.595] [0.605]
Enrolled in 2-year college 0.019 0.069  0.05 0.031  0.009 0.047  0.031 0.056
(0.042) (0.052) 0.457 (0.046) (0.046) 0.776 (0.056) (0.053) 0.617 (0.041) (0.061) 0.737
[0.472] [0.354]  [0.445] [0.406]  [0.440] [0.413]  [0.434] [0.436]
Enrolled in 4-year college 0.127*** 0.090*  0.108** 0.118***  0.193*** 0.085*  0.083** 0.133**
(0.041) (0.051) 0.569 (0.047) (0.045) 0.87 (0.055) (0.051) 0.151 (0.04) (0.061) 0.489
[0.323] [0.321]  [0.345] [0.297]  [0.320] [0.274]  [0.330] [0.326]
Graduated from college −0.01 0.031  −0.035 0.049  0.011 0.013  0.005 −0.019
(0.043) (0.039) 0.477 (0.043) (0.043) 0.164 (0.052) (0.048) 0.981 (0.038) (0.061) 0.736
[0.203] [0.062]  [0.174] [0.119]  [0.154] [0.147]  [0.137] [0.172]
Four semesters or more 0.085** 0.104**  0.049 0.140***  0.056 0.123**  0.075* 0.083
(0.042) (0.052)) 0.784 (0.046) (0.046) 0.159 (0.056) (0.052) 0.384 (0.041) (0.062) 0.913
[0.398] [0.335]  [0.407] [0.336]  [0.405] [0.303]  [0.385] [0.366]
Observations 593 365  482 476  340 372  620 285
GenderNeighborhood PovertyMiddle School QualityInferred Ethnicity
Female (1)Male (2)p-value (3)High (4)Low (5)p-value (6)High (7)Low (8)p-value (9)Hispanic (10)Non-Hispanic (11)p-value (12)
Enrolled in college 0.094** 0.112**  0.090** 0.113**  0.160*** 0.074  0.080** 0.110**
(0.039) (0.051) 0.785 (0.043) (0.044) 0.715 (0.051) (0.053) 0.245 (0.039) (0.056) 0.656
[0.620] [0.536]  [0.614] [0.559]  [0.580] [0.537]  [0.595] [0.605]
Enrolled in 2-year college 0.019 0.069  0.05 0.031  0.009 0.047  0.031 0.056
(0.042) (0.052) 0.457 (0.046) (0.046) 0.776 (0.056) (0.053) 0.617 (0.041) (0.061) 0.737
[0.472] [0.354]  [0.445] [0.406]  [0.440] [0.413]  [0.434] [0.436]
Enrolled in 4-year college 0.127*** 0.090*  0.108** 0.118***  0.193*** 0.085*  0.083** 0.133**
(0.041) (0.051) 0.569 (0.047) (0.045) 0.87 (0.055) (0.051) 0.151 (0.04) (0.061) 0.489
[0.323] [0.321]  [0.345] [0.297]  [0.320] [0.274]  [0.330] [0.326]
Graduated from college −0.01 0.031  −0.035 0.049  0.011 0.013  0.005 −0.019
(0.043) (0.039) 0.477 (0.043) (0.043) 0.164 (0.052) (0.048) 0.981 (0.038) (0.061) 0.736
[0.203] [0.062]  [0.174] [0.119]  [0.154] [0.147]  [0.137] [0.172]
Four semesters or more 0.085** 0.104**  0.049 0.140***  0.056 0.123**  0.075* 0.083
(0.042) (0.052)) 0.784 (0.046) (0.046) 0.159 (0.056) (0.052) 0.384 (0.041) (0.062) 0.913
[0.398] [0.335]  [0.407] [0.336]  [0.405] [0.303]  [0.385] [0.366]
Observations 593 365  482 476  340 372  620 285

Notes: This table reports the effect of attending a school in the Noble Network of Charter Schools for various subgroups of the data. We report intent-to-treat estimates throughout, controlling for the variables summarized in table 1, cohort fixed effects, and cohort effects interacted with an indicator for having a sibling entering the same lottery. Hispanic ethnicity is inferred if 80 percent of Census respondents with the student's last name identify as Hispanic. Graduation effects exclude the 2005 lottery cohort. Heteroskedasticity-robust standard errors are reported in parentheses. The control mean of each outcome within each group is reported in square brackets below the standard error.

*p < 0.1; **p < 0.05; ***p < 0.01.

There are few consistent patterns in the results, though we are underpowered to detect moderate differences. None of the estimated parameters are statistically different across subgroups. At most, we find suggestive evidence that effects on total enrollment and enrollment in four-year colleges are larger for students from high-quality middle schools—the opposite effect found by Deming et al. (2014). The differences are within the range of normal statistical variation, however, so we caution against interpreting them too strongly. Similarly, students from lower-poverty neighborhoods have higher estimated effects, though there are no statistically significant differences.

It is perhaps noteworthy that we do not observe any differential effects by gender, since, as Angrist et al. (2016) note, relatively few late-stage interventions show promising results for boys. In this respect, the lack of gender differences is consistent with other work examining the effect of No Excuses charters on college enrollment (Dobbie and Fryer 2015; Angrist et al. 2016). Similarly, using our surname-based classifications, we cannot detect any differences in treatment effectiveness between Hispanic and non-Hispanic students. (Recall that we do not directly observe race or ethnicity in our dataset, and our sample does not contain enough students with distinctly black or white last names to examine other racial subgroups.)

Finally, we investigated whether our treatment effect estimates differ across cohorts. If learning-by-doing helps teachers and school administrators prepare low-income students for college, one might expect Noble's treatment effect to increase over time. On the other hand, the network was preparing to open two new campuses in the fall of 2006, and it is possible that this effort diverted resources from the main campus. In figure 3, we plot ITT estimates of the effect on college enrollment separately for each lottery. The point estimates increase monotonically over time, reaching 16.6 (5.0) percentage point effect for the 2005 cohort. However, we cannot reject the null hypothesis of equal treatment effects across all three cohorts. At most, we can say that Noble's effectiveness did not appreciably decline over this period.

Figure 3.

Treatment Effects on College Enrollment by Lottery Cohort

Notes: See the notes for table 3 for details of the regression specification. The caption shows the results of an F-test of the null hypothesis that the treatment effect is constant across cohorts.

Figure 3.

Treatment Effects on College Enrollment by Lottery Cohort

Notes: See the notes for table 3 for details of the regression specification. The caption shows the results of an F-test of the null hypothesis that the treatment effect is constant across cohorts.

Figure 4a.

Nonexperimental Estimates of College Enrollment for Later Cohorts: Class of 2013

Figure 4a.

Nonexperimental Estimates of College Enrollment for Later Cohorts: Class of 2013

Figure 4b.

Nonexperimental Estimates of College Enrollment for Later Cohorts: Class of 2012

Notes: See the notes for table 5 for variable definitions. Ninth- and tenth-grade tests were administered in the fall semester for these cohorts. Ninth-grade test scores are not available for the class of 2012. All scores have been standardized to have mean zero and standard deviation 1. Because these variables are measured at the school level, one standard deviation in this sample is smaller than the true student-level standard deviation.

Figure 4b.

Nonexperimental Estimates of College Enrollment for Later Cohorts: Class of 2012

Notes: See the notes for table 5 for variable definitions. Ninth- and tenth-grade tests were administered in the fall semester for these cohorts. Ninth-grade test scores are not available for the class of 2012. All scores have been standardized to have mean zero and standard deviation 1. Because these variables are measured at the school level, one standard deviation in this sample is smaller than the true student-level standard deviation.

### External Validity

As with any randomized experiment, it is worth stepping back to consider the external validity of our estimates—that is, the extent to which these results might generalize to other settings. Any such discussion is necessarily somewhat speculative, as our experimental data cannot directly speak to generalizability. However, evidence from several recent papers can help us make some informed conjectures about the scope of our results.

First, there is strong evidence that replicating the practices of No Excuses charter schools is sufficient to produce significant gains in student achievement. Fryer (2014) evaluates school turnaround efforts in which low-performing public schools in Houston, Denver, and Chicago adopted educational practices common to successful charter schools. He finds large treatment effects in math. In a context closer to our own, Cohodes, Setren, and Walters (2018) show that successful Boston charter schools generate similar achievement gains when they open new campuses. Similarly, Tuttle et al. (2015) show that Knowledge is Power Program charter schools have sustained their achievement effects through a period of rapid growth (though the magnitude of the effects declined somewhat in the middle of their sample period).

Although these papers analyze effects on test scores, not college enrollment, it is noteworthy that similar charter school models have maintained their effectiveness as they scale. We interpret the findings as evidence that school policies and practices are the most important determinants of student outcomes; effectiveness is generally not determined by an unreplicable, unobserved ingredient found only at one campus.

Some researchers also note that lottery estimates of charter school treatment effects necessarily draw from oversubscribed schools (Tuttle, Gleason, and Clark 2012). Because these schools are more popular than undersubscribed charters (almost by definition), their effectiveness is unlikely to be representative of the entire charter sector. We therefore are careful to interpret our results as representative of No Excuses charters specifically, whose general popularity and success in raising test scores has been documented by a host of lottery-based studies (Angrist et al. 2010; Abdulkadiroglu et al. 2011; Dobbie and Fryer 2013, 2015; Tuttle et al. 2015).

At the same time, Chabrier, Cohodes, and Oreopoulos (2016) note that a significant portion of No Excuses charters’ effectiveness in increasing test scores is explained by their tendency to locate in poor, urban areas. As figure 1 shows, Noble campuses draw from these very neighborhoods. Therefore, we should not conclude that Noble schools would generate similar treatment effects for wealthy, suburban, or rural students. Given the immense remaining socioeconomic and racial disparities in college-going, however, we believe that increasing college enrollment among underserved communities is still a desirable and policy-relevant goal.

Given the balance of the evidence, we believe it is reasonable to conclude that our lottery findings are likely to extend to similar charter schools in urban areas. We test this assumption directly using publicly available data on other Noble campuses. These nonexperimental results are consistent with our lottery estimates, though their validity relies on much stronger identifying assumptions.

### Nonexperimental Estimates

The experimental results clearly demonstrate that early cohorts attending Noble Street Charter Schools are more likely to enroll in college, enroll in selective four-year institutions, and remain enrolled for at least four semesters. We have interpreted these results to mean that the academic practices and policies practiced throughout the Noble network lead to improved college outcomes. One possible objection to this interpretation, of course, is that we have thus far only analyzed students attending one campus. Even though other Noble schools follow the same general practices and policies, it is possible there is something unique about the original campus that accounts for the effects we estimate.

As we discussed in section 3, however, our lottery design is not feasible for later cohorts of Noble students. The network's rapid expansion absorbed the excess demand that is essential for our identification strategy. We must therefore turn to nonexperimental methods and aggregated data sources to provide evidence on the external validity of our estimates.23 As discussed earlier, CPS publishes a variety of demographic and performance measures at the school–grade level. Most important for our purposes are the college enrollment data. Using their own NSC data pull, CPS has calculated and released the average college enrollment rate for every high school between 2010 and 2013. We use these as a performance measure for later cohorts.24

Estimating a plausible specification also requires, at the very least, a measure of the baseline ability of each graduating class. Ideally, we would observe middle school performance outcomes for each senior class, allowing us to construct ability measures that do not reflect any treatment impacts. However, because we only observe school-level averages, this design is not feasible. Fortunately, as mentioned earlier, during our sample period all CPS students took standardized tests in the fall of their freshman year. We can therefore use each cohort's average score on their ninth-grade Explore exam. These measurements are taken so early in students’ high school careers that we can plausibly assume they reflect incoming ability, not the influence of the school they attend.

The basis of our empirical strategy is to use incoming ability to predict college-going rates for that cohort, as can be seen in figure 4. Each dot represents the graduating class at a single high school. On the horizontal axis of panel A, we plot the school's average comprehensive score on the Explore exam taken in the fall of 2009. The vertical axis records 2013 college enrollment rates. As is immediately apparent, this measure of baseline ability explains most of the between-school variation in college enrollment—the R2 statistic of the quadratic fit plotted in the figure is over 70 percent.

The shaded markers denote the seven Noble campuses with graduating seniors in 2013. Despite their location in the middle of the incoming-score distribution, their college enrollment rates are among the highest in the district. In particular, all seven campuses are located well above the regression line. Many more Noble graduates enter college than one would predict based on their incoming ability.

Test results are not available prior to 2009. Thus, we can only observe both ninth-grade test scores and college enrollment rates for a single cohort (the class of 2013). To provide further evidence, however, we can repeat the exercise using tenth-grade test scores for the class of 2012. Because this measurement is taken after students have already spent a year in high school, this is not an ideal control. In particular, we might expect some of the early achievement gains induced by Noble to be reflected in tenth-grade scores. If this is the case, we would underestimate the true effect of attending Noble on college-going. With these caveats in mind, panel B displays the same graphical results for the class of 2012. Reassuringly, the general pattern is similar to panel A. Noble's enrollment rates are among the best in the district, and rates at all five schools surpass the best-fit line by a wide margin.

For more detail, and to investigate the robustness of this approach, table 5 presents regression results analogous to the plots in figure 4. Each column shows the results of regressing each school's college enrollment rate on the baseline ability measure, its square, and a varying set of controls. In all specifications we weight each observation by the number of students in the ninth-grade class.

Table 5.
Nonexperimental Estimates for Two Later Cohorts
Class of 2013Class of 2012
(1)(2)(3)(4)(5)(6)
Noble 0.193*** 0.098** 0.128*** 0.117*** 0.022 0.096**
(0.02) (0.042) (0.041) (0.02) (0.037) (0.043)
Charter  0.104** 0.099***  0.100*** 0.076***
(0.04) (0.038)  (0.034) (0.029)
(0.012) (0.012) (0.025)
(0.005) (0.005) (0.005)
(0.014) (0.014) (0.03)
(0.005) (0.005) (0.006)
Free-lunch eligible   (0.08   0.084
(0.135)   −0.179)
Special education   −0.708***   (0.432)
(0.239)   (0.274)
Bilingual   0.064   −0.233*
(0.19)   (0.128)
Black   −0.06   −0.032
(0.064)   (0.074)
Hispanic   −0.159**   −0.147*
(0.073)   (0.082)
Observations 109 109 109 102 102 102
R2 0.774 0.801 0.843 0.731 0.755 0.845
Class of 2013Class of 2012
(1)(2)(3)(4)(5)(6)
Noble 0.193*** 0.098** 0.128*** 0.117*** 0.022 0.096**
(0.02) (0.042) (0.041) (0.02) (0.037) (0.043)
Charter  0.104** 0.099***  0.100*** 0.076***
(0.04) (0.038)  (0.034) (0.029)
(0.012) (0.012) (0.025)
(0.005) (0.005) (0.005)
(0.014) (0.014) (0.03)
(0.005) (0.005) (0.006)
Free-lunch eligible   (0.08   0.084
(0.135)   −0.179)
Special education   −0.708***   (0.432)
(0.239)   (0.274)
Bilingual   0.064   −0.233*
(0.19)   (0.128)
Black   −0.06   −0.032
(0.064)   (0.074)
Hispanic   −0.159**   −0.147*
(0.073)   (0.082)
Observations 109 109 109 102 102 102
R2 0.774 0.801 0.843 0.731 0.755 0.845

Notes: This table reports nonexperimental estimates of the effect of attending a school in the Noble Network of Charter Schools for two recent cohorts of students. The sample includes all Chicago Public Schools high schools with graduating seniors in 2013 (columns 1—3) or 2012 (columns 4—6). All variables are school-level averages. The dependent variable in each regression is the fraction of graduating seniors who enroll in college. Each observation is weighted by the number of students in the original ninth-grade class. Test score controls are the average composite score on the Explore (ninth grade) and Plan (tenth grade) standardized test, normalized to have mean zero and standard deviation 1. Because these variables are measured at the school level, one standard deviation in this sample is smaller than the true student-level standard deviation. Ninth-grade test scores are not available for the class of 2012. Heteroskedasticity-robust standard errors are reported in parentheses.

*p < 0.1; **p < 0.05; ***p < 0.01.

Columns 1 and 4 reiterate the graphical argument above. Controlling only for a quadratic in baseline ability, we estimate the effect of attending Noble on college attendance to be 19.2 (2.1) percentage points for the class of 2013 and 12.0 (2.0) percentage points for the class of 2012. Columns 2 and 5 add an indicator variable for charter schools to this specification. Because Noble is itself a charter school, one should add this coefficient to the Noble effect when comparing Noble to traditional CPS public schools. The sum of the Noble and charter coefficient is quite similar to the earlier estimates, as one would expect. For the class of 2012, however, we cannot reject the hypothesis that Noble is as effective as the average charter school. When we add a series of demographic control variables, however, Noble's performance exceeds even other charter schools in both years—we estimate effects of 12.8 (9.9) and 9.6 (7.6) percentage points.

We should be careful when interpreting the nonexperimental results, however. The research design does not benefit from the random variation used in our earlier analysis, so a causal interpretation requires an admittedly strong selection-on-observables assumption. Nevertheless, we find it reassuring that the best evidence we can muster indicates that Noble students continue to outperform expectations even during the network's rapid expansion.

## 6.  Discussion

Critics have long contended that the academic success demonstrated in many high-performing charter schools can be explained by test preparation and a paternalistic environment that does not improve true academic skills or promote noncognitive development. This paper presents evidence from the Noble Network of Charter Schools in Chicago indicating that No Excuses schools also have a positive impact on college attendance, quality, and persistence. We believe this is a cause for cautious optimism, as Noble's educational model is broadly consistent with the practices of other high-performing charter schools.

Our analysis of Noble's impact on college outcomes builds upon studies of other high-performing charter schools in New York and Massachusetts that use similar practices, including extending the length of the school day and year, facilitating regular small group instruction, using data to plan lessons and target remediation, and providing regular feedback to teachers on their instruction (Dobbie and Fryer 2015; Angrist et al. 2016). We provide the most robust evidence to date that the impacts of these practices are both broad and long-lasting: our point estimates are large and precisely estimated, we see no evidence of fade-out, and the effects are driven by enrollment at selective, four-year institutions.

Furthermore, our results connect to a growing body of literature that indicates the achievement gains of the No Excuses approach is broadly replicable (Fryer 2014; Tuttle et al. 2016; Cohodes, Setren, and Walters 2018). Although our data only allow us to estimate lottery-based models for a single campus, we use nonexperimental methods to test whether the gains were preserved as the network expanded. Encouragingly, these results are in line with our experimental findings.

It is important to emphasize that our new findings are not in conflict with prior work on the medium-run effects of high-performing charter schools. Rather, our contributions are primarily a function of improved statistical power, stemming from a particularly strong first-stage attendance effect. Viewed in concert with work by Dobbie and Fryer (2015) and Angrist et al. (2016), we see an increasingly clear picture of a rapidly growing style of school management25 that boosts the college enrollment of poor urban high schoolers. Unfortunately, the similarity of the educational approaches considered in each study means that we can say very little about what mechanisms might be driving our results (at most, one might infer that the neighborhood-level interventions of the Harlem Children's Zone are not essential for increasing college enrollment, a conclusion shared by Dobbie and Fryer [2015]). Accordingly, understanding which No Excuses strategies are most important for generating long-term gains is an important topic for future research. Nevertheless, we are confident this educational model has proved to be—and will continue to be—an effective part of the fight to increase low-income students’ human capital.

## Notes

1.

See, for example, Angrist et al. (2010), Tuttle et al. (2010), Abdulkadiroglu et al. (2011), Dobbie and Fryer (2011), and Curto and Fryer (2014).

2.

In the National Longitudinal Survey of Youth 1979 sample (NSLY79), a one-standard-deviation increase in a student's Armed Forces Qualifying Test (AFQT) score is associated with a 23.2 percentage point increase in the likelihood of attending college (authors' calculations based on ordinary least squares regression of college attendance on AFQT scores and a full set of race dummy variables for youths born before 1961). Controlling for test scores in the NLSY79 sample is sufficient to close—and even reverse—the black–white gap in college attendance (Lang and Manove 2011).

3.

Booker et al. (2011) find that charter attendance is positively correlated with college enrollment after controlling for observable characteristics of students, and Sass et al. (2016) show these effects extend to college persistence. Using similar methods, Dobbie and Fryer (2016) find that charter attendance has a small effect on two-year college enrollment only, though schools adhering to the No Excuses philosophy also improve enrollment at four-year colleges. Because these estimates do not benefit from random or quasi-experimental variation in charter enrollment, however, they require stronger identifying assumptions than studies exploiting admissions lotteries.

5.

Other well-known No Excuses charter management organizations include Yes Prep (Houston), MATCH (Boston), Mastery (Philadelphia), Excel (Boston), Green Dot (Los Angeles), Success Academies (New York City), and Uncommon Schools (New Jersey, New York, and Massachusetts).

6.

Beginning with the 2012–13 school year, CPS and the Chicago Teacher's Union agreed to extend the school day by 34 minutes (to 7.5 hours per day) and extend the school year by two weeks (to 180 days per year).

7.

Through the 2013–14 school year, one unique aspect of Noble's culture of high expectations was their use of financial penalties to disincentivize undesirable behavior. For example, students who earned three demerits for rules infractions not only earned a three-hour detention, but were also required to pay a $5 fine. Students with twelve or more detentions were required to cover fees totaling$140 for a behavior improvement class. Starting with the 2014–15 school year, this practice is no longer used by any school in the Noble network.

8.

The online Appendix can be accessed on Education Finance and Policy’s Web site at www.mitpressjournals.org/doi/suppl/10.1162/edfp_a_00244.

9.

If both students are put on the waitlist, they are both assigned to the more favorable waitlist position.

10.

More precisely, we can assess the maximal extent to which attrition might bias our results if we impute missing outcomes according to the “worst case”' scenario for our results—that is, imputing zeros for treated students and ones for control students. In this scenario, our main treatment effect estimates decline by between 1 and 2 percentage points. Their statistical significance is unchanged. These results are available from the authors upon request.

11.

Because the waitlist order is assigned randomly, using any cutoff to assign treatment and control is a valid instrument for attending Noble. We chose to include the top ten students in treatment based on an informal review of the lottery files; as online Appendix table A.1 shows, many students near the top of the waitlist end up attending Noble, while the likelihood of attending drops off considerably after ten. Using a lower threshold or assigning all waitlisted students to the control group does not change our results.

12.

These data can be accessed here: http://cps.edu/SchoolData/Pages/SchoolData.aspx.

13.

In our analysis, we code these twenty-one students as never enrolling in college. If we had a similar external source on the college enrollments of control students, we could plausibly update our dataset. Given that we do not have such information, correcting only the treated false negatives (or dropping them from our sample) would positively bias our estimated treatment effects.

14.

None of our results is sensitive to dropping students with siblings in the same lottery.

15.

Both the Explore exam and the tenth-grade Plan exam are produced by ACT, Inc., and test the same four subjects as the ACT test: math, reading, English, and science. Both exams consist of a separate, multiple-choice section for each of the four subjects. We use the single “composite” score calculated by test providers throughout our analysis, except where noted.

16.

Table 1 includes data from twenty-three charter schools because the “experimental sample period” extends forward until 2008.

17.

The data are available at https://www2.census.gov/topics/genealogy/2000surnames/. We use the 2000 Census data because it is the decennial Census closest in time to the lotteries.

18.

Consumer Financial Protection Bureau (2014) and Elliot et al. (2009) show that surname-based proxies are very strong predictors of Hispanic ethnicity in samples of mortgage applicants and health plan enrollees, respectively. In particular, adding home ZIP Codes to the predictive model only marginally improves one's ability to accurately classify Hispanic ethnicity in either population.

19.

It should be noted that our point estimates are fairly close to those of Angrist et al. (2016), who estimate an 11.5 (8.4) percentage point increase in enrollment. Our estimates are substantially more precise, however, owing to an extremely powerful first stage. Across the three cohorts we consider, 79.5 percent of students offered admission enroll at Noble, compared with only 2.5 percent of control students.

20.

We obtained average SAT scores for entering freshmen from the U.S. News and World Report. Certain schools provide average ACT scores instead of SAT scores. We use the ACT's concordance table (available at www.act.org/solutions/college-career-readiness/compare-act-sat/) to translate these scores into the equivalent SAT score.

21.

We obtained our NSC data in November 2012. If a member of the 2005 lottery cohort made regular progress through high school and immediately entered a four-year college after graduating, she would have been a college senior at the time.

22.

As we only observe seven semesters of expected college enrollment for the 2005 cohort, we truncate the graph at this point to maintain a consistent sample.

23.

We made several attempts over the course of three years to enter into a data-sharing agreement with CPS that would allow us to analyze confidential microdata, but none of our attempts was successful.

24.

CPS calculates college enrollment rates by dividing the number of enrollments by the number of graduating seniors. Of course, not every student who enrolls as a ninth grader eventually graduates, introducing the potential for selection bias. Although we cannot directly rule out this explanation without student-level panel data, two facts suggest that this is not a major concern in our setting. First, we obtain similar results when we calculate the enrollment rates as a fraction of all students who start ninth grade at a given school. This suggests that Noble does not achieve its results by “weeding out” weaker students before graduation. Second, recall from table 1 that Noble's dropout rates are significantly lower than other CPS schools, and their graduation rates are much higher. We interpret these facts as evidence that our nonexperimental effects are not driven by deselecting students after they start high school.

25.

For instance, Knowledge is Power Program now manages 200 schools and enrolls 80,000 students. Noble operates 16 high schools in Chicago.

## Acknowledgments

Special thanks to Kyle Cole, Tom Mathews, Jessica Flores, and Noble Street Charter School staff for their support and leadership during this project. We are grateful to Roland Fryer, Josh Goodman, and Marty West for helpful comments and suggestions. A previous version of this paper circulated under the title “Do ‘No Excuses’ Charter Schools Raise More than Test Scores? Evidence from a High School Network in Chicago.” Financial support from the Education Innovation Laboratory at Harvard University and a National Science Foundation Graduate Research Fellowship is gratefully acknowledged. All errors are our own.

## REFERENCES

,
Atila
,
Joshua
Angrist
,
Susan
Dynarski
,
Thomas J.
Kane
, and
Parag
Pathak
.
2011
.
Accountability in public schools: Evidence from Boston's charters pnd Pilots
.
Quarterly Journal of Economics
126
(
2
):
699
748
.
,
Atila
,
Joshua
Angrist
,
Yusuke
Narita
, and
Parag
Pathak
.
2015
.
Research design meets market design: Using centralized assignment for impact evaluation
.
NBER Working Paper No. 21705
.
Acemoglu
,
Daron
, and
David H.
Autor
.
2010
. Skills, tasks and technologies: Implications for employment and earnings. In
Handbook of labor economics
, vol.
4B
,
edited by
Orley
Ashenfelter
and
David
Card
, pp.
1043
1171
.
San Diego, CA
:
Elsevier B.V
.
Angrist
,
Joshua D.
,
Sarah R.
Cohodes
,
Susan M.
Dynarski
,
Parag A.
Pathak
, and
Christopher R.
Walters
.
2016
.
Stand and deliver: Effects of Boston's charter high schools on college preparation, entry, and choice
.
Journal of Labor Economics
34
(
2
):
275
318
.
Angrist
,
Joshua D.
,
Susan M.
Dynarski
,
Thomas J.
Kane
,
Parag A.
Pathak
, and
Christopher R.
Walters
.
2010
.
Inputs and impacts in charter schools: KIPP Lynn
.
American Economic Review
100
(
2
):
239
243
.
Angrist
,
Joshua D.
, and
Guido
Imbens
.
1994
.
Identification and estimation of local average treatment effects
.
Econometrica
62
(
2
):
467
475
.
Autor
,
David H.
,
Lawrence F.
Katz
, and
Melissa S.
Kearney
.
2008
.
Trends in U.S. wage inequality: Revising the revisionists
.
Review of Economics and Statistics
90
(
2
):
300
323
.
Barrow
,
Lisa
, and
Cecilia E.
Rouse
.
2005
.
Does college still pay
?
The Economists’ Voice
2
(
4
):
Article 3
.
Booker
,
Kevin
,
Brian
Gill
,
Tim
Sass
, and
Ron
Zimmer
.
2011
.
The effects of charter high schools on educational attainment
.
Journal of Labor Economics
29
(
2
):
377
415
.
Chabrier
,
Julia
,
Sarah
Cohodes
, and
Philip
Oreopoulos
.
2016
.
What can we lean from charter school lotteries
?
Journal of Economic Perspectives
30
(
3
):
57
84
.
Chetty
,
Raj
,
John
Friedman
, and
Jonah
Rockoff
.
2014
.
Measuring the impacts of teachers II: Teacher value-added and student outcomes in adulthood
.
American Economic Review
104
(
9
):
2633
2679
.
Cohodes
,
Sarah
,
Elizabeth
Setren
, and
Christopher R.
Walters
.
2018
.
Can successful schools replicate? Scaling up Boston's charter school sector
.
School Effectiveness and Inequality Initiative Discussion Paper No. 2016.06
.
Consumer Financial Protection Bureau
.
2014
.
Using publicly available information to proxy for unidentified race and ethnicity
.
Available
http://files.consumerfinance.gov/f/201409_cfpb_report_proxy-methodology.pdf".
Accessed 24 September 2018
.
Curto
,
Vilsa
, and
Roland G.
Fryer
.
2014
.
The potential of urban boarding schools for the poor: Evidence from SEED
.
Journal of Labor Economics
32
(
1
):
65
93
.
Deming
,
David J.
,
Justine S.
Hastings
,
Thomas J.
Kane
, and
Douglas O.
Staiger
.
2014
.
School choice, school quality, and postsecondary attainment
.
American Economic Review
104
(
3
):
991
1013
.
Dobbie
,
Will
, and
Roland G.
Fryer
.
2011
.
Are high-quality schools enough to increase achievement among the poor? Evidence from the Harlem Children's Zone
.
American Economic Journal: Applied Economics
3
(
3
):
158
187
.
Dobbie
,
Will
, and
Roland G.
Fryer
.
2013
.
Getting beneath the veil of effective schools: Evidence from New York City
.
American Economic Journal: Applied Economics
5
(
4
):
28
60
.
Dobbie
,
Will
, and
Roland G.
Fryer
.
2015
.
The medium-term impact of high-achieving charter schools
.
Journal of Political Economy
123
(
5
):
985
1037
.
Dobbie
,
Will
, and
Roland G.
Fryer
.
2016
.
Charter school and labor market outcomes
.
NBER Working Paper No. 22502
.
Dynarski
,
Susan M.
2008
.
Building the stock of college-educated labor
.
Journal of Human Resources
43
(
3
):
576
610
.
Elliot
,
Marc N.
,
Peter A.
Morrison
,
Allen
Fremont
,
Daniel F.
McCaffrey
,
Philip
Pantoja
, and
Nicole
Lurie
.
2009
.
Using the Census Bureau's surname list to improve estimates of race/ethnicity and associated disparities
.
Health Services and Outcomes Research Methods
9
:
69
82
.
Fryer
,
Roland
.
2014
.
Injecting charter school best practices into traditional public schools: Evidence from field experiments
.
Quarterly Journal of Economics
129
(
3
):
1355
1407
.
Goldin
,
Claudia
, and
Lawrence F.
Katz
.
2007
.
Long-run changes in the U.S. wage structure: Narrowing, widening, polarizing
.
NBER Working Paper No. 13568
.
Greenstone
,
Michael
, and
Looney
.
2011
.
Where is the best place to invest \$102,000: In stocks, bonds, or a college degree?
Washington, DC
:
The Brookings Institute
.
Heckman
,
James
.
2012
.
The case for investing in disadvantaged young children
.
Available
www.eenee.de/dms/EENEE/Policy_Briefs/PolicyBrief1-2012.pdf".
Accessed 24 September 2018
.
Heckman
,
James
,
Lance
Lochner
, and
Petra
Todd
.
2008
.
Earnings functions and rates of return
.
NBER Working Paper No. 13780
.
Lang
,
Kevin
, and
Michael
Manove
.
2011
.
Education and labor market discrimination
.
The American Economic Review
101
(
4
):
1467
1496
.
Oreopoulos
,
Philip
, and
Uros
Petronijevic
.
2013
.
Making college worth it: A review of research on the returns to higher education
.
NBER Working Paper No. 19053
.
Ravitch
,
Diane
.
2010
.
The death and life of the great American school system: How testing and choice are undermining education
.
:
Basic Books
.
Sass
,
Tim R.
,
Ron W.
Zimmer
,
Brian P.
Gill
, and
T.
Kevin Booker
.
2016
.
Charter high schools' effects on long-term attainment and earnings
.
Journal of Policy Analysis and Management
35
(
3
):
683
706
.
Thernstrom
,
Abigail
, and
Stephan
Thernstrom
.
2004
.
No Excuses: Closing the racial gap in learning
.
New York
:
Simon & Schuster
.
Tuttle
,
Christina Clark
,
Bing-ru
Teh
,
Ira
Nichols-Barrer
,
Brian P.
Gill
, and
Philip
Gleason
.
2010
.
Student characteristics and achievement in 22 KIPP middle schools: Final report
.
Ava-ilable
https://www.mathematica-mpr.com/our-publications-and-findings/publications/student-characteristics-and-achievement-in-22-kipp-middle-schools.
Accessed 25 September 2018
.
Tuttle
,
Christina Clark
,
Philip
Gleason
,
Virginia
Knechtel
,
Ira
Nichols-Barrer
,
Kevin
Booker
,
Gregory
Chojnacki
,
Thomas
Coen
, and
Lisbeth
Goble
.
2015
.
Understanding the effect of KIPP as it scales: Volume
I
,
impacts on achievement and other outcomes
.
Available
https://www.mathematica-mpr.com/-/media/publications/pdfs/education/kipp_scale-up_vol1.pdf?la=en.
Accessed 24 September 2018
.
Tuttle
,
Christina Clark
,
Philip
Gleason
, and
Melissa
Clark
.
2012
.
Using lotteries to evaluate schools of choice: Evidence from a national study of charter schools
.
Economics of Education Review
31
(
2
):
237
253
.
Whitman
,
David
.
2008
.
Sweating the small stuff: Inner-city schools and the new paternalism
.
Washington, DC
:
Thomas B. Fordham Institute
.