Does Greater Autonomy Improve School Performance? Evidence from a Regression Discontinuity Analysis in Chicago

In an era of educational accountability, school leaders face increasing pressure to align their efforts with the educational needs of students and to improve school performance. To achieve these goals, policy makers have paid increasing attention to schools’ organizational capacity and the institutional relationship between schools and large, complex district bureaucracies. Recognizing that decision makers at the school level are most aware of and potentially most able to efficiently respond to schools’ organizational needs, school districts throughout the United States have recently begun granting more autonomy to local public schools. Over the past decade, decentralization of decision-making authority to local schools has unfolded in some of the nation's largest school districts.¹

Decentralization of decision making as an education reform strategy, however, has a long history in the United States. In response to the growth in state funding in the 1970s, school-based (site-based) management (SBM) gained traction in the 1980s among education reformers who believed greater school-based autonomy could improve school performance (Chubb and Moe 1990). SBM was often implemented by appointing a council at each school, made up of teachers, parents, and community members, that was given some responsibility for budget, personnel, and curriculum; in practice, however, local school councils tended to have advisory rather than decision-making authority (Wohlstetter and Buffett 1992; Mohrman and Wohlstetter 1994). By the early 1980s, approximately 60 percent of school districts with enrollments of at least 50,000 students had experimented with some form of SBM (Van Langen and Dekkers 2001), but there is little evidence that these structural reforms had any impact on school performance (Fullan 1993). Around the same time, researchers began to attend to the relationship between school achievement and how schools and school districts are structured and make decisions, with much evidence pointing to the principal's central role in school governance (Purkey and Smith 1983).

In the most recent iteration of decentralization, the locus of decision-making resides with principals rather than local school councils. The extent of autonomy has also expanded to include teacher professional development, school climate, safety strategies, and the school's calendar and schedule, alongside curriculum and budget. SBM programs, by contrast, generally limited control to only one or two areas of decision-making authority among budget, personnel, and curriculum decisions (Wohlstetter and Mohrman 1996). Moreover, whereas SBM historically gave local school councils budgetary authority, evidence suggests this authority was nominal at best—upward of 95 percent of the school budget was often determined before dollars were allocated to the school (Wohlstetter and Mohrman 1996).

More recently, federal education policy has reflected the shift toward decentralization of decision-making authority to principals. Under the 2010 Race to the Top (RTTT) competition, the U.S. Department of Education gave priority to states that not only expanded the number of authorized charter schools but also instituted a broader reform strategy enabling local school districts to operate autonomous public, non-charter schools. Under RTTT, autonomous public schools have greater authority to define their curriculum, hire and replace staff, organize their school day or year, and control their budget (USDOE 2009).²

Giving principals greater autonomy moves the locus of decision making to the organizational level where schools provide educational services, the goal of which is to improve student learning and achievement. Scholars argue that school leaders (in most cases, the principal) must control the key mechanisms of an educational governance system (Hansen and Roza 2005). These mechanisms include decisions pertaining to school budgets, personnel and staffing, curriculum and instruction, and general operational and administrative conditions (such as the organization of the school day and calendar) (Ouchi 2006). Of these four mechanisms, Ouchi (2004) notes that budgetary control is the crucial element and that “true decentralization” requires that budgetary control be given to each school (p. 22). The theory of comparative advantage provides one justification for school-based autonomy (Hill 2004; Hansen and Roza 2005). Because educational production (student learning) and school-based inputs to the educational production function (curriculum, instruction, and classroom management) occur at the school level, school-level professionals are strategically positioned to organize instruction and respond to students’ individual learning needs (Hansen and Roza 2005). Indeed, many organizational theory scholars believe that large bureaucracies—state agencies with at least 3,000 employees—can realize improvements in performance only through decentralized decision making (Ouchi and Goldschmidt 2009). On the other hand, some scholars argue that one of the main disadvantages of decentralization is that some communities are unable and unequipped to handle the complex problems associated with local management. Indeed, these scholars note that large public agencies play a fundamentally important role in recruiting human capital (De Vries 2000). In the context of urban schools, this amounts to the district assuming responsibility for recruiting and hiring qualified teachers and skilled school staff. The major concern regarding greater autonomy is the extent to which an unequal distribution of school resources exists. In districts where resource quality, such as the human capital capacity of teachers and school principals, varies across schools, the decentralization of decision-making authority may exacerbate the unequal distribution of schooling outcomes.

Whether school-level professionals are better positioned than central administrators to organize local schools for improvement depends on a number of factors. Among them is the capacity of local school leaders to more effectively identify areas for improvement (among the school-based inputs to educational production) and make changes to school organization in ways that lead to improvements in instruction and student achievement. If, however, the costs (both informational and transitional) to principals and teachers associated with the identification and implementation of organizational change are large, we might not expect improvements in school performance. Indeed, when an institution embarks on new innovations requiring the leader to engage new knowledge and skills, such as those required of principals under decentralization, transition difficulties may produce an implementation dip—a decline in organizational performance as leaders encounter an innovation that requires new skills and new understandings of organizational management (Fullan 2001; Herold and Fedor 2008). As a consequence, greater autonomy may lead to short-term declines in school performance as principals learn how to best use their new organizational and managerial options. The timing of organizational change is also critical. If principals have little time to translate their new autonomy into meaningful changes to how schools operate, it is unlikely that we would see any significant changes in performance. If principals become more adept over time at incorporating new leadership responsibilities into their management of school operations, however, schools may realize longer-term performance improvements. The extent to which short-term outcomes are impacted by principals’ new decision-making authority is an empirical question addressed in the context of this paper.

As local school districts across the United States are increasingly encouraged to take innovative approaches to school reform strategies, we need a clearer understanding of greater school-based autonomy's potential to improve school performance. Despite the emphasis on reform strategies that decentralize decision-making authority, little evidence exists on the impact that greater local autonomy (among non-charter schools) has on school performance. In a multi-district study of decentralization, Ouchi and Goldschmidt (2009) show that the provision of school autonomy is correlated with student achievement through the school's student–teacher ratio (which they refer to as Total Student Load or TSL); however, the authors do not provide evidence of a direct effect of school-based autonomy on student performance.³ Lottery-based estimates from Boston find that students attending pilot schools, which have decision-making authority over school budgets, curriculum, and scheduling but remain part of the Boston Public School district, realized mixed achievement results in math and reading (Abdulkadiroglu et al. 2011). In the international context, a recent cross-national comparison of decentralized decision-making authority finds that autonomy reforms are related to improvements in student achievement in countries with greater levels of economic development, although adversely affecting achievement in less-developed countries with lower levels of economic development (Hanushek, Link, and Woessmann 2011). Following a 1988 reform in the United Kingdom that gave public secondary schools the option to opt out of local school district control and become autonomous schools funded directly by the central government, Clark (2009) finds that these newly autonomous secondary schools realized large achievement gains in the pass rates of eleventh-grade students.

In this paper, I explore a recent policy change in the Chicago Public Schools (CPS) that gave principals greater autonomy among a select group of schools. I first describe the nature of the autonomy granted to CPS elementary school principals and explore the choices school leaders made that reflect their preferences about areas for autonomy. My findings suggest that school leaders were more likely to exercise greater autonomy over their budget and curricular and instructional strategies than over professional development and the school's calendar and schedule. In addition, higher-achieving autonomous schools were less likely to use their newly provided autonomy, selecting, on average, one less autonomy option (of the ten options made available to them) than did lower-achieving autonomous schools. I then explore the impact of greater school-based autonomy on math and reading performance among CPS elementary schools. Using regression discontinuity methods, I find that, for schools at the discontinuity margin, there is no statistically significant evidence that the receipt of autonomy differentially impacted math or reading achievement after two years of autonomy. However, when attention is focused on the extensive margin of school performance—the share of students meeting state-determined proficiency standards in Illinois—evidence suggests that autonomous schools increased reading proficiency rates after two years of autonomy.

In the 2005–06 school year, CPS initiated the Autonomous Management and Performance Schools (AMPS) program. This program granted certain academic, programmatic, and operational freedoms to select schools (Cohort 1). CPS granted AMPS schools increased decision-making authority in four areas: (1) budget; (2) curriculum, instruction, and assessment; (3) calendar and schedule; and (4) professional development. These four domains, which Hansen and Roza (2005) describe as the key areas in which schools should be provided greater autonomy, reflect the definition of autonomous schools under RTTT. Indeed, some of the key policy makers overseeing the RTTT competition—in particular, Secretary of Education Duncan—were instrumental in creating the AMPS initiative in Chicago. Of the 576 regular CPS schools (486 elementary and 90 high schools), 73 elementary and 10 high schools were granted Cohort 1 AMPS status.⁴ For the purposes of this paper, I focus exclusively on the elementary schools.

CPS elementary schools were granted Cohort 1 AMPS status primarily based on the schools’ prior academic achievement.⁵ According to Dr. Barbara Eason-Watkins, who was the CPS Chief Education Officer, the district sorted schools based on their academic performance from the 2003–04 school year (school growth over time was not considered), with additional consideration given to the soundness of the school's fiscal operations.⁶ In very few cases, consideration was given to schools for which the district had confidence in the principal's leadership.⁷ The measure of academic performance used by CPS to select schools was a composite of the 2004 Illinois Standards Achievement Test (ISAT). A school's 2004 ISAT composite results were based on the share of a school's students who were proficient on the math, reading, and science portions of the 2004 ISAT.⁸ Though CPS did not use an explicit threshold to determine AMPS status, CPS policy makers indicated through informal discussions with the author that they followed a heuristic approach—ISAT composite proficiency rates between 55 and 60 percent—when selecting schools. My empirical identification of the achievement threshold described later in detail confirms an achievement threshold of 55 percent of students proficient on the 2004 ISAT composite measure.

Schools were notified of their AMPS status in the summer before the 2005–06 school year (approximately June 2005). According to AMPS area officers, schools and school leaders were not notified prior to receipt of AMPS status that the opportunity to acquire greater school-based autonomy would become available to them in the 2005–06 school year.⁹ Cohort 1 AMPS schools were then given two years of autonomy before their AMPS status was to be re-evaluated by the school district.

The Chief Executive Officer of CPS, Arne Duncan, encouraged the district to play a minimal role in managing the newly autonomous schools. Duncan encouraged CPS to give these schools the “freedom to innovate” and emphasized that the “best thing (the district) can do is get out of their way” (Elmore, Grossman, and King 2006, p. 9). In this context, should policy makers expect that greater autonomy will improve school performance? Or, should district leaders’ offer of greater autonomy be viewed as a gamble that school leaders take upon acceptance of increased decision-making authority? Indeed, Melissa Megliola-Zaikos, the CPS district official principally responsible for supporting the AMPS schools, recognized the inherent gamble schools were taking to participate in this initiative. She noted that the district was “asking schools to take new risks with us. We did not want anyone to fear being penalized if performance went down as a result” (Elmore, Grossman, and King 2006, p. 10).

Before the provision of greater autonomy in Chicago, schools had very clear expectations of district requirements concerning the organization and management of school operations. For example, the district set and maintained the school calendar, with specified days for staff professional development. The type of professional development and new teacher induction was mandated by the school district. If a principal wished to fund additional staff development or an innovative after-school program from the existing school budget, he or she was required to seek district approval to reallocate financial resources. The principal's actions were largely circumscribed by district rules and oversight. Under the AMPS initiative, however, participating schools and principals were given the opportunity to choose not only which (of ten) autonomies to take advantage of but also how to use them. This new decision-making authority represented a deviation from principals’ and school leaders’ prior experience with the management and operation of their schools. Indeed, principals now had the option to make substantive changes to school operations and management. They could alter the curriculum, reallocate resources, change instructional practices, and choose professional development models for new teachers.

Data for this paper are from CPS student-level administrative files and publicly available CPS school files.¹⁰ The student-level data include a rich set of demographic characteristics, including a student's race, gender, grade, and birth date (which I use to create a continuous measure of a student's age). Information is also available about whether a student receives free or reduced-price lunch, has an individualized education plan (IEP), or is limited English proficient (LEP).¹¹ I link students to their schools both within and across years using student and school identifiers.

Student-level test scores in math and reading are available for the 2005–06 and 2006–07 school years. The reading and mathematics scale scores from the ISAT are available for elementary school students in grades 3–8. The ISAT is the exam that Illinois uses to measure a school's progress toward meeting adequate yearly progress benchmarks for purposes of school performance evaluation under the No Child Left Behind (NCLB) Act.

The provision of school autonomy in Chicago is a school-level intervention. As a result, it is important to consider school composition in the empirical analysis. The most important school-level variable is school achievement in the 2003–04 academic year—the primary criterion upon which schools were granted AMPS status in 2005–06. A school's prior achievement is measured by the share of students meeting or exceeding Illinois state standards on the 2004 ISAT composite measure. Additional school-level characteristics include total school enrollment, gender composition (percentage of male and female students), racial composition (percentage of African American, Hispanic, white, Native American, Asian, and multi-race students), the proportion of students identified as LEP, the proportion of students with an IEP, and the proportion of students receiving free or reduced-price school lunch.

To identify which schools were granted Cohort 1 AMPS status in the 2005–06 school year, the AMPS office provided data on both the extensive and intensive margins of school autonomy in Chicago. The extensive margin characterizes whether a CPS elementary school was granted AMPS status in the 2005–06 school year; this is an indicator variable that takes on the value of one for schools offered AMPS status and zero otherwise. The intensive margin captures the extent to which AMPS schools took up autonomy, summarized by the number of autonomies (out of ten offered) that schools chose. Table 1 summarizes the school-level characteristics of the sample.

Of the 486 non-charter regular CPS elementary schools in 2005–06, information for 450 schools is available. In most cases, schools excluded from the sample did not have test score information for the 2004 ISAT composite measure, the prior achievement variable used to grant schools AMPS status. As a result, all analyses are done for the sample of 450 schools, which include all 73 AMPS and 377 non-AMPS elementary schools.

Across the four areas in which AMPS principals were granted greater decision-making authority—budget; curriculum, instruction, and assessment; calendar and schedule; and professional development—principals had the option to choose any of ten specific autonomies. Table 2 describes these autonomies and summarizes their distribution across the 73 elementary schools offered Cohort 1 AMPS status.

On average, Cohort 1 AMPS schools chose approximately six autonomies.¹² Moreover, 63 percent of AMPS schools (46 of 73) chose at least six autonomies, indicating that most schools offered greater autonomy through this district initiative actively engaged their new decision-making authority. Of the ten options, AMPS schools most frequently (82 percent) chose the Budget Transfers option, granting schools the ability to transfer money across line items without having their budgets reviewed by or requiring approval from the local area office. Ouchi (2004) describes budgetary control as “crucial to decentralization” (p. 22), and the fact that this option was chosen most often demonstrates that school leaders granted AMPS status also viewed budgetary control as the most critical component in the provision of autonomy over school operations.

To further explore the mechanisms through which school autonomy unfolded in Chicago, I explore the distribution of autonomy choices made by AMPS schools. Specifically, I collapse the ten autonomy choices into one of the four decision-making areas in which schools were granted greater decision-making authority.¹³Table 3 shows the distribution of school choices across each of these four decision areas, as well as how principals jointly exercised their autonomy choice across the four decision areas.

The vast majority of AMPS schools chose the Budgetary Control and the Curriculum, Instruction, and Assessment areas. In particular, 84 percent of all AMPS schools chose greater autonomy over budget-related decisions, and 88 percent of all AMPS schools chose to have greater autonomy and decision-making authority around the type of curriculum utilized in the classroom and the manner in which instruction occurred. These two decision areas independently represented a much larger share of the autonomies chosen by schools. Just over half of all AMPS schools chose the option to alter the school and day calendar (55 percent), whereas approximately two-thirds (69 percent) of all AMPS schools chose to exercise greater autonomy over professional development. The joint distributions provide insight into the mix of strategies that schools exercised in their decision to undertake school-based autonomy. For example, schools that chose budgetary authority were far more likely to also choose greater autonomy over curriculum, instruction, and assessment. In particular, 95 percent of schools that selected among the Budgetary Control area also selected among the Curriculum, Instruction, and Assessment decision area.¹⁴ The conditional probability of selecting from the Calendar and Schedule and Professional Development areas was much lower. Specifically, conditional on a school selecting among the Budgetary Control area, 61 and 74 percent of schools chose the Calendar and Schedule and Professional Development areas, respectively. Therefore, evidence suggests that most autonomous schools in Chicago chose to exercise their autonomy in areas that allowed them more flexibility in budgetary concerns while simultaneously enabling them to have more options over how instruction took place during the school day.

Although a principal's individual preferences for areas of autonomy likely influenced the options they chose, principals also faced potential constraints when determining the extent of autonomy take-up. In an effort to promote organizational change and innovation in schools, the district gave schools two years of autonomy before reevaluating their autonomous status. However, principals might have limited the extent of their school's participation in autonomy as a response to the extent of teacher buy-in necessary to initiate and successfully implement organizational change. Moreover, school-level autonomy was also accompanied by a reduction in district supports, some of which principals might have viewed as essential for limiting the risk of performance declines following organizational change. For example, schools that chose to implement their own attendance plans lost access to the district-level resources that work with schools to reduce truancy.¹⁵ Additionally, structural factors at the school level, including the characteristics of students the school served, may have also influenced the extent of autonomy take-up. In particular, did lower-achieving (among newly autonomous) schools have more to gain from greater autonomy? Did principals in schools with larger student enrollments face greater barriers to implementing organizational change? Table 4 summarizes a series of regressions that explore the influence of such structural factors on autonomy take-up across AMPS schools.

Evidence suggests that school size, based on student enrollment, had no influence on the number of autonomies principals selected, or on whether a principal selected any given autonomy. However, higher-achieving schools, on average, selected one less autonomy (of the ten options available) than did lower achieving schools, suggesting that, indeed, lower-achieving schools may have had more to gain from organizational change. In particular, lower-achieving schools were more likely to make changes to the school calendar, attendance plan, school improvement plan, and budget transfers.

To explore whether CPS elementary schools benefited from greater autonomy and realized achievement gains, I use a regression discontinuity approach. As previously discussed, Cohort 1 AMPS status was based primarily on a school's prior year achievement. Though no explicit achievement threshold was used to determine AMPS status, the reliance on prior school achievement as the primary selection criterion created a discontinuity in the likelihood of receiving AMPS status. I estimate this point of discontinuity (what I call the “achievement threshold”) in the probability of receipt of AMPS status. I then use this achievement threshold as an exogenous source of variation to estimate the impact of school autonomy in an instrumental variables framework. The fundamental assumption underlying the regression discontinuity approach is that the achievement threshold provides exogenous variation in the treatment (e.g., AMPS selection) and that unobserved characteristics of schools vary continuously with a school's achievement in the 2004 school year, which is the observable characteristic that determines AMPS status (Imbens and Lemieux 2008). Moreover, receiving AMPS status should be the sole reason for any discontinuity in outcomes. I proceed by describing the strategy for estimating the achievement threshold and then turn to formalizing and estimating a model of the impact of AMPS status on school achievement. I address the assumptions underlying this empirical approach in the context of the analysis.

In equation 1, AMPS_s represents actual receipt of school-based autonomy for school s beginning in the 2005–06 school year; Achievement_s,2004 is the percentage of students in school s meeting or exceeding Illinois state standards on the 2004 ISAT composite measure; X_s is a vector of school-level characteristics for the 2005–06 school year, and includes total school enrollment, gender composition (percentage of male and female students), racial composition (percentage of African American, Hispanic, white, Native American, Asian, and multi-race students), the proportion of students identified as LEP, the proportion of students with an IEP and the proportion of students receiving free or reduced-price school lunch; and μ_s is a random error. The variable Threshold_s is the achievement threshold to be estimated from the data, and a dummy variable whereby: Threshold_s = I[Achievement_s,2004 > η], where η is the threshold value. The function g(.) is a smooth function modeled with higher-order polynomials.

I estimate variants of equation 3 that include linear and quadratic polynomials for g(.), both with and without school-level controls in the range {40, 75} for Achievement_s,2004. For all variants of equations 1–3, the R² reaches a maximum at 55 (e.g., 55 percent of students meeting or exceeding Illinois state standards on the 2004 ISAT composite measure). This point is henceforth considered the achievement threshold and used as an instrumental variable in the subsequent regression discontinuity analysis.¹⁶Figure 1 shows the discontinuity in the probability of receiving AMPS status as a function of the 2004 ISAT composite measure at the estimated 55 percent achievement threshold.

Fundamental to this approach is the assumption that an achievement threshold actually exists. In Card, Mas, and Rothstein (2008), it is not clear, a priori, that a discontinuity exists, raising the concern that they might locate a discontinuity point even if one is not present. Like Bertrand, Hanna, and Mullainathan (2010), however, who explore affirmative action policies in India, the authors know that a threshold exists—there is an entrance exam score below which no additional students can gain university admission because all enrollment seats are filled. In my case, there is a composite school performance score below which no school was eligible for AMPS status. In particular, any elementary school scoring below 40 percent on the 2004 ISAT composite measure was ineligible to receive greater autonomy through the AMPS initiative.¹⁷ As can be seen from figure 1, only four (of 73) AMPS schools had 2004 ISAT composite performance below 55 percent.¹⁸

Another concern is the use of average school performance to determine participation in the AMPS initiative as well as to estimate the impact of autonomy on subsequent school achievement outcomes. In particular, test scores have been shown to be a noisy measure of school performance due to transitory error shocks, providing a misleading indicator of a school's true year-to-year performance (Kane and Staiger 2002). In such cases, Chay, McEwan, and Urquiola (2005) point out that mean reversion due to idiosyncratic testing noise may lead to misleading estimates of policy effects (for example, in the context of difference-in-differences strategies). Chay, McEwan, and Urquiola (2005) show that regression discontinuity designs can help overcome the potential biases induced by mean reversion when test-based ratings are used to assign program participation. I turn now to formalizing the regression discontinuity strategy used to estimate the impact of autonomy on school outcomes.

In equation 4, Y_ist is an outcome for student i in school s at the end of school year t and Autonomy_is,2006 is a variable indicating whether student i attended a school in 2005–06 that received AMPS status. The variables Z and X are student- and school-level covariates, respectively, and υ_ist is a random error term.

The key estimation issue concerning equation 4 is that the provision of school-based autonomy (e.g., receipt of AMPS status) is likely correlated with unobserved characteristics of the school. Specifically, even after conditioning on student and school characteristics, the conditional mean zero assumption—E(υ_ist|Z_ist, X_st, Autonomy_is,2006)—is likely violated. For example, unobserved characteristics of schools, such as school culture and other idiosyncrasies, are likely positively correlated with the provision of school autonomy.¹⁹ Therefore, to the extent that unobserved school heterogeneity is positively correlated with student outcomes, the endogeneity of school autonomy status will induce an upward bias into the estimate of the impact of AMPS status and the receipt of school-based autonomy on a range of student (and school) outcomes.

For equation 5, AMPS_s,2006 represents actual receipt of school-based autonomy for the 2005–06 school year for school s; Threshold_s is an indicator variable that equals one for schools at or above the achievement threshold (55 percent), and zero otherwise; Below_s is an indicator variable that equals one for schools below the achievement threshold, and zero otherwise; ISATdist_s is the distance between the school's share of students meeting or exceeding state standards on the 2004 ISAT composite measure and the achievement threshold; X is a vector of school-level characteristics for the 2005–06 school year, including total school enrollment, gender composition (percentage of male and female students), racial composition (percentage of African American, Hispanic, white, Native American, Asian, and multi-race students), the proportion of students identified as LEP, the proportion of students with an IEP, and the proportion of students receiving free or reduced-price school lunch; and is a random error. The estimated magnitude of the discontinuity in the probability of receiving autonomy (⁠⁠), from equation 1, is approximately 30 percentage points, and highly statistically significant.²⁰

I use the estimated value of AMPS_s,2006 from equation 5 to estimate the impact of autonomy on school performance. The parameter β in equation 6 estimates the difference in school outcomes at the achievement threshold. Given that I have access to student-level test score data, I use these data in the following way. I first regress the student-level math and reading test score outcomes separately on a vector of student characteristics—gender, race, lunch status, bilingual status, special education status, grade, and age—to partial out the idiosyncratic effect of student-level heterogeneity on student performance. I next recover the residuals from these regressions and aggregate the residuals to the school level. These aggregated residuals become the school-level mean performance measure (Y_st) for school s in year t.²¹ Because CPS did not reevaluate a school's Cohort 1 AMPS status until after two full school years with autonomy, I estimate both one- and two-year effects of autonomy for the t school years 2005–06 and 2006–07, respectively.

The range of the running variable over which to estimate the local linear regressions is of particular import. The choice of bandwidth involves considering the bias–precision tradeoff. A larger bandwidth will incorporate more school observations, generating more precise estimates. The cost of a wider bandwidth is the potential bias of the treatment effect, however, as the linear specification is less likely to be accurate further away from the achievement threshold. Lee and Lemieux (2010) suggest two approaches for selecting an optimal bandwidth that balances this bias–precision tradeoff. The first is a rule of thumb selection formula, and the second, following Imbens and Kalyanaraman (2009), is a data-dependent method for choosing the optimal bandwidth.²² The optimal bandwidth generated by these two methods is approximately 7 percentage points. Therefore, for estimation purposes, the preferred range of the running variable is between 48 and 62 percentage points on the 2004 ISAT composite measure.

As discussed, the first assumption underlying the “fuzzy” regression discontinuity approach used in this paper is that the achievement threshold provides exogenous variation in the treatment status (e.g., AMPS selection). The first stage regression results indicate that, at the achievement threshold, the probability of receiving AMPS status increases by approximately 30 percentage points, and is highly statistically significant.

The second assumption concerns schools’ ability to manipulate the forcing variable—2004 school achievement—to become eligible for AMPS status. According to Lee and Lemieux (2010), “this is probably the most important question to ask when assessing whether a particular application should be analyzed as an RD [regression discontinuity] design” (p. 292). More specifically, this assumption generates the local random assignment around the achievement threshold. Although this assumption cannot be tested directly, Lee and Lemieux (2010) recommend assessing the continuity of the distribution of the assignment variable. Figure 2 shows the frequency distribution of the assignment variable. Because schools and principals were unaware before summer 2005 that the opportunity for greater school-based autonomy would become available, it is implausible that schools would have had the ability to manipulate their 2004 test score results to create such an opportunity for their schools. Figure 2 confirms this, showing no evidence of a discontinuity at the achievement threshold.²³

The third assumption underlying the empirical approach is that unobserved characteristics of schools vary continuously with a school's achievement in the 2004 school year—the observable characteristic that determines AMPS status (Imbens and Lemieux 2008; Lee and Lemieux 2010). As Lee and Lemieux (2010) note: “As in a randomized experiment, the distribution of observed baseline covariates should not change discontinuously at the threshold…. This is akin to the tests performed to empirically assess whether the randomization was carried out properly in randomized experiments” (pp. 292, 296). Though I am unable to empirically verify this assumption for unobserved characteristics of schools, I do explore whether the main school covariates vary continuously around the achievement threshold. Figure 3 confirms that the distribution of the share of male students, the share of students by race and ethnicity, the share of students who receive free or reduced-price lunch, and the share of students classified as LEP varies continuously at the 55 percent achievement threshold point. There is some evidence that the share of students who have an IEP and the size of the school (enrollment) vary discontinuously at the threshold. This indicates, for example, that schools with larger enrollments may have been given some priority, at the achievement threshold, for AMPS status. This does not invalidate the fuzzy regression discontinuity design, however. As long as schools cannot precisely manipulate their position vis-à-vis the assignment variable, the treatment near the threshold remains as if randomized (Lee and Lemieux 2010). Like experimental settings where there is some covariate imbalance across treatment and control groups, conditioning on the variables that are discontinuous at the threshold achieves balance on these covariates.

Of primary interest is whether greater autonomy improves school performance. I explore two margins of school performance. The intensive margin considers average school performance based on standardized values of student-level ISAT math and reading scale scores. The extensive margin considers a school's proficiency rate—the share of students who meet or exceed Illinois state proficiency standards in math and reading—as the outcome measure. Column 1 of table 5 offers preliminary evidence on whether autonomy improves school performance by reporting ordinary least squares (OLS) estimates of the relationship between AMPS status and school math and reading achievement after the first (2005–06) and second (2006–07) years of autonomy. On average, AMPS elementary schools perform better than non-AMPS schools. After the first year of autonomy, AMPS schools performed 0.23 standard deviations better than non-AMPS schools in math and 0.22 standard deviations better in reading. Moreover, there is a 22 and 21 percentage point difference in the proficiency rate in math and reading, respectively, between AMPS and non-AMPS schools. These performance differences persist after the first year of autonomy.

As is evidenced in table 1, however, AMPS schools are quite different from non-AMPS schools. Indeed, AMPS schools teach fewer students from economically disadvantaged circumstances—60.7 percent of AMPS students receive free or reduced-price lunch, compared with 82.7 percent in non-AMPS schools—and serve a smaller share of minority students—68.0 percent of students in AMPS schools are African American or Hispanic, compared with 89.7 percent of students in non-AMPS schools. Once observable differences in school characteristics are accounted for (see column 2, table 5), autonomy no longer significantly predicts a school's average math and reading achievement, while achievement differences in math and reading proficiency rates persist (except when focused on reading proficiency rates for the full sample of schools in year 1).

Based on the evidence from the OLS regressions, conditional on school characteristics, greater autonomy does not impact average school achievement (the intensive margin of school performance), but continues to have an effect on reading and math proficiency rates (the extensive margin of school performance). For the OLS regression estimates to provide a valid causal effect of autonomy, however, no differences, observable or unobservable, must remain between AMPS and non-AMPS schools once we condition on school characteristics, such as the share of students receiving free lunch and the share of minority students. But there are a number of reasons why low-performing schools (those in the bottom quartile of the performance distribution) are different relative to high-performing elementary schools. Factors such as the extent of neighborhood resources, parental involvement, and teacher quality likely differ systematically across low- and high-achieving schools. Therefore, aside from indicating that AMPS schools perform better than non-AMPS schools, the OLS estimates provide little guidance on how the provision of autonomy directly impacts school performance. To gain traction on the causal effect of autonomy, I turn to the instrumental variable estimates in the context of the regression discontinuity design.

Figure 4 presents visual evidence of the magnitude of the discontinuity in math and reading outcomes at the achievement threshold. The figure illustrates the distribution of the outcomes—math and reading results after the first (2005–06) and second (2006–07) years of autonomy—with the running variable, the 2004 ISAT composite measure, on the horizontal axis. Across the full sample of elementary schools, panels A–D of figure 4 suggest that there was no difference in the average math or reading performance on the intensive margin, at the 55 percent achievement discontinuity threshold. Though there appears to be no difference in proficiency rates after the first year of autonomy (panels E and F of figure 4), panels G and H suggest that math and reading proficiency rates in year 2 increased at the achievement threshold, with a more pronounced shift in reading.

To further explore the magnitude of the discontinuity, columns 4–5 of table 5 provide instrumental variable estimates of the impact of school-based autonomy in Chicago on school math and reading performance using the preferred bandwidth specification. The year 1 and year 2 results confirm the visual evidence presented in figure 4; school-based autonomy did not have a statistically significant effect on reading or math on the intensive margin of school performance. Although the impact on math achievement is –0.15 standard deviations after the first year, there is no conclusive evidence of a statistically significant impact. After the second year of autonomy, the impact estimates for math and reading are small and not statistically different from zero.

Therefore, based on a school's average math and reading performance, the empirical evidence suggests AMPS schools did not realize relative performance improvements in the two years of autonomy. However, for the purposes of school sanctions under NCLB legislation, schools are evaluated on the share of their students proficient in math and reading. It is plausible, then, that schools used their greater autonomy to develop strategies to target students at the proficiency benchmark, focusing on improving the school's proficiency rate by allocating effort to students at the margin of proficiency while reducing effort given to very high- and very low-achieving students. In fact, evidence from Chicago suggests that after the introduction of NCLB, math and reading test scores improved for students in the middle of the achievement distribution, near the proficiency benchmark, but not among the lowest-performing students; indeed, for students in the bottom 20 percent of the performance distribution, there is evidence of lower-than-expected performance in math (Neal and Schanzenbach 2010).

Looking at the extensive margin of school performance, the regression discontinuity estimates indicate a large positive effect of autonomy on reading proficiency rates. I am unable to reject the null of no effect after year 1, but there is evidence of a statistically significant impact in reading proficiency, on the order of 18 percentage points, after year 2. There is no statistically significant evidence, however, of any impact on math proficiency rates in either year. Though there were no significant changes in average math or reading after two years of autonomy, this evidence suggests that simply looking at school proficiency rates as the only measure of performance masks real differences in average school performance, differences that more adversely impact the lowest- and highest-achieving students than students in the middle of the achievement distribution. Although student reading performance, on average, did not benefit from attendance at more autonomous schools, it appears that students at the proficiency benchmark likely did benefit.

The specific mechanisms through which students in the middle of the achievement distribution benefitted can't be determined from this analysis. But it is clear that autonomous schools may have used their autonomy for more targeted instruction, or perhaps even allocated resources to enhance the potential that students at the margin of proficiency would meet achievement standards. Indeed, AMPS schools chose approximately two more autonomies at the achievement threshold.²⁴ Moreover, each of the ten autonomy choices exhibits a statistically significant increase in the probability that it was used by schools; that is, each of the ten choices is significantly and positively related to the achievement threshold. Four autonomy choices—After School Counts, School Improvement Plan, Attendance Plan, and Budget Transfers—stand out. The probability of a school choosing each of these four autonomy options increases by approximately 25 percentage points at the achievement threshold. This increase is 6–10 percentage points greater than the increase for the other six autonomy options. I cannot attribute the effect of autonomy to these options, but this evidence provides more insight into the choices schools made at the achievement threshold. Indeed, each of these options is part of either the Curriculum, Instruction, and Assessment or Budgetary Control decision areas, which influence the type of instruction in the classroom and the allocation of resources in the school.

Though the empirical strategy limits my ability to look out past two years, the positive effect on reading proficiency rates after two years suggests that as principals and school leaders are given time to learn how to use their resources and teachers become accustomed to new organizational changes, school performance can improve over time. As a result, the practice of implementing autonomy strategies appears to require both learning on behalf of school leaders and teachers and time to update how they use their new autonomy. Finally, an important caveat pertains to the generalizability of these results to the full distribution of CPS elementary schools. In particular, the impact estimates represent a localized effect (e.g., local average treatment effect, or LATE) and capture the impact of autonomy in Chicago for schools at the 55 percent achievement threshold. As a consequence, this LATE estimate does not generalize to schools across the achievement distribution and therefore does not provide for an estimate of the causal effect of autonomy in Chicago for schools with much higher prior achievement (for example, 75 percent of students meeting proficiency on the 2004 ISAT composite measure).

To explore the robustness of the findings to bandwidth choice, columns 6–8 of table 5 and figure 5 present regression discontinuity estimates from alternative bandwidths. The findings on the impact of school autonomy are robust across a range of bandwidth choice. Indeed, there is no evidence of a statistically significant effect of autonomy on average reading or math achievement after two years of autonomy. The only exception is for math achievement in years 1 and 2 when attention is paid to the full sample of schools. Though the impact of autonomy on average achievement appears negative, the inclusion of schools farthest away from the achievement threshold likely biases the impact estimates, generating misleading conclusions about the impact of autonomy. The impact of autonomy on reading proficiency rates in year 2 is robust to alternative specifications (at the 10 percent level of significance), while there remains no evidence of a statistically significant autonomy effect on math proficiency in years 1 and 2 or on reading proficiency in year 1.

Over the past decade, some of the nation's largest school districts have given greater autonomy to local public (non-charter) school principals. The trend toward greater school-based autonomy as a district-level plan to improve educational performance has found support at the national level. Indeed, some of the chief architects of Chicago's AMPS initiative, including Secretary of Education Arne Duncan, modeled federal policy after Chicago's experience with school autonomy. Specifically, the 2010 Race to the Top competition awarded states for designing innovative school reform strategies that let local school districts provide greater school-based autonomy. Up to this point, however, little evidence existed on how this emerging district-level reform strategy has unfolded. This paper offers initial evidence on how school leaders utilized greater decision-making authority in Chicago—with attention to the mechanisms through which this autonomy operated—and whether autonomy substantively impacted school achievement.

I have described how schools and school leaders responded to the offer of autonomy. The majority of AMPS schools (63 percent) chose at least six of the ten autonomy options; budgetary control and greater autonomy over curriculum, instruction, and assessment were the most desired autonomy areas. I find that elementary schools in Chicago that received AMPS status did not experience statistically significant changes in average math or reading performance relative to non-AMPS schools. There do appear to be statistically significant differences in reading proficiency rates after two years of autonomy, however, suggesting that schools require time to efficiently implement autonomy, and may be targeting resources to marginal students. As a result, policy makers should not expect immediate performance gains from the provision of autonomy but rather, if such policies are pursued, should allow time for schools and school leaders to learn how best to implement their new autonomy. Policy makers should be aware of the potential adverse impact on the lowest- and highest-achieving students along the performance distribution if resources are focused on improving proficiency rates, rather than average school performance.

Compare these findings to a similar study of decentralization of decision-making authority in the Boston Public Schools (Abdulkadiroglu et al. 2011), where the authors explored the impact of pilot schools on student achievement. The pilot schools determined their own budgets, staffing, curricula and scheduling, and their teachers were Boston Teachers Union members, very similar to the AMPS initiative in CPS. Lottery-based estimates of the effect of pilot schools for students in grades 3–8 were mixed; the authors found positive effects on elementary (grades 3–4) reading but not math, and no effect on reading among middle school students (grades 6–8) but a negative effect on math performance. In CPS, the effect of autonomy on average math achievement after one year is imprecisely estimated, although the magnitude of the negative effect is very similar to the estimated effect of attending a pilot school among sixth to eighth graders in Boston. Moreover, though I cannot reject zero impacts of autonomy on reading achievement, I do find large and marginally statistically significant effects on school proficiency rates in reading. Clark (2009) also assesses the impact of greater autonomy on pass rates, similar to my consideration of proficiency rates. He looks at public high schools in the U.K. between 1988 and 1997, and finds large, immediate, and persistent effects on pass rates—the fraction of eleventh-grade students who passed five or more General Certificate of Secondary Education exams, standardized exams taken by all eleventh graders in the U.K. When I explore proficiency rates among a sample of students in grades 3–8, I similarly find large effects on reading proficiency, but only after two years of autonomy.

Ultimately, the findings from Chicago are short-term in nature. If schools do require more time to incorporate their new autonomy into changes in school organization, then in the long term autonomy might increase not only proficiency rates but also benefit all students along the distribution of student achievement. This, of course, is an empirical question, and presents a robust area for future research. Moreover, an assessment of school performance should extend beyond how students perform on standardized test scores and incorporate the full range of educational and social experiences that may result from greater school-level autonomy. For example, does greater autonomy improve student safety and establish a school climate that is more conducive to learning? Does autonomy offer teachers more control over their instructional practice inside the classroom and professional collaboration opportunities with other teachers outside the classroom? Understanding how greater autonomy affects these (and other) areas would provide a more nuanced understanding of the impact that decentralized decision-making has on school organization and performance.

The author thanks Kerwin Charles, Stephen Raudenbush, and Ofer Malamud for helpful discussions and suggestions on earlier drafts of this article. The author benefitted from the suggestions of two anonymous referees and comments from seminar and conference participants at the University of Chicago, the School of Public Affairs at CUNY-Baruch College, the Association for Public Policy Analysis and Management, and the American Educational Research Association. The author thanks Jonathan Wallace for editorial assistance and the Chicago Public Schools district and the Consortium on Chicago School Research for providing access to the data for this article. The author acknowledges funding from the Institute of Education Sciences.