Abstract

Most teacher layoffs during the Great Recession were implemented following inverse-seniority policies. In this paper, I examine the implementation of a discretionary layoff policy in Charlotte Mecklenburg Schools. Administrators did not uniformly lay off the most or least senior teachers but instead selected teachers who were previously retired, late-hired, unlicensed, low-performing, or nontenured. Using quasi-experimental variation within schools across grades, I then estimate the differential effects of teacher layoffs on student achievement based on teacher seniority and effectiveness. Mathematics achievement in grades that lost an effective teacher, as measured by principal evaluations or value-added scores, decreased 0.05 to 0.11 standard deviations more than in grades that lost an ineffective teacher. In contrast, teacher seniority has limited predictive power on the effects of layoffs. Simulation analyses show that the district selected teachers who were, on average, less effective than those teachers identified under an inverse-seniority policy, and also reduced job losses.

Layoffs, as painful as they are, should fall on the least-effective teachers when layoffs are absolutely unavoidable.

—Arne Duncan, U.S. Secretary of Education, 22 March 2011

1.  Introduction

Personnel reductions are a common cost-cutting measure that firms use in response to changes in market demand and macroeconomic downturns. In the public education sector, where personnel costs represent between 60 and 80 percent of total expenditures, school districts can rarely avoid layoffs when faced with a major budget shortfall (Roza 2007). Beginning in 2008, public school districts across the country were forced to implement widespread teacher layoffs in response to large decreases in local and state tax revenues brought on by the Great Recession. The scale at which these layoffs occurred and the ways in which they were implemented have caused educators, policy makers, and researchers to reexamine these practices.

For the majority of districts, collectively bargained agreements, state laws, and legal precedent mandate that these layoffs are based on seniority. The implementation of these long-standing last-hired, first-fired layoff policies has generated considerable criticism among policy organizations and in the popular press because such policies eliminate the jobs of early-career teachers who may be more effective than some of their more experienced peers.1 An analysis of seniority-based layoffs in Washington State confirmed that teachers who received reduction in force (RIF) notices were no less effective, on average, than those whose jobs were not threatened (Goldhaber and Theobald 2013). In recent years, several state legislatures and district superintendents have attempted to amend or eliminate last-in first-out policies, and advocacy groups have challenged their constitutionality under the equal protection clause in a widely publicized lawsuit (Vergara v. California).

The results of several simulation analyses suggest layoff policies that prioritize teacher effectiveness, as measured by valued-added scores, could result in the selection of less-effective teachers than those who would lose their jobs under inverse-seniority policies (Boyd et al. 2011; Goldhaber and Theobald 2013). Measures of effectiveness derived from student achievement, however, are controversial, only available for a minority of teachers, and, with high stakes attached, can lead to gaming behaviors such as teaching to the test (Jacob 2005) or even outright cheating (Jacob and Levitt 2003). Furthermore, proposals for layoffs based strictly on objective measures of effective teaching focus narrowly on teachers’ ability to raise achievement on standardized tests, while ignoring other educational priorities, organizational needs, and labor market constraints.

Subjective performance evaluations, such as ratings by principals, provide an alternative measure of effectiveness that consider multiple criteria, are widely available, and are capable of distinguishing among the very best and worst teachers (Jacob and Lefgren 2008). Administrators are often reluctant to give employees poor evaluations, however, given existing job protections and the costs negative evaluations can impose on workplace productivity, morale, and trust (Prendergast and Topel 1993; MacLeod 2003; Weisberg et al. 2009). Subjective ratings are also susceptible to rater biases (Prendergast 1999; Goldin and Rouse 2000) and contextual influences that can lower the reliability of scores (Hill, Charalambous, and Kraft 2012). Thus, open questions remain about the efficacy of performance-based layoffs that use either value-added measures or principal evaluations given the potential implementation challenges, unintended consequences, and moderate reliability of these measures.

In this study, I analyze the implementation and consequences of discretionary layoffs in the eighteenth largest public school district in the nation to provide some of the first empirical evidence on performance-based layoffs in education. In total, the Charlotte Mecklenburg Schools (CMS) in North Carolina eliminated almost 2,000 employees, including over 1,000 teaching positions, in the two years following the onset of the Great Recession. CMS identified candidates for layoffs using a set of five general criteria: duplicative positions, enrollment trends, job performance, job qualifications, and length of service. My administrative data set contains both principal evaluation scores, which directly informed the layoff selection process, as well as the necessary data to estimate value-added scores, which were not used by the district. These data also allow me to compare subjective and objective measures of teacher effectiveness by incorporating both measures throughout my analyses.2

Studying the implementation of layoffs in CMS provides a unique opportunity to shed light on several central questions surrounding discretionary layoff policies. Would administrators simply defer to seniority despite the flexibility afforded by such policies? Would they instead target the highest paid (i.e., most senior) teachers for layoffs in an effort to maximize costs savings? Or would they use their discretion to lay off teachers based on multiple factors? I explore these questions by comparing the predictive power of a variety of RIF criteria. I then estimate the grade-specific effects of teacher layoffs on student achievement in CMS with particular attention to the differential effects of layoffs based on teacher seniority and effectiveness. This is the first analysis, that I am aware of, which links the characteristics of teachers being laid off to changes in student achievement. I identify credible estimates by exploiting quasi-experimental variation where some rising cohorts of students in a school entered grades in which a teacher was laid off, while others did not. The key identifying assumption in my analysis requires that the grades in which layoffs occurred in a school were unrelated to the past achievement and characteristics of rising class cohorts in a school. Empirical tests lend strong support to this assumption. Finally, I simulate alternative layoff selection policies and compare the differences in average cost saving and teacher characteristics across layoff pools.

I find that layoffs in CMS were concentrated among probationary teachers with fewer than four years of seniority. Principal evaluations were also strong predictors of the probability of being laid off in CMS, as well as licensure status and licensure type. Layoffs were particularly concentrated among high school teachers as well as foreign language and arts teachers. I find suggestive but inconsistent evidence that, on average, layoffs had negative grade-specific effects on student achievement. These average estimates, however, mask wide variability in the impact of laying off individual teachers. Mathematics achievement in grades that lost an effective teacher (at the 75th percentile), as measured by subjective or objective metrics, decreased by between 0.05 and 0.11 standard deviations more than in grades that lost an ineffective teacher (at the 25th percentile). In contrast, I find that the marginal difference between laying off a senior versus early-career teacher is substantially smaller and statistically insignificant. Simulation analyses provide further evidence that seniority-based layoffs increase job losses but also demonstrate how inverse-layoff policies based on a single performance measure are suboptimal compared with a policy that considers both objective and subjective measures.

Together, these findings have important implications for teacher evaluation systems and layoff policies. They offer new evidence of the predictive validity of both principal evaluations and value-added scores. They provide evidence on the importance of prioritizing performance over seniority when districts are forced to implement teacher layoffs. They also illustrate the value of allowing principals and districts discretion when conducting layoffs rather than constraining these difficult decisions with inflexible policies based on any single measure.

2.  Teacher Layoffs in U.S. Public Schools

The Great Recession and Teacher Layoffs

Reductions in force in U.S. public schools broadly mirror the larger macroeconomic cycles of the U.S. economy. Just over three decades ago, teachers faced similar rounds of layoffs on the heels of the energy crisis and increasing stagflation. Districts are often able to forestall layoffs during less-severe economic downturns through a combination of hiring freezes, natural attrition, and incentives for early retirement. Sharp decreases in tax revenues during the Great Recession, however, left many states unable to maintain previous levels of funding for public education. The federal government responded in 2009 with the American Recovery and Reinvestment Act, which allocated over $53.6 billion for state education expenditures, and again in 2010 with the Education Jobs and Medicaid Assistance Act, which provided $10 billion for local school districts. Despite this federal aid, districts were unable to avoid reductions in force.

Estimates of the total number of teacher layoffs in this period range between the tens and hundreds of thousands annually (Martinez 2010). Ellerson (2010) reported that 37 percent of districts cut core classroom teaching positions in 2009–10, and over 60 percent planned cuts in 2010–11. The threat of potential job loss was real for many more teachers. Budget uncertainties and contract requirements to provide early notice of the possibility of termination led some districts to distribute “pink slips” to most teachers they employed.3

Seniority-Based Layoff Policies

Layoff policies in most U.S. public school systems are governed by state legislation or collective bargaining agreements between districts and local teachers’ unions. Beginning in the 1970s, inverse-seniority layoff clauses became widely incorporated into teacher union contracts and adopted by over a dozen state legislatures in an effort to combat discriminatory and nepotistic employment practices. Today, the vast majority of districts are beholden to collective bargaining agreements or laws that continue to prioritize seniority over any other layoff selection criteria.4 Of the one hundred public school districts in the National Council on Teacher Quality's Teacher Contract Database (TR3) in 2010, seventy-five followed seniority-based layoff policies (NCTQ 2010).

Pure seniority-based policies maximize the number of jobs that need to be cut to reach a given budget reduction by requiring the least-experienced, and thus lowest-paid, teachers to be the first ones laid off (Roza 2009). These policies also erode districts’ recent efforts to recruit, select, and train highly qualified teachers and may undermine future initiatives to attract talented novice teachers. Furthermore, such policies can cause schools that are primarily staffed with inexperienced teachers to lose large portions of their faculty, some of whom must be replaced with veteran teachers who are forcibly transferred from other schools (Sepe and Roza 2010; Medina 2011).

3.  A Discretionary Layoff Policy

Charlotte Mecklenburg Schools is the largest school district in North Carolina, and it serves over 137,000 pre-K to twelfth-grade students and employs over 9,000 teachers. Policies governing teacher contracts in CMS are determined by the state legislature, as North Carolina is one of five states where collective bargaining is explicitly illegal.5 Local education agencies such as CMS are allowed to determine RIF policies unilaterally, but only in accordance with federal and state fair labor practices, policies, and statutes. CMS School Board policy requires that superintendents make a recommendation to the school board regarding the need for a reduction in force and the criteria to be used in the reduction. Since 2003, CMS School Board policy has stipulated five broad criteria for selecting teachers as part of involuntary reductions: (1) structural considerations that allow for the elimination of duplicative or excess personnel and positions; (2) organizational considerations that permit the superintendent to account for future school enrollment projections; (3) job performance considerations broadly defined as employees’ recent performance on the job; (4) job qualifications, such as tenure status,6 education, licensure type, and licensure status, as well as more abstract qualities such as leadership abilities and future potential; and (5) length of continuous, full-time service with the district (i.e., seniority).7

In March 2009, then-Superintendent Gorman presented the CMS School Board with recommendations for employee reductions, including at least 456 classroom teaching positions, in order to make up an $87 million budgetary shortfall. The superintendent's proposal outlined three key steps that would be used to implement the layoffs: (1) the district would allocate layoffs across schools based on projected enrollment trends, (2) principals would identify position categories (such as grade levels or subjects) that would be reduced, and (3) district officials would select which teacher(s) among those in the identified position categories would be laid off (see Appendix  A for details). The CMS School Board ultimately approved the superintendent's proposal in 2009, as well as a similar proposal the following year for the elimination of approximately 600 classroom teachers. Notably, student performance was not included as a criterion to be used in the RIF process in either of the two years.

Although the board-approved RIF procedures specified a clear sequence of steps, the school board also granted the superintendent discretionary authority for “limited exceptions” to these procedures. In practice, principals commonly identified specific teachers rather than position categories to be considered for layoffs within their school. Given the discretionary nature of the RIF process, the significant implementation challenges inherent in executing the proposed procedures, and the flexibility principals had to determine which teachers could be considered for layoffs, it remained unclear which teachers would ultimately lose their jobs.

4.  Data and Measures

I use an administrative data set that links students, teachers, and test records across a 13-year panel of data from 1997–98 to 2009–10. Student data include demographic information and annual state test results in reading and mathematics. Data on teachers from human resource files include demographic information, tenure status, licensure type and status, title descriptions, as well as scores on a performance assessment rubric evaluated by principals. I combine this panel data set with RIF data files provided by CMS for the 2008–09 and 2009–10 school years that identify 654 laid-off teachers in the summer of 2009 and 433 in 2010.8 I am able to match 1,043 of these teachers (96 percent of all RIFed teachers) to my full panel of administrative data using unique identifiers. I complement these quantitative data with relevant state statutes, CMS School Board documents, newspaper articles, and interviews with CMS administrators familiar with the layoff process.

I construct a variety of measures for the teacher characteristics specified in the Board-approved RIF criteria. I create a measure of seniority using employment records from human resource files. For those teachers new to the district since 1997–98, I define seniority as the number of years in the data set a teacher was employed by CMS. For teachers who were hired prior to the 1997–98 school year, I define seniority as the minimum of either the difference between the current academic year and teachers’ district hire date or their level of salary experience credit.

Detailed data on teacher licensure status allow me to construct indicators for probationary (nontenured) teachers, retired teachers who have since been rehired (commonly referred to as “double dippers” because they collect a salary and pension simultaneously), and teachers with a licensure deficiency (teachers without a license or with a conditional or temporary license). I also create a set of indictors for licensure type that includes mathematics, English language arts, science, social studies, foreign language, arts, physical education, English as a second language, special education, elementary, and early education licenses. Finally, I create an indicator for late-hired teachers who were hired after the start of the academic year using detailed hire-date records.

Principal Evaluations of Teacher Performance

I construct performance evaluation scores using principals’ ratings of teachers on a statewide evaluation protocol. Since 2001, North Carolina has required that public school teachers be evaluated using the Teacher Performance Appraisal Instrument–Revised (TPAI-R). Principals rate teachers on eight overall domains based on evidence collected through classroom observation, teaching artifacts, and discussions. The domains covered by the instrument include management of instructional time, management of student behavior, instructional presentation, instructional monitoring, instructional feedback, facilitating instruction, communicating within the education environment, and performing non-instructional duties. Domain scores are assigned using a four-point scale ranging from “Unsatisfactory” to “Above Standard.”

All probationary teachers and teachers with below-standard performance are evaluated on an annual basis, and tenured teachers are evaluated at least once every five years or at the recommendation of an administrator. In order to maximize the number of teachers with an evaluation score in my analytic sample, I construct a rolling average of all available evaluation scores for each teacher in each year. I first calculate year-specific evaluation scores by assigning values of 1 (“Unsatisfactory”) through 4 (“Above Standard”) to the four rating categories and averaging scores across the eight domains of the TPAI-R rubric. I then standardize scores in each year to be mean zero with unit variance such that a one unit difference can be interpreted as moving one standard deviation higher in the distribution of teacher effectiveness. Finally, I average all available scores in my panel of data up to and including a given year. I use these rolling average scores in all analyses unless otherwise noted.

Value-Added Measures of Teacher Contributions to Student Achievement

I estimate teacher effects by attempting to isolate a teacher's value added to her students’ academic achievement on standardized tests for mathematics and English teachers in fourth through eighth grades—the grades and subjects in which the necessary baseline and outcome testing data are available. A large body of literature has examined the consequences of different value-added modeling approaches which attempt to account for the nonrandom sorting of students to teachers (Todd and Wolpin 2003; McCaffrey et al. 2004; Harris and Sass 2006; Kane and Staiger 2008; Koedel and Betts 2011; Chetty, Friedman, and Rockoff 2014; Guarino, Reckase, and Wooldridge 2015). I adopt the widely used covariate-adjustment model from the education production function literature as my preferred specification. My model controls flexibly for prior student achievement as well as a variety of student, classroom, and school characteristics (see Appendix  B for full description).

I isolate the permanent teacher effect separately from idiosyncratic class-year shocks by estimating effects across multiple years following the empirical Bayes approach described by Kane and Staiger (2008). Using this approach, I estimate a rolling average value-added score for each teacher in each year in mathematics and in reading by using all available data up to and including a given year. I use these rolling value-added estimates unless otherwise noted. I present value-added scores in test-score standard deviation units in all of my descriptive statistics, but restandardize these measures in a teacher-year-level data set for use in my regression analyses. This allows for a more meaningful comparison of the coefficients associated with value-added scores and principal evaluation scores.

5.  Empirical Framework

Describing the Layoff Selection Process

I construct a teacher-year data set for all K–12 classroom teachers employed by CMS in the 2008–09 and 2009–10 academic years. My final analytic sample includes 17,409 teacher-year records.9 I then fit a series of logistic regressions that model the conditional probability of being laid off, RIF, as a function of district RIF criteria for teacher j in year t:10
formula
1

Here, the parameters of interest are the coefficients in vector, , that capture the log odds that teachers with a given RIF criteria were selected for layoffs. I present these coefficients in subsequent tables as average marginal effects calculated across the analytic sample in order to facilitate interpretation.

Estimating the Differential Effect of Layoffs on Student Achievement

I estimate the differential effect of reductions in force in 2008–09 on student achievement in 2009–10 across three measures central to the debate on layoff policy: seniority, principal evaluations scores, and value-added scores. I use a six-year student-level panel data set (2004–05 to 2009–10) to more accurately account for potential confounding trends in achievement over time between schools and within schools across grades.11 I restrict these data to include only students who can be linked to their mathematics or English teacher in fourth through eighth grades, the grades for which both current and prior standardized test scores are available.12 Research frequently finds that teachers have smaller effects on students’ achievement on standardized reading tests (Hanushek and Rivkin 2010) as is the case in the present analysis. Thus, I focus on student achievement in mathematics and present parallel results for reading in Appendix table C.1.

In order to isolate plausibly exogenous variation in layoffs, I focus my analyses on the differential impact of layoffs across grades within schools. My identification strategy approximates a natural experiment by comparing, within a school, the performance of students who entered a grade in which a teacher was laid off to the performance of students who entered a grade that did not experience a layoff. Layoffs were not chosen at random in CMS. Analyses in the following sections show that a teacher's performance and a variety of other characteristics are associated with being selected for layoffs. Within a school, however, the selection of a teacher in a given grade for layoffs was arguably unrelated to the achievement of the rising cohort of students in the grade below. I test for potential violations of this assumption by comparing the prior academic achievement of students who entered grades in which a teacher was laid off to the prior achievement of students who entered grades that did not lose a teacher due to layoffs. Results presented herein demonstrate that there is no evidence that principals selected teachers for layoffs based on the achievement of rising cohorts of students.

This approach provides a credible and policy-relevant estimate of the average effect of layoffs in CMS in affected grades within schools. More specifically, this estimate captures the net effect of multiple features that contribute to a layoff “treatment” including: (1) the loss of the effect of the laid-off teacher on the rising cohort of students; (2) the absence of a laid-off teacher's positive (or negative) peer effects on their grade-level colleagues; and (3) the efforts of principals to mitigate the grade-specific effects of layoffs by reassigning personnel, reallocating resources, adjusting class sizes, or hiring new teachers. My within-school grade-specific identification strategy, however, does not capture any schoolwide or district-wide effects of layoffs. For example, the loss of a teacher could affect the morale of the entire staff; a laid-off teacher might have influenced colleagues and students across her building; or principals might take steps to buffer their entire school from the effect of layoffs. These schoolwide impacts will not be reflected in my estimates. Further, the loss of key instructional support staff and student support personnel across both years likely had negative consequences for instructional quality and student achievement across the district.13 I adopt this narrow approach, which focuses on credibly exogenous variation, because comparing the relative effects of layoffs across teachers with different levels of seniority and performance is of first-order importance, while establishing the full effect of layoffs is secondary.

I implement this approach by constructing a dichotomous indicator for whether any classroom teachers were laid off in each grade of a school in the previous year, LAYOFF.14 I create corresponding measures of the average seniority, principal evaluation scores, and value-added scores of laid-off teachers in the previous school-year-grade. I calculate average seniority and principal evaluation scores of laid-off teachers using all data through the year in which they were laid off, 2008–09. I calculate subject-specific average value-added scores using all data through the year prior to the layoffs, 2007–08, to guard against the potential for correlated errors among individual students’ test scores over time that could bias my estimates.15 Furthermore, the current year standardized achievement results needed to calculate value-added scores are rarely, if ever, available to districts before they must notify teachers of layoffs. I include the average value-added scores in the same subject as the achievement outcome. I represent these three measures generically as C.16

Building on similar modeling approaches by Jepsen and Rivkin (2009) and Jackson and Bruegmann (2009), I fit a series of models where students’ test scores are a function of LAYOFF, a given average characteristic of the laid-off teachers (C), control variables, and select sets of fixed effects as follows:
formula
2

Here, the outcome of interest, , is the standardized scaled score on a state end-of-grade test in a given subject for student i, in grade g, with teacher j, in school s, in year t. I include grade-specific cubic functions of students’ prior-year achievement, , in both mathematics and reading, as well as vectors of controls for observable student characteristics , the characteristics of a student's peers with the same teacher (), and grade-by-year fixed effects ().17 The inclusion of school-by-year fixed effects () accounts for any schoolwide year-specific shocks to student achievement, such as the turnover of a principal or the introduction of a new curriculum, by restricting my comparison to students within the same school in the same year. The inclusion of school-by-grade fixed effects () removes any permanent differences in average teacher effectiveness across grades within a school that could bias my estimates. Such differences might arise if, for example, less effective teachers were systematically assigned to teach lower grades. I estimate standard errors clustered at the teacher-level to account for the potential of correlated errors among students taught by the same teacher.

, the coefficient associated with a given C, captures the estimated differential effect of laying off a teacher with one year more seniority (or one standard deviation higher evaluation or value-added score) in year t–1 on the academic achievement of students in year t in the grade and school in which the laid-off teacher taught.18 Interpreting the coefficients associated with these interaction terms as causal estimates imposes two important additional assumptions. First, principals’ ability to mitigate the effect of layoffs must be uncorrelated with the seniority and effectiveness of laid-off teachers. For example, if principals in schools where early-career teachers were laid off were more effective at buffering students from the consequences of layoffs compared to principals in schools where more senior teachers were laid off, my estimates would exaggerate any negative differential effect of laying off more senior teachers. Second, because my estimation strategy does not capture schoolwide effects of layoffs, these estimates may understate or overstate the total marginal effect of layoffs based on measures of seniority or effectiveness. If seniority or measures of teacher effectiveness are differentially predictive of the schoolwide effects of layoffs, it is possible that the relative magnitudes of the grade-specific marginal effects I estimate would be different from the total marginal effects.

In section 7, I test the robustness of my primary specifications, examine their identifying assumptions, and extend my analyses in several ways. I account for the potential threat of student sorting by adding student fixed effects. I also examine the effects of subject-specific layoffs and explore two potential mechanisms, increases in class-size and grade-specific turnover.

6.  The Implementation and Differential Effects of Layoffs in CMS

Descriptive Characteristics of Laid-Off Teachers

In table 1, I present averages of teacher characteristics across RIFed teachers and non-RIFed teachers in 2009 and 2010 for my full analytic sample, for a sample that excludes returning retired teachers, and for tenured teachers. These results show that the teachers selected for layoffs were likely to be nontenured teachers, returning retired teachers, teachers hired after the start of the school year, teachers with a licensure deficiency, and low-performing teachers. Over 84 percent of laid-off teachers in 2009 and 2010 were probationary teachers despite the flexibility afforded to the district by North Carolina's ban on collective bargaining. In follow-up interviews, administrators explained that the additional requirements of laying off a tenured teacher caused them to focus first on probationary teachers for whom they could simply not renew their contracts. They saw terminating the job of tenured teachers who met performance requirements as counterproductive because state law guarantees laid-off tenured teachers first rights to accept any open position for which they qualify, up to three years after being laid off.

Table 1. 
Teacher Characteristics across RIFed and Non-RIFed Teachers in 2009 and 2010
20092010
RIFedNon-RIFedDifferenceNRIFedNon-RIFedDifferenceN
 Panel A: All Teachers 
Experience 13.07 10.21 2.86*** 8,918 5.60 11.03 −5.43*** 8,491 
Seniority 8.04 6.14 1.90*** 8,918 1.87 6.98 −5.11*** 8,491 
Probationary teacher 0.842 0.421 0.421*** 8,918 0.844 0.329 0.515*** 8,491 
Returning retired 0.225 0.000 0.225*** 8,918 0.003 0.001 0.002 8,491 
teacher 
Late hire 0.267 0.011 0.256*** 8,918 0.419 0.009 0.410*** 8,491 
Licensure deficiency 0.162 0.038 0.124*** 8,918 0.169 0.022 0.147*** 8,491 
Evaluation score: −0.507 0.058 −0.565*** 4,729 −1.211 0.055 −1.266*** 8,038 
1-year 
Evaluation score 0.394 −0.016 −0.378*** 7,837 −0.902 0.026 −0.928*** 8,374 
Math value-added −0.004 0.004 −0.008 1,847 −0.062 0.007 −0.069*** 1,869 
score 
Reading value-added −0.005 0.004 −0.009 1,270 −0.014 0.001 −0.015+ 1,897 
score 
 Panel B: Excluding Returning Retired Teachers 
Evaluation score: −0.864 0.058 −0.922*** 4,600 −1.219 0.055 −1.274*** 8,031 
1-year 
Evaluation score −0.697 −0.016 −.681*** 7,703 −0.909 0.025 −0.934*** 8,362 
Math value-added −0.025 0.004 −0.029* 1,821 −0.061 0.007 −0.068*** 1,864 
score 
Reading value-added −0.013 0.004 −0.017* 1,261 −0.014 0.001 −0.015+ 1,894 
score 
 Panel C: Tenured Teachers 
Evaluation score: −1.415 0.317 −1.732*** 877 −1.440 0.152 −1.592*** 5,270 
1-year 
Evaluation score −0.867 0.115 −0.982*** 3,911 −0.875 0.136 −1.011*** 5,463 
Math value-added −0.051 0.008 −0.059* 1,127 −0.120 0.008 −0.128** 1,326 
score 
Reading value-added −0.014 0.008 −0.022 661 −0.046 0.002 −0.048* 1,333 
score 
20092010
RIFedNon-RIFedDifferenceNRIFedNon-RIFedDifferenceN
 Panel A: All Teachers 
Experience 13.07 10.21 2.86*** 8,918 5.60 11.03 −5.43*** 8,491 
Seniority 8.04 6.14 1.90*** 8,918 1.87 6.98 −5.11*** 8,491 
Probationary teacher 0.842 0.421 0.421*** 8,918 0.844 0.329 0.515*** 8,491 
Returning retired 0.225 0.000 0.225*** 8,918 0.003 0.001 0.002 8,491 
teacher 
Late hire 0.267 0.011 0.256*** 8,918 0.419 0.009 0.410*** 8,491 
Licensure deficiency 0.162 0.038 0.124*** 8,918 0.169 0.022 0.147*** 8,491 
Evaluation score: −0.507 0.058 −0.565*** 4,729 −1.211 0.055 −1.266*** 8,038 
1-year 
Evaluation score 0.394 −0.016 −0.378*** 7,837 −0.902 0.026 −0.928*** 8,374 
Math value-added −0.004 0.004 −0.008 1,847 −0.062 0.007 −0.069*** 1,869 
score 
Reading value-added −0.005 0.004 −0.009 1,270 −0.014 0.001 −0.015+ 1,897 
score 
 Panel B: Excluding Returning Retired Teachers 
Evaluation score: −0.864 0.058 −0.922*** 4,600 −1.219 0.055 −1.274*** 8,031 
1-year 
Evaluation score −0.697 −0.016 −.681*** 7,703 −0.909 0.025 −0.934*** 8,362 
Math value-added −0.025 0.004 −0.029* 1,821 −0.061 0.007 −0.068*** 1,864 
score 
Reading value-added −0.013 0.004 −0.017* 1,261 −0.014 0.001 −0.015+ 1,894 
score 
 Panel C: Tenured Teachers 
Evaluation score: −1.415 0.317 −1.732*** 877 −1.440 0.152 −1.592*** 5,270 
1-year 
Evaluation score −0.867 0.115 −0.982*** 3,911 −0.875 0.136 −1.011*** 5,463 
Math value-added −0.051 0.008 −0.059* 1,127 −0.120 0.008 −0.128** 1,326 
score 
Reading value-added −0.014 0.008 −0.022 661 −0.046 0.002 −0.048* 1,333 
score 

Notes:*p < 0.05; **p < 0.01; ***p < 0.001; +p < 0.10. p-values are derived from regressions of the given characteristic on an indicator for layoff status. Principal evaluation scores are standardized in a teacher-year-level data set. Value-added scores are expressed in student-test-score standard deviations. These analyses include 619 RIFed teachers in 2009 and 384 RIFed teachers in 2010 for whom data on all descriptive measures other than performance are available.

Figure 1 illustrates how layoffs were heavily concentrated among nontenured teachers and, in 2009, teachers with thirty or more years of seniority. The increase in the probability of being laid off among teachers with thirty or more years of experience is driven by the nonrenewal of returning retired teachers who constituted over 21 percent of all RIFed teachers in 2009.19 These teachers who were collecting both a salary and a pension were among the very first the district targeted for layoffs (see Appendix  A), despite the fact that they were substantially more effective than the average CMS teacher. Their average evaluation scores were two thirds of a standard deviation higher than the district average and their average value-added scores were 0.048 standard deviations (SD) and 0.034 SD higher in mathematics and reading, respectively.

Figure 1.

Distribution of CMS Teachers by Seniority for RIFed and Non-RIFed Teachers in 2009 and 2010.

Figure 1.

Distribution of CMS Teachers by Seniority for RIFed and Non-RIFed Teachers in 2009 and 2010.

Overall, RIFed teachers in 2009 were rated 0.38 SD lower by principals (approximately one fifth of a point on a 4-point scale). These same teachers had slightly lower value-added scores in mathematics and reading compared with non-RIFed teachers. When returning retired teachers are excluded from these estimates (table 1, panel B), or the sample is restricted to tenured teachers (table 1, panel C), these differences become even greater. RIFed teachers who were not returning retired teachers had value-added scores that were, on average, 0.029 SD lower in mathematics and 0.017 SD lower in reading. In 2010, the differences in effectiveness between RIFed and non-RIFed teachers increased even further. RIFed teachers in 2010 received principal evaluation scores that were 0.93 SD lower than non-RIFed teachers. I also find that RIFed teachers in 2010 had significantly lower value-added scores, on average, than non-RIFed teachers (−0.068 SD lower in mathematics and −0.015 SD lower in reading).

Examining the distribution of unstandardized one-year principal evaluation scores for non-RIFed, 2009 RIFed, and 2010 RIFed teachers provides further insights into the layoff process. Figure 2 suggests that evaluation scores were used to directly inform the layoff process. Only 4 percent of teachers with an average rating of “At Standard” or above (an average of 3 on the 4-point scale) were laid off, whereas 58 percent of all teachers whose average evaluation scores were below “At Standard” were laid off.20 In contrast, the continuous distributions of value-added scores in figure 3 illustrate that although RIFed teachers, particularly 2010 RIFed teachers, had lower value-added scores on average, the layoff selection process did not operate directly through these scores. The differences in the average value-added scores of RIFed and non-RIFed teachers are not surprising given the weak-to-modest positive correlations between principal evaluation and value-added scores in my data. Similar to previous studies, I find that principal evaluation scores are correlated 0.27 and 0.19 with math and reading value-added scores, respectively (Jacob and Lefgren 2008; Harris and Sass 2009).21

Figure 2.

Distributions of Average 1-Year Raw Evaluation Scores for RIFed and Non-RIFed Teachers in 2009 and 2010.

Figure 2.

Distributions of Average 1-Year Raw Evaluation Scores for RIFed and Non-RIFed Teachers in 2009 and 2010.

Figure 3.

Distributions of Mathematics and Reading Value-Added Scores for Non-RIFed Teachers and RIFed Teachers in 2009 and 2010.Note: Value-added scores are rolling averages expressed in student-test-score standard deviations.

Figure 3.

Distributions of Mathematics and Reading Value-Added Scores for Non-RIFed Teachers and RIFed Teachers in 2009 and 2010.Note: Value-added scores are rolling averages expressed in student-test-score standard deviations.

The Distribution of Layoffs across Schools

Overall, teacher layoffs were widely distributed among the 165 K–12 schools in CMS. Across both years, 64 percent of schools laid off between 1 percent and 10 percent of their teaching staff, and another 16 percent of schools laid off more than 10 percent but less than 20 percent of their classroom teachers. Only nine small elementary schools and one small high school avoided layoffs for classroom teachers in both years. The highest concentration of staff layoffs was 29 percent. In table 2, I present the probability that a teacher was laid off across school levels and by quartiles of school characteristics. In 2009, 9.6 percent of high school teachers were laid off, and 7.4 percent lost their jobs in 2010. High school teachers were approximately twice as likely to be laid off as elementary school teachers, and 40 percent more likely to be laid off than middle school teachers.

Table 2. 
The Probability a Teacher Was Laid Off by School Type and Characteristics in 2009 and 2010
20092010
Panel A: School Type
Pre-KElemMiddleHighPre-KElemMiddleHigh
 0.079 0.056 0.068 0.096 0.028 0.028 0.053 0.074 
 Panel B: Quartiles of School Characteristics 
 Bottom 2nd 3rd Top Bottom 2nd 3rd Top 
State Performance Index 0.078 0.062 0.064 0.073 0.060 0.047 0.049 0.033 
Average days absent 0.064 0.060 0.062 0.090 0.027 0.031 0.045 0.075 
% African American students 0.074 0.051 0.063 0.089 0.030 0.035 0.046 0.070 
% Hispanic students 0.077 0.069 0.067 0.064 0.033 0.052 0.057 0.039 
% Limited English proficient 0.070 0.075 0.067 0.065 0.040 0.040 0.060 0.041 
students 
% Special education students 0.088 0.059 0.061 0.074 0.041 0.038 0.049 0.055 
Average mathematics 0.084 0.058 0.042 0.057 0.052 0.044 0.026 0.016 
achievement 
Average reading achievement 0.089 0.048 0.051 0.053 0.054 0.040 0.025 0.020 
20092010
Panel A: School Type
Pre-KElemMiddleHighPre-KElemMiddleHigh
 0.079 0.056 0.068 0.096 0.028 0.028 0.053 0.074 
 Panel B: Quartiles of School Characteristics 
 Bottom 2nd 3rd Top Bottom 2nd 3rd Top 
State Performance Index 0.078 0.062 0.064 0.073 0.060 0.047 0.049 0.033 
Average days absent 0.064 0.060 0.062 0.090 0.027 0.031 0.045 0.075 
% African American students 0.074 0.051 0.063 0.089 0.030 0.035 0.046 0.070 
% Hispanic students 0.077 0.069 0.067 0.064 0.033 0.052 0.057 0.039 
% Limited English proficient 0.070 0.075 0.067 0.065 0.040 0.040 0.060 0.041 
students 
% Special education students 0.088 0.059 0.061 0.074 0.041 0.038 0.049 0.055 
Average mathematics 0.084 0.058 0.042 0.057 0.052 0.044 0.026 0.016 
achievement 
Average reading achievement 0.089 0.048 0.051 0.053 0.054 0.040 0.025 0.020 

Notes: The State Performance Index rates schools based on three measures: percent of students at proficient or above on state exams, average student growth on state exams, and Adequate Yearly Progress (AYP) status following federal guidelines. These analyses include 619 RIFed teachers in 2009 and 384 RIFed teachers in 2010 for whom data on all descriptive measures other than performance are available.

Teachers working in schools that served larger proportions of students who were African American, who scored lower on achievement tests, and who attended school less frequently were more likely to lose their jobs. Teachers in schools in the top quartile of African American student enrollment were approximately 3 percentage points more likely to be laid off compared with teachers at schools in the bottom quartile. Teachers in schools where students’ scores were in the top quartile of average achievement in both mathematics and reading were also approximately 3 percentage points less likely to be laid off compared with teachers in schools in the bottom quartile. There were no consistent differences in the distribution of layoffs across schools by their state performance rating or by the proportion of students who were Hispanic, limited English proficient, or receiving special education services.22

What Criteria Did CMS Prioritize When Selecting Teachers for Layoffs?

I fit a series of logistic regression models to better understand the relative importance CMS administrators and principals placed on different school board–approved RIF criteria. In table 3, I begin with a specification that includes indicators for each year a teacher has probationary status within CMS (column 1). Notably, being a novice teacher was associated with a 22 percentage point higher probability of being laid off as compared with tenured teachers, but drops precipitously to between 2.2 and 6.3 percentage points for teachers with between one and three years of seniority. When I include other school board–approved RIF criteria in column 2, the relative probability a novice teacher was laid off is reduced by 13.5 percentage points, even without accounting for principal evaluation scores. Comparing these conditional average marginal probabilities that a CMS teacher was laid off given her seniority to estimates from Washington State suggests that districts in North Carolina had more flexibility to lay off teachers based on criteria other than seniority. For Washington State teachers, the marginal effect of having between zero and three years of seniority ranges from 9.2 to 11.6 percentage points (Goldhaber and Theobald 2013; table 4, p. 512), whereas corresponding marginal effect estimates for CMS in column 2 of table 3 are half as large. Other criteria used to identify teachers for layoffs, such as if a teacher was a returning retired teacher or was hired after the start of the school year, are much stronger predictors than seniority. These indicators are associated with 89 and 40 percentage point increases in the probability of layoffs, respectively.

Table 3. 
The Relationship between Measures of RIF Criteria and the Probability of Being Laid Off
Full SampleEvaluation Score Sample
(1)(2)(3)(4)(5)
0 years seniority 0.222*** 0.087*** 0.176*** 0.112*** 0.040*** 
 (0.013) (0.011) (0.013) (0.010) (0.008) 
1 year seniority 0.035*** 0.038*** 0.028*** 0.009 0.016** 
 (0.008) (0.007) (0.008) (0.006) (0.006) 
2 years seniority 0.063*** 0.065*** 0.057*** 0.040*** 0.046*** 
 (0.009) (0.008) (0.009) (0.007) (0.007) 
3 years seniority 0.022* 0.043*** 0.017* 0.010 0.032*** 
 (0.009) (0.009) (0.008) (0.007) (0.008) 
Retired teacher  0.890***   0.914*** 
  (0.018)   (0.012) 
Late hire  0.398***   0.327*** 
  (0.029)   (0.030) 
Licensure deficiency  0.065***   0.051*** 
  (0.010)   (0.009) 
Evaluation score    −0.039*** −0.040*** 
    (0.003) (0.003) 
Licensure type fixed effects    
Observations 17,409 17,409 16,211 16,211 16,211 
Full SampleEvaluation Score Sample
(1)(2)(3)(4)(5)
0 years seniority 0.222*** 0.087*** 0.176*** 0.112*** 0.040*** 
 (0.013) (0.011) (0.013) (0.010) (0.008) 
1 year seniority 0.035*** 0.038*** 0.028*** 0.009 0.016** 
 (0.008) (0.007) (0.008) (0.006) (0.006) 
2 years seniority 0.063*** 0.065*** 0.057*** 0.040*** 0.046*** 
 (0.009) (0.008) (0.009) (0.007) (0.007) 
3 years seniority 0.022* 0.043*** 0.017* 0.010 0.032*** 
 (0.009) (0.009) (0.008) (0.007) (0.008) 
Retired teacher  0.890***   0.914*** 
  (0.018)   (0.012) 
Late hire  0.398***   0.327*** 
  (0.029)   (0.030) 
Licensure deficiency  0.065***   0.051*** 
  (0.010)   (0.009) 
Evaluation score    −0.039*** −0.040*** 
    (0.003) (0.003) 
Licensure type fixed effects    
Observations 17,409 17,409 16,211 16,211 16,211 

Notes:*p < 0.05; **p < 0.01; ***p < 0.001. Cells represent average marginal effect estimates derived from logistic regression models. Robust standard errors reported in parentheses. Principal evaluation scores are standardized in a teacher-year-level data set. These analyses include 619 RIFed teachers in 2009 and 384 RIFed teachers in 2010 for whom data on all descriptive measures other than performance are available.

Table 4. 
The Relationship between Licensure Type and the Probability of Being Laid Off
Full SampleEvaluation Score Sample
(1)(2)(3)(4)(5)
Licensure Type      
Mathematics 0.035*** 0.016** 0.035*** 0.028*** 0.013* 
 (0.008) (0.006) (0.008) (0.007) (0.005) 
English language arts 0.014** 0.007+ 0.017** 0.022*** 0.011* 
 (0.005) (0.004) (0.006) (0.006) (0.005) 
Science 0.017* 0.005 0.016* 0.010 0.002 
 (0.007) (0.005) (0.008) (0.007) (0.005) 
Social studies 0.013* 0.008+ 0.012+ 0.012+ 0.009+ 
 (0.006) (0.005) (0.007) (0.006) (0.005) 
Foreign language 0.060*** 0.026** 0.061*** 0.059*** 0.026* 
 (0.015) (0.010) (0.015) (0.014) (0.010) 
Arts 0.024** 0.024** 0.028** 0.034*** 0.032*** 
 (0.009) (0.008) (0.010) (0.010) (0.009) 
Physical education 0.017+ 0.016+ 0.021* 0.030** 0.019+ 
 (0.010) (0.009) (0.011) (0.011) (0.010) 
English as a second language −0.002 0.001 −0.002 0.002 0.008 
 (0.009) (0.008) (0.009) (0.010) (0.010) 
Special education −0.005 0.003 −0.002 0.000 0.006 
 (0.005) (0.005) (0.006) (0.006) (0.006) 
Seniority (0–3 years)    
Retired teacher    
Late hire    
Licensure deficiency    
Evaluation score    
Observations 17,409 17,409 16,211 16,211 16,211 
Full SampleEvaluation Score Sample
(1)(2)(3)(4)(5)
Licensure Type      
Mathematics 0.035*** 0.016** 0.035*** 0.028*** 0.013* 
 (0.008) (0.006) (0.008) (0.007) (0.005) 
English language arts 0.014** 0.007+ 0.017** 0.022*** 0.011* 
 (0.005) (0.004) (0.006) (0.006) (0.005) 
Science 0.017* 0.005 0.016* 0.010 0.002 
 (0.007) (0.005) (0.008) (0.007) (0.005) 
Social studies 0.013* 0.008+ 0.012+ 0.012+ 0.009+ 
 (0.006) (0.005) (0.007) (0.006) (0.005) 
Foreign language 0.060*** 0.026** 0.061*** 0.059*** 0.026* 
 (0.015) (0.010) (0.015) (0.014) (0.010) 
Arts 0.024** 0.024** 0.028** 0.034*** 0.032*** 
 (0.009) (0.008) (0.010) (0.010) (0.009) 
Physical education 0.017+ 0.016+ 0.021* 0.030** 0.019+ 
 (0.010) (0.009) (0.011) (0.011) (0.010) 
English as a second language −0.002 0.001 −0.002 0.002 0.008 
 (0.009) (0.008) (0.009) (0.010) (0.010) 
Special education −0.005 0.003 −0.002 0.000 0.006 
 (0.005) (0.005) (0.006) (0.006) (0.006) 
Seniority (0–3 years)    
Retired teacher    
Late hire    
Licensure deficiency    
Evaluation score    
Observations 17,409 17,409 16,211 16,211 16,211 

Notes:*p < 0.05; **p < 0.01; ***p < 0.001; +p < 0.10. Cells represent average marginal effect estimates derived from logistic regression models. Robust standard errors reported in parentheses. The omitted licensure category in all models is elementary licensures. These analyses include 619 RIFed teachers in 2009 and 384 RIFed teachers in 2010 for whom data on all descriptive measures other than performance are available.

Results from these analyses are consistent with those from a sample of teachers for whom evaluation scores are available. When all RIF criteria are added to the model (column 5), I estimate that being evaluated as one standard deviation lower by a principal is associated with a 4 percentage point increase in the probability of being laid off. Nevertheless, this average marginal effect masks the differential relationship between teachers’ evaluation scores and the probability of layoffs. In figure 4, I plot the predicted probability of layoffs across the sample distribution of standardized rolling evaluation scores for teachers new to the district, teachers with three years of seniority (i.e., teachers currently in their fourth year of teaching who would receive tenure if they are rehired), and for tenured teachers. This figure illustrates two key findings. First, the probability of being laid off increases precipitously in all three plots as evaluation scores drop below two standard deviations, suggesting that the district targeted the very lowest performing teachers across all levels of seniority. Second, lower-performing tenured teachers were more likely to be laid off than higher-performing untenured teachers. For example, the probability that a tenured teacher with an evaluation score of −1 SD was laid off was 4.4 percentage points, which is greater than the corresponding probability for newly hired teachers with at least an average evaluation score (probability <3.5 percentage points).

Figure 4.

The Predicted Probability of Being Laid Off for New Hires (0 years seniority), Teachers with 3 Years of Seniority, and Tenured Teachers across the Range of Standardized Principal Evaluation Scores from the Model Described in column 5 of table 3.

Figure 4.

The Predicted Probability of Being Laid Off for New Hires (0 years seniority), Teachers with 3 Years of Seniority, and Tenured Teachers across the Range of Standardized Principal Evaluation Scores from the Model Described in column 5 of table 3.

I extend these analyses by refitting the full model in column 5 and substituting in each of the eight evaluation domains on the TPAI-R rubric. In Appendix table C.2, I find that Management of Instructional Behavior, Management of Instructional Time, and Instructional Presentation are the strongest predictors of the probability of being laid off. Further analyses also suggest that the higher rates of layoffs in schools with more African American and lower-achieving students are driven by teacher sorting across schools. Conditioning on the full set of RIF criteria, the relationships between these individual school characteristics and the probability of being laid off, as seen in table 2, reverse, but are of extremely small magnitude (less than a 0.7 percentage point difference per SD change in a given school characteristic).

These descriptive analyses also reveal that among all teachers, foreign language and arts teachers were the most likely to be laid off. In table 4, I present estimates of the relationship between licensure type and the probability of being laid off relative to teachers with elementary licensures. Column 1 shows how foreign language, mathematics, and arts teachers were all significantly more likely to be laid off. When I include controls for the full set of RIF criteria both with and without evaluation scores, foreign language and arts teachers have the highest probabilities of being laid off. The reduction in the relative probability of being laid off for mathematics teachers suggests that many of these teachers were laid off based on other RIF criteria. Conditional on all RIF criteria, including evaluation scores, foreign language and arts teachers had approximately a 3 percentage point greater probability of being laid off compared with teachers with elementary licensures.

How Teacher Seniority and Effectiveness Moderate the Effects of Layoffs

I examine the moderating effect of the characteristics of laid-off teachers in 2008–09 on mathematics achievement in the following year by isolating plausibly exogenous variation in layoffs within a school across grade levels. I formally test the core assumption of my modeling approach—that principals did not choose specific grades for layoffs based on the relative achievement of the rising cohorts of students in a school—by fitting a modified version of model 2. I regress prior test scores of the rising cohorts of students on LAYOFF and my full set of two-way fixed effects. In table 5, I present estimates of the coefficient associated with LAYOFF that are both near zero and statistically insignificant. These estimates provide little evidence that principals selected layoffs based on the achievement of rising cohorts of students.23 Parallel tests based on student demographic characteristics produce estimates equal to or less than 0.023, none of which is statistically significant.

Table 5. 
Tests of the Relationship between Prior Student Achievement and Teacher Layoffs in a Student's Grade in the Previous Year
Outcome
Prior Math Achievement (1)Prior Reading Achievement (2)
LAYOFF 0.002 −0.030 
 (0.074) (0.064) 
Observations 241,572 241,572 
Outcome
Prior Math Achievement (1)Prior Reading Achievement (2)
LAYOFF 0.002 −0.030 
 (0.074) (0.064) 
Observations 241,572 241,572 

Notes: Standard errors clustered by teacher are reported in parentheses. Regressions include grade-by-year, school-by-year, and school-by-grade fixed effects.

In table 6, I fit a series of models to estimate the direct effect of layoffs in CMS, and to compare the moderating effects of three teacher characteristics on the relationship between layoffs and student achievement in the following year.24 In my baseline model, I find a small negative and statistically insignificant relationship between layoffs and student achievement. I then expand the model to examine whether this average effect masks important variation in the relationship between layoffs and student achievement related to the characteristics of laid-off teachers. In columns 2 through 4, I present estimates of the differential effect of layoffs based on the average seniority, principal evaluation scores, and value-added scores of laid-off teachers in the previous school-grade-year as described earlier.

Table 6. 
The Moderating Effect of Teacher Characteristics on the Relationship between Teacher Layoffs in the Previous Year and Student Achievement in Mathematics
(1)(2)(3)(4)(5)(6)(7)
LAYOFF −0.014 −0.000 −0.038 0.030 −0.042 0.022 −0.014 
 (0.024) (0.026) (0.024) (0.047) (0.028) (0.049) (0.050) 
Seniority  −0.002   0.001 0.001 0.004 
  (0.002)   (0.002) (0.003) (0.003) 
Evaluation   −0.034+  −0.035+  −0.035+ 
   (0.017)  (0.020)  (0.020) 
Math Value-Added    −0.083*  −0.092* −0.094* 
    (0.035)  (0.040) (0.040) 
Observations 241,572 241,572 241,572 241,572 241,572 241,572 241,572 
(1)(2)(3)(4)(5)(6)(7)
LAYOFF −0.014 −0.000 −0.038 0.030 −0.042 0.022 −0.014 
 (0.024) (0.026) (0.024) (0.047) (0.028) (0.049) (0.050) 
Seniority  −0.002   0.001 0.001 0.004 
  (0.002)   (0.002) (0.003) (0.003) 
Evaluation   −0.034+  −0.035+  −0.035+ 
   (0.017)  (0.020)  (0.020) 
Math Value-Added    −0.083*  −0.092* −0.094* 
    (0.035)  (0.040) (0.040) 
Observations 241,572 241,572 241,572 241,572 241,572 241,572 241,572 

Notes:*p < 0.05; +p < 0.10. Standard errors clustered by teacher are reported in parentheses. All regressions include grade-specific cubic functions of students’ prior-year achievement in both mathematics and reading, grade-by-year, school-by-year, and school-by-grade fixed effects as well as student and peer characteristics. Student characteristics include gender, race, limited English proficiency status, and special education status. Peer characteristics include the means of all student characteristics as well as mean prior year achievement in mathematics and reading. Principal evaluation scores are averages of all scores through the year a teacher was laid off. Value-added scores are estimated using all data through the year prior to layoffs and are restandardized in a teacher-year-level data set. When average seniority or performance measures for laid-off teachers in a prior school-grade-year are missing or undefined, zero is imputed and separate indicator variables are included for each type of missingness.

I find that seniority has limited predictive power on the effect of teacher layoffs. The estimated coefficient associated with laying off a teacher with one more year of experience is near zero and not statistically significant (−0.002 SD, p = .242). Even if we ignored the lack of statistical significance and scaled the point estimate to compare laying off a teacher with seven years of seniority versus a novice teacher (a one standard deviation difference), the magnitude would only increase to −0.014 SD. Supplemental analyses where seniority is specified as a binary variable produce similar results. In contrast, laying off a teacher one standard deviation higher in the distribution of teacher effectiveness as judged by principals or measured by value-added scores lowered student achievement by 0.034 SD (p = .050) and 0.083 SD (p = .017), respectively. Thus, the differential effect on student achievement between laying off a teacher ranked at the 75th percentile of the distribution of evaluation scores and one at the 25th percentile is 0.046 SD. The corresponding difference for value-added scores is 0.112 SD. These estimates show that the effect of reductions in force depends primarily on the effectiveness of those teachers selected for layoffs.

Next, I fit models that allow for the effect of teacher layoffs on future student achievement to differ by both seniority and measures of teacher effectiveness simultaneously. Results presented in columns 5–7 illustrate that seniority contains little information about the future effect of laying off a teacher that is not captured by performance measures. When I include principal evaluation scores or value-added scores in the model, the relationship between the seniority of laid-off teachers and mathematics achievement is even further reduced and switches signs. In contrast, the coefficients associated with measures of principal evaluation scores and value-added score remain nearly identical. In my full model that includes all three teacher characteristics, I find that both principal evaluation and value-added scores maintain their predictive power, conditional on the other, suggesting these measures are capturing somewhat different aspects of teacher effectiveness. Estimates from models that use evaluation scores or value-added measures constructed from the most recent three years of data produce nearly identical results (see Appendix table C.3).

Finally, these results suggest that laying off a low-performing teacher raised student achievement the following year in the grade taught relative to other grades. Using estimates from my full model in column 7, I find that the linear combination of the main effect of LAYOFF and the marginal effect of laying off a teacher who received an evaluation score of −0.5 SD or lower is positive, holding all else constant. The same linear combination for value-added scores becomes positive when a teacher has a score of −0.15 teacher-level SD or lower. It is possible, however, that negative schoolwide effects of layoffs not captured by these estimates could partially offset or dominate these positive within-school, across-grade effects.

7.  Robustness Tests and Extensions

Student Sorting

One potential threat to identifying credible estimates is the possibility that students and their families responded strategically to layoffs by switching schools. Model 2 addresses this threat by including sets of controls for student and peer characteristics. Nevertheless, student sorting on unobserved characteristics related to layoffs in the previous year could bias my results. I examine this potential threat by testing the sensitivity of my estimates to the inclusion of student fixed effects in place of prior achievement scores and student characteristics. This approach further restricts my estimates to within-student differences in achievement across time, greatly limiting the endogenous student selection process as a threat. Thus, these estimates are identified using only a subsample of students with two or more observations. This effectively removes 13 percent of the student-year observations in the analytic sample and over 27 percent of the student observations in 2010.25

Estimates from models that include student fixed effects are consistent with the previous results although somewhat less precise given the limited remaining variation for identifying parameters. As shown in table 7, the most notable difference is the increase in the estimated average effect of layoffs on student achievement in math, which becomes −0.071 (p = .031). Further analyses suggest this difference is largely driven by the sample restriction imposed by including student fixed effects. An estimate from my preferred specification in model 2 using a restricted sample of students with at least two years of data is nearly as large (–0.047, p = .210), but is not statistically significant.26 This suggests that student sorting on unobserved characteristics may mask a negative main effect of layoffs that operates independently of any compositional changes in teacher quality.

Table 7. 
The Moderating Effect of Teacher Characteristics on the Relationship between Teacher Layoffs in the Previous Year and Student Achievement in Mathematics, Conditioning on Student Fixed Effects
Student Fixed Effects
(1)(2)(3)(4)(5)(6)(7)
LAYOFF −0.071* −0.063+ −0.097** −0.019 −0.091** −0.012 −0.034 
 (0.033) (0.033) (0.035) (0.069) (0.034) (0.070) (0.071) 
Seniority  −0.003   −0.002 −0.002 −0.001 
  (0.004)   (0.004) (0.005) (0.005) 
Evaluation   −0.037  −0.036  −0.036 
   (0.023)  (0.024)  (0.024) 
Math Value-Added    −0.113+  −0.107 −0.104 
    (0.064)  (0.067) (0.066) 
Observations 241,572 241,572 241,572 241,572 241,572 241,572 241,572 
Student Fixed Effects
(1)(2)(3)(4)(5)(6)(7)
LAYOFF −0.071* −0.063+ −0.097** −0.019 −0.091** −0.012 −0.034 
 (0.033) (0.033) (0.035) (0.069) (0.034) (0.070) (0.071) 
Seniority  −0.003   −0.002 −0.002 −0.001 
  (0.004)   (0.004) (0.005) (0.005) 
Evaluation   −0.037  −0.036  −0.036 
   (0.023)  (0.024)  (0.024) 
Math Value-Added    −0.113+  −0.107 −0.104 
    (0.064)  (0.067) (0.066) 
Observations 241,572 241,572 241,572 241,572 241,572 241,572 241,572 

Notes:*p < 0.05; **p < 0.01. Standard errors clustered by teacher are reported in parentheses. See table 6 notes for further details.

Estimates of the differential effect of layoffs based on seniority, evaluation scores, and value-added scores are practically unchanged. The coefficient associated with average evaluation scores in column 3 becomes slightly more negative (–0.037, p = .115) but is no longer statistically significant. The marginal effect of layoffs across value-added scores also becomes slightly more negative (–0.113, p = .077) and remains significant at the 10 percent level. Estimates from the full model in column 7 continue to show that measures of teacher effectiveness dominate seniority as a predictor of the effect of layoffs.

Identifying Assumptions

Here I explore the direction, and degree to which, potential violations of the identifying assumptions for estimates of the marginal effect of layoffs across teacher characteristics could bias my estimates. Estimating the correlation between principals’ ability to mitigate the effect of layoffs and teachers’ seniority or effectiveness is largely intractable because this ability of principals is unobserved. One plausible violation would be a negative correlation where principals who were more effective at mitigating layoff impacts were also more successful at identifying and securing the layoff of low-performing teachers, whereas principals who were less effective at mitigating the impacts of layoffs had teachers laid off who were relatively higher performing. This scenario would bias my estimates toward zero. It seems likely, however, that the layoff selection process in CMS, where central office officials, not principals, made the final layoff decisions, would prevent any strong systematic relationship between principals’ ability to mitigate layoff impacts and teachers’ characteristics. Furthermore, teacher sorting does not appear to have constrained the option to lay off low-performing or novice teachers in some schools but not others, potentially inducing a mechanical correlation between principal ability and laid-off teachers’ characteristics. Analyses of variance show that over 84 percent of the variation in seniority, evaluation ratings, and value-added scores exists within schools.

Next, I consider whether seniority or measures of teacher effectiveness are differentially predictive of the schoolwide effects of layoff. My previous estimates suggest that measures of effectiveness are much stronger predictors of the grade-specific effects of layoffs than seniority. One possibility is that schoolwide peer effects follow the same pattern as grade-level peer effects. Jackson and Bruegmann (2009) provide direct evidence of the relative magnitude of grade-level peer effects based on measures of seniority and value added. They found that when the average seniority and value added of a teacher's grade-level peers are both included in a model, only value added is a strong and significant predictor of student achievement. These findings mirror my results from model 2 when both measures are included (table 6, column 6). If schoolwide peer effects follow this same pattern, then my results on the total effect of layoffs will understate the relative difference in the predicative power of teacher effectiveness over seniority.

I explore this assumption in my own data by estimating the relationship between layoffs and schoolwide achievement using a difference-in-differences design. To implement this approach, I modify model 2 by replacing grade-specific predictors for LAYOFF and the average characteristics of laid-off teachers, C, with corresponding schoolwide measures. I also replace school-by-grade and school-by-year fixed effects with school fixed effects in order to contrast the average achievement in a school prior to layoffs with achievement in 2010, the first year after layoffs. The coefficients associated with C capture the conditional correlation between first-difference estimates of schoolwide layoff effects and the seniority or effectiveness of laid-off teachers. I find no evidence for differential schoolwide effects of layoffs based on the seniority or value-added scores of laid-off teachers. However, I estimate that laying off a teacher that is rated 1 SD higher by her principal is associated with a −0.026 SD (p = .000) difference in student achievement. This makes sense given that principals can at least partially observe teachers’ schoolwide peer effects and the TPAI-R evaluation rubric explicitly incorporates teachers’ performance outside of their classroom, such as in dimension 7.2: “Teacher participates in the development of a broad vision of the school.” These findings suggest that my within-school across-grade modeling approach may underestimate the marginal negative effect of laying off a teacher with a high evaluation score relative to a teacher with a low score.

Subject-Specific Layoffs

The primary analyses presented here pool layoffs across all classroom teachers and examine their effect on students’ achievement in mathematics. This broad approach helps to increase the precision of my estimates and reflects the joint production process of education, although it obscures any potential differences of layoffs based on the subject taught by teachers. We might expect the loss of an effective math teacher to have a larger impact on student achievement in mathematics than the loss of an effective English teacher. I explore this possibility by fitting models that compare the effect on mathematics achievement of laying off a math teacher compared with laying off only non-math teachers in the prior school-grade-year.

I find suggestive evidence that laying off math teachers has a larger effect on mathematics achievement than laying off non-math teachers. As shown in table 8 column 1, the coefficient associated with math teacher layoffs is over 3.5 times that of nonmath teacher layoffs, although these estimates are not statistically significantly different from zero or each other. I extend this model by allowing the differential effect of seniority and performance to vary across math and nonmath teacher layoffs. The results suggest that teacher characteristics matter more for math teachers than nonmath teachers when examining their effect on mathematics achievement. Across all three measures, the characteristics of laid-off math teachers are significant predictors, whereas those of nonmath teachers are not. This contrast is particularly large for value-added scores where the coefficient associated with math value-added scores is −0.077, and the coefficient associated with reading value added is −0.006. However, I am unable to reject the null of equivalence between math and non-math teachers for the pairs of coefficients associated with these characteristics.

Table 8. 
The Relationship between Teacher Layoffs in a Previous Year and Student Achievement in Mathematics across Math and Non-Math Teachers
(1)(2)(3)(4)
LAYOFF Math −0.021 0.000 −0.039 0.017 
 (0.025) (0.031) (0.025) (0.051) 
LAYOFF Non-Math −0.006 −0.004 −0.033 0.034 
 (0.028) (0.029) (0.029) (0.061) 
Seniority Math  −0.003+   
  (0.002)   
Seniority Non-Math  0.000   
  (0.002)   
Evaluation Math   −0.027+  
   (0.016)  
Evaluation Non-Math   −0.021  
   (0.020)  
Math Value-Added    −0.077+ 
    (0.041) 
Reading Value-Added    −0.006 
    (0.021) 
Observations 241,572 241,572 241,572 241,572 
p-value from Joint F-test – 0.302 0.793 0.198 
(1)(2)(3)(4)
LAYOFF Math −0.021 0.000 −0.039 0.017 
 (0.025) (0.031) (0.025) (0.051) 
LAYOFF Non-Math −0.006 −0.004 −0.033 0.034 
 (0.028) (0.029) (0.029) (0.061) 
Seniority Math  −0.003+   
  (0.002)   
Seniority Non-Math  0.000   
  (0.002)   
Evaluation Math   −0.027+  
   (0.016)  
Evaluation Non-Math   −0.021  
   (0.020)  
Math Value-Added    −0.077+ 
    (0.041) 
Reading Value-Added    −0.006 
    (0.021) 
Observations 241,572 241,572 241,572 241,572 
p-value from Joint F-test – 0.302 0.793 0.198 

Notes:+p < 0.10. Standard errors clustered by teacher are reported in parentheses. P-values are from joint F-tests of coefficient equality between the math and nonmath seniority, evaluation, and value-added coefficients in each model. See table 6 notes for further details.

Class-Size and Grade-Specific Turnover

Increases in class size are one primary mechanism through which layoffs could have affected student achievement in the following year. Across the district, average class sizes increased by less than one student across all grades from 2008–09 to 2009–10.27 I estimate whether grade-specific layoffs in a given school year caused larger increases in class size in the affected grade by re-estimating model 2 using average class size as the outcome. As shown in table 9, I find no statistically significant relationship between layoffs in a given school-grade and average class size in the same school-grade the following year, and am able to reject differential changes as small as one student, on average.

Table 9. 
Examining Class Size and Grade-Specific Turnover as Potential Mediators of the Effects of Teacher Layoffs
Outcomes
Class Size (1)Proportion of Teachers Who Taught in Same Grade & School Last Year (2)
LAYOFF −0.005 −0.018 
 (0.412) (0.013) 
Observations 239,661 212,587 
Outcomes
Class Size (1)Proportion of Teachers Who Taught in Same Grade & School Last Year (2)
LAYOFF −0.005 −0.018 
 (0.412) (0.013) 
Observations 239,661 212,587 

Notes: Standard errors clustered by school-grade-year are reported in parentheses given the outcomes are defined at the school-grade-year level. Column 1 reports results from a regression of average class size in a school-grade-year on the number of teachers laid off in a student's grade in the prior year, conditional on the full set of covariates and fixed effects in model 2 in the paper and described in the notes of table 6. Column 2 reports results from a similar model where the outcome is a measure of the proportion of teachers in a school-grade-year who taught in the same grade and school the previous year.

A second possible mechanism could be the degree to which teachers were reassigned to different grade levels as a result of layoffs. Research has documented that grade-specific experience constitutes a large portion of the total returns to experience that teachers accrue over their careers (Ost 2014). I examine this potential mechanism by calculating the proportion of teachers in a given school-grade-year who taught in that same school and grade in the previous year. Not surprisingly, I find the rate of teachers returning to the same grade and school jumped from 52 percent in 2008–09 to 68 percent in 2009–10. This large increase likely reflects the rapid decrease in alternative labor market opportunities for teachers during the recession. I then examine whether there was any differential degree of grade-switching across grades within a school by refitting model 2 using this school and grade specific measure of retention. Again, I find no statistically significant effect, although I cannot rule out differential effects as large as a 4 percentage point increase in grade-specific turnover. Together, these exploratory analyses suggest that the district was able to distribute any effect of the layoffs on class-size and grade-switching evenly across grades within a school.

8.  Comparing Alternative Layoff Policies

Comparing the characteristics of laid-off teachers selected under alternative layoff policies highlights the differential consequences these policies can have for district budgets and teacher effectiveness. In table 10, I contrast the average characteristics of teachers laid off in CMS in 2009 and 2010 to the characteristics of teachers who would have been selected under six alternative policies: inverse seniority, inverse evaluation scores (1-year and average), inverse value-added scores28 (1-year and average), and inverse composite performance scores.29 I obtain these estimates by replicating the within-school selection process used by CMS where the number of layoffs per school was predetermined by the district based on enrollment projections and current staffing levels. Thus, I maintain the same number of teachers for layoffs within each school as the actual policy (as well as the same total number), but change the decision rule that determines which teacher(s) in a school were selected. I also conduct simulations of a district-wide selection process holding constant the total reduction in salary as opposed to the total number of layoffs. Selecting teachers across the district for layoffs allows me to estimate the number of layoffs needed under each alternative selection rule to reach an equivalent level of savings achieved by the actual layoffs.

Table 10. 
The Average Characteristics of Teachers Selected through Simulated Layoff Policies
Within-School Selection Process with a Fixed Number of Layoffs in Each SchoolDistrict-Wide Selection Process with Fixed Total Salary Reduction Target
Number of Teachers with DataTotal Salary ($)Avg. SeniorityAvg. Evaluation ScoreAvg. Math VA ScoreAvg. Reading VA ScoreTotal Layoffs
 Panel A: 2009 
Actual policy 9,153 30,462,669 7.8 −0.390 −0.004 −0.005 645 
Actual policy w/o retired teachersa 8,969 21,405,140 2.7 −0.687 −0.024 −0.013 505 
Inverse seniority 9,153 25,047,152 0.1 −0.389 −0.007 −0.001 781 
Inverse evaluation score: 1-year 4,825 26,355,558 1.8 −1.022 −0.035 −0.011 762 
Inverse evaluation score 8,024 28,464,276 4.0 −1.246 −0.044 −0.013 671 
Inverse value-added score: 1-year 1,726 27,706,350 3.7 −0.296 −0.056 −0.017 691 
Inverse value-added score 2,241 28,907,214 5.0 −0.289 −0.083 −0.036 671 
Inverse composite evaluation & value-added 8,285 28,826,970 4.4 −1.208 −0.138 −0.052 673 
 Panel B: 2010 
Actual policy 8,784 16,049,349 1.8 −0.915 −0.063 −0.018 398 
Inverse seniority 8,686 15,025,467 0.2 −0.746 −0.018 −0.014 425 
Inverse evaluation score: 1-year 8,212 17,026,574 3.6 −1.16 −0.057 −0.025 364 
Inverse evaluation score 8,561 17,416,962 4.2 −1.349 −0.055 −0.014 372 
Inverse value-added score: 1-year 1,654 16,816,704 3.3 −0.423 −0.058 −0.016 358 
Inverse value-added score 2,396 17,906,866 5.6 −0.33 −0.087 −0.032 351 
Inverse composite evaluation & value-added 8,568 17,513,618 4.6 −1.305 −0.155 −0.060 371 
Within-School Selection Process with a Fixed Number of Layoffs in Each SchoolDistrict-Wide Selection Process with Fixed Total Salary Reduction Target
Number of Teachers with DataTotal Salary ($)Avg. SeniorityAvg. Evaluation ScoreAvg. Math VA ScoreAvg. Reading VA ScoreTotal Layoffs
 Panel A: 2009 
Actual policy 9,153 30,462,669 7.8 −0.390 −0.004 −0.005 645 
Actual policy w/o retired teachersa 8,969 21,405,140 2.7 −0.687 −0.024 −0.013 505 
Inverse seniority 9,153 25,047,152 0.1 −0.389 −0.007 −0.001 781 
Inverse evaluation score: 1-year 4,825 26,355,558 1.8 −1.022 −0.035 −0.011 762 
Inverse evaluation score 8,024 28,464,276 4.0 −1.246 −0.044 −0.013 671 
Inverse value-added score: 1-year 1,726 27,706,350 3.7 −0.296 −0.056 −0.017 691 
Inverse value-added score 2,241 28,907,214 5.0 −0.289 −0.083 −0.036 671 
Inverse composite evaluation & value-added 8,285 28,826,970 4.4 −1.208 −0.138 −0.052 673 
 Panel B: 2010 
Actual policy 8,784 16,049,349 1.8 −0.915 −0.063 −0.018 398 
Inverse seniority 8,686 15,025,467 0.2 −0.746 −0.018 −0.014 425 
Inverse evaluation score: 1-year 8,212 17,026,574 3.6 −1.16 −0.057 −0.025 364 
Inverse evaluation score 8,561 17,416,962 4.2 −1.349 −0.055 −0.014 372 
Inverse value-added score: 1-year 1,654 16,816,704 3.3 −0.423 −0.058 −0.016 358 
Inverse value-added score 2,396 17,906,866 5.6 −0.33 −0.087 −0.032 351 
Inverse composite evaluation & value-added 8,568 17,513,618 4.6 −1.305 −0.155 −0.060 371 

Notes: Within-school simulated layoff policies replicate the selection process used by CMS where the number of layoffs per school was first determined by the district based on enrollment projections and current staffing levels. The actual number of layoffs in a school is fixed across all within-school simulations. District-wide simulations select teachers across the district for layoffs based on a given inverse selection process until the total salaries of laid-off teachers equals that of the teachers selected in the actual layoffs. Principal evaluation scores and value-added scores are both rolling averages unless otherwise noted. Principal evaluation scores are standardized in a teacher-year-level data set. Value-added scores are expressed in student-test-score standard deviations. Average evaluation scores and value-added scores for the samples of teachers selected for layoffs across simulated policies are calculated for those teachers for whom the relevant scores are available.

aThe statistics for the actual policy without retired teachers are not simulated. They present the same descriptive statistics as the actual policy without the 140 returning retired teachers who were laid off in 2009. Only one returning retired teacher was laid off in 2010.

As shown in panel A of table 10, discretionary teacher layoffs in 2009 reduced the budget of CMS by almost $30.5 million in annual salaries, over $5 million more than a comparable inverse seniority policy.30 It would have required an additional 136 teacher layoffs to achieve an equivalent reduction in salary expenditures under a seniority-based policy because of the high savings the district realized by targeting returning retired teachers. In fact, layoffs under each of the performance-based policies would also result in less total savings than the district realized in 2009. An inverse seniority policy in 2010 would have again required more layoffs (27) to achieve the same payroll savings, whereas inverse performance policies would have required at least 26 fewer layoffs to reach the same level of savings.

In 2009, it appears as though the district selection process produced teachers who were, on average, no more or less effective than those who would have been selected under an inverse-seniority policy. However, these 2009 averages mask the process whereby the district almost uniformly laid off all returning retired teachers who were collecting both salaries and pension benefits. In the second row of panel A, I present the average characteristics of laid-off teachers excluding these 140 returning retired teachers. Excluding these veteran teachers reveals how the district selected teachers for layoffs who were, on average, less effective than those identified under an inverse-seniority policy. Non-retired laid-off teachers in 2009 received evaluation scores that were 0.3 SD lower, on average, than teachers who would have been laid off under an inverse-seniority policy. In 2010, CMS selected teachers for layoffs who were rated, on average, 0.17 SD lower by their principals and had 0.045 SD lower value-added scores in mathematics than those teachers who would have been laid off under an inverse-seniority policy.

Results in table 10 also show that all of the inverse performance-based policies would have resulted in laying off substantially less-effective teachers, on average, than an inverse-seniority layoff procedure. Across both years, inverse value-added policies would have resulted in the lowest average value-added scores among laid-off teachers but higher average principal evaluation scores than under either the discretionary policy or an inverse-seniority process. These results suggest that a layoff policy based exclusively on objective measures of teacher effectiveness would fail to consider all aspects of teacher performance that principals value. These simulations also show how a policy that considers both evaluation and value-added scores results in the selection of teachers with nearly as low value-added scores as a pure inverse-value-added score policy, but substantially lower evaluation scores. Of course, the value of subjective performance measures depends on principals’ capacity to assess teachers’ effectiveness and willingness to differentiate among them on formal evaluations, both of which may vary across districts.

9.  Conclusion

In 2009, and again in 2010, Charlotte Mecklenburg Schools laid off hundreds of teachers following sharp cuts in local and state funding for public education. CMS implemented layoffs with broad discretion afforded by North Carolina's ban on collective bargaining. In addition to concentrating layoffs among nontenured teachers, returning retired teachers, late-hired teachers, and teachers with licensure deficiencies, the district also targeted underperforming teachers. Fifty-eight percent of all teachers who received a “Below Standard” or “Unsatisfactory” rating on any of the eight evaluation rubric domains were laid off.

Although these findings are specific to one district, they provide the first evidence of a district's revealed preferences when implementing discretionary layoffs at scale. CMS principals and administrators appear to have considered multiple teacher characteristics rather than defaulting to an inverse-seniority process or targeting the highest paid teachers as some have claimed would happen. This example also provides an additional case study for examining trends in layoffs across districts. Similar to districts in Washington State, CMS schools were most likely to lay off teachers who taught electives, such as foreign language and the arts (Goldhaber and Theobald 2013).

Layoffs in CMS also provide a compelling policy context in which to examine the validity of objective and subjective measures of teacher effectiveness. I find evidence that laying off a more effective teacher, as measured by either subjective or objective performance metrics, decreased mathematics achievement in the following year compared with laying off an ineffective teacher. In contrast, laying off a more senior teacher resulted in at most a substantially smaller decrease in achievement when compared with laying off an early-career teacher. When compared simultaneously, measures of teacher effectiveness strictly dominate seniority as predictors of the effect of teacher layoffs on future achievement. Teacher experience matters; experienced teachers are, on average, more effective than their early-career colleagues and make important contributions to their schools outside of the classroom. However, performance measures are better indicators of the likely effect of layoffs on achievement compared to seniority.

Simulation analyses illustrate that the district was able to use its discretion to lay off less-effective teachers in both 2009 and 2010 compared to an inverse-seniority policy. An inverse layoff policy using a combination of principal evaluations and value-added scores would have lowered the effectiveness of laid-off teachers by even more in each year. Rockoff et al. (2012) found that principals randomly assigned to receive value-added scores for their teachers were able to use these data to improve their personnel decisions, which led to small improvements in teacher effectiveness. These results, combined with the findings herein, suggest that CMS could have further reduced the negative effect of layoffs had they also used value-added scores to inform the layoff process.

Laying off teachers based on their seniority in the district, rather than their performance in the classroom, results in greater job losses and exacerbates the negative effects of layoffs on student achievement. Layoff policies that do not incorporate increasingly available measures of teacher effectiveness fail to consider all the best available information when making high-stakes decisions. However, exchanging one inflexible inverse layoff criterion for another will not provide districts with any discretion in navigating a complex process aimed at preventing a variety of negative consequences. Going forward, it will be important to examine the long-run effects of layoffs and to obtain a more detailed understanding of an optimal selection process should districts and unions decide that layoffs are unavoidable.

Notes

1. 

See National Council on Teacher Quality (2010), The New Teacher Project (2010), USA Today (2011), and Abramson (2011) for examples.

2. 

Previous studies comparing similar measures include Jacob and Lefgren (2008), Harris and Sass (2009), and Rockoff and Speroni (2011).

3. 

Over 6,000 Los Angeles (CA) Unified School District teachers received pink slips in 2009, although only one out of every three of these teachers ultimately lost their job (Billups 2009). In two extreme cases, the Detroit (MI) Public Schools and the Providence (RI) Public School Department sent layoff notices to every teacher in the district in anticipation of school closings and massive layoffs (Luhby 2011).

4. 

In states where collective bargaining is explicitly illegal, local school boards maintain the authority to determine RIF policies.

5. 

The four other states are Texas, Georgia, South Carolina, and Virginia. State legislatures in Wisconsin, Ohio, and Michigan have all recently considered restricting collective bargaining rights.

6. 

Pursuant to the 1997 Excellent Schools Act, teachers become eligible for tenure or “career status” after they have been employed by a North Carolina public school system for four consecutive years. Tenure is awarded by majority vote of the local school board members. Tenured teachers who transfer districts within the state may be awarded tenure immediately by the board of their new district or may be subject to a one-year probationary period. See G.S. 115c-325 of the Excellent Public Schools Act for complete details and specific language (www.ncga.state.nc.us/EnactedLegislation/Statutes/HTML/BySection/Chapter_115c/GS_115C-325.html).

7. 

See CMS School Board Policies, Section G, Policy Code GCQA and GCQB for complete details and specific language (www.cms.k12.nc.us/boe/Pages/BoardPolicies.aspx).

8. 

All of the teachers who were laid off by CMS lost their jobs and were placed in the RIF Pool as described in Appendix  A. Of the teachers laid off in 2008–09, 185 were rehired by the district before the start of the following school year to fill other open positions. The following year, 174 laid-off teachers were rehired after losing their jobs and being placed in the RIF Pool.

9. 

I define teachers as individuals in the Human Resources employment files who are paid based on the teacher salary schedule, who have titles indicating they are classroom teachers, and who are matched to a specific school.

10. 

I conduct parallel analyses which include school-by-year fixed effects to model the within-school selection process where layoffs were allocated across schools based on enrollment projections and student-teacher ratio targets, and then principals and HR personnel selected teachers for layoffs. This is not my preferred approach because it requires that I drop all teachers in school years where there were no layoffs. Results from these analyses are consistent with those presented subsequently. Additional analyses using linear probability models to model the within-school selection process with the full analytic sample also produce similar estimates.

11. 

When I vary the number of years included in my analytic sample I find that point estimates remain largely unchanged but that standard errors increase as I restrict the data range.

12. 

Analyses using mathematics test scores as outcomes include 2,921 teachers and 139 schools.

13. 

Over the two years, the district laid off 38 math and literacy facilitators/coaches and 23 media and technology specialists as well as 19 counselors, 14 school psychologists, 11 social workers, and 7 deans of students.

14. 

I specify a dichotomous measure as a parsimonious and nonparametric approach. Exploratory analyses using a linear specification of the number of layoffs in a school-grade-year, the proportion of teachers laid off in a school-grade-year, or a set of binned indicators all decrease the precision of my estimates and provide little evidence of a linear or otherwise parametric functional form.

15. 

Model 2 examines the relationship between laid-off teachers’ value-added scores estimated using data through 2008–09 and students’ conditional achievement in 2010, relative to students in other grades in the same school. Some students who contribute to the value-added estimates of laid-off teachers in 2009 also contribute test scores in 2010, creating the potential for bias arising from correlated errors on both the left and right hand side of the regression.

16. 

C is missing when all laid-off teachers in a school-grade-year are missing evaluation or value-added scores. It is undefined for all school-grade-year cells in which a layoff did not occur in the previous year. I impute zeros for both types of missingness and include separate indicator variables for each type of missingness.

17. 

I include indicators for the student's gender, race, limited English proficiency status, and special education status. For peer characteristics, I include the means of all of these predictors as well as mean prior year achievement in mathematics and reading. I restrict the sample to exclude any teacher-year in which fewer than five students had valid test scores. I exclude any class with more than 90 percent of students requiring special educational services.

18. 

In supplemental analyses, I find that nonparametric parameterizations of C produce a very similar pattern of results as those reported subsequently. This is important given evidence of the nonlinear relationship between experience and teacher effectiveness (Papay and Kraft forthcoming).

19. 

Returning retired teachers retain their seniority and full salary but not their tenure status.

20. 

To receive an average rating of below “At Standard,” a teacher must have been rated “Below Standard” or “Unsatisfactory” on at least one of the eight rubric domains.

21. 

These correlations are from rolling averages of principal evaluation and value-added scores that are not disattenuated for measurement error. Correlations among principal evaluation and value-added scores derived from a single year are 0.22 and 0.16 for math and reading, respectively.

22. 

Table 2 presents estimates of the probability of layoffs across quartiles of a given school characteristic and are nearly identical to estimates that condition on student-teacher ratios.

23. 

Corresponding tests in my sample of English teachers also provide no evidence of selection based on the academic ability of rising cohorts

24. 

Appendix table C.1 presents corresponding results for reading achievement.

25. 

Students who only contribute one observation to the analytic sample are predominantly eighth graders in 2005 (30 percent) (the first year of the panel) and fourth graders in 2010 (25 percent) (the last year of the panel). In total, 11,769 of the 43,103 student observations in 2010 are effectively removed from the analytic sample when student fixed effects are included.

26. 

Using this restricted sample, estimates of the marginal effect of layoffs remain large and significant for measures of effectiveness, and near zero and not statistically significant for seniority.

27. 

Changes in class size are calculated using data reported by the North Carolina Department of Instruction. Class averages for “typical classes” are reported by school and grade for elementary and middle schools. Class averages for tested subjects are reported at the high school level.

28. 

For elementary school teachers who have value-added scores in both subjects, I use the average of their scores across subjects.

29. 

I calculate composite performance scores by averaging principal evaluation scores and value-added measures based on all available years of data (restandardized in a teacher-year-level data set so that each score is weighted equally). I include all teachers with at least one performance measure in the sample of teachers considered for layoffs. Corresponding simulation results implemented across the district result in very similar findings for all six alternative policies.

30. 

This estimate does not include the nine teachers who were laid off that I am unable to match to my administrative data. Including their salaries would raise this figure by between $315,000 and $675,000.

Acknowledgments

I thank Andrew Baxter, Thomas Tomberlin, Kim Brazzell, and the Charlotte-Mecklenburg Schools for their support of this research. I am grateful to Thomas Bailey, Sarah Cohodes, David Deming, Susan Moore Johnson, Lawrence Katz, Richard Murnane, Jonah Rockoff, and Martin West for their helpful comments on earlier drafts of this manuscript.

REFERENCES

Abramson
,
Larry
.
2011
.
Pressure mounts to axe teacher seniority rules
.
National Public Radio
:
Weekend Sunday Edition
.
6 March
. Available www.npr.org/2011/03/06/134275966/pressure-mounts-to-ax-teacher-seniority-rules.
Accessed July 2015
.
Billups
,
Andrea
.
2009
.
Massive layoffs force teachers to re-evaluate careers
.
The Washington Times
,
9
August
.
Boyd
,
Donald
,
Hamp
Lankford
,
Susanna
Loeb
, and
James
Wyckoff
.
2011
.
Teacher layoffs: An empirical illustration of seniority versus measures of effectiveness
.
Education Finance and Policy
6
(
3
):
439
454
. doi:10.1162/EDFP_a_00041
Chetty
,
Raj
,
John N.
Friedman
, and
Jonah E.
Rockoff
.
2014
.
Measuring the impacts of teachers I: Evaluating bias in teacher value-added estimates
.
American Economic Review
104
(
9
):
2593
2632
. doi:10.1257/aer.104.9.2593
Ellerson
,
Noelle M.
2010
.
A cliff hanger: How America's public schools continue to feel the impact of the economic downturn
.
Arlington, VA
:
American Association of School Administrators
.
Goldhaber
,
Dan
, and
Roddy
Theobald
.
2013
.
Managing the teacher workforce in austere times: The determinants and implications of teacher layoffs
.
Education Finance and Policy
8
(
4
):
494
527
. doi:10.1162/EDFP_a_00111
Goldin
,
Claudia
, and
Cecelia
Elena
Rouse
.
2000
.
Orchestrating impartiality: The impact of ‘blind’ auditions on female musicians
.
American Economic Review
90
(
4
):
715
741
. doi:10.1257/aer.90.4.715
Guarino
,
Cassandra
,
Mark D.
Reckase
, and
Jeffrey M.
Wooldridge
.
2015
.
Can value-added measures of teacher performance be trusted
?
Education Finance and Policy
10
(
1
):
117
156
. doi:10.1162/EDFP_a_00153
Hanushek
,
Eric A.
, and
Steven G.
Rivkin
.
2010
.
Generalizations about using value-added measures of teacher quality
.
American Economic Review
100
(
2
):
267
271
. doi:10.1257/aer.100.2.267
Harris
,
Douglas
, and
Tim R.
Sass
.
2006
.
Value-added models and the measurement of teacher quality
.
Unpublished paper, Florida State University
.
Harris
,
Douglas
, and
Tim R.
Sass
.
2009
.
What makes for a good teacher and who can tell?
CALDER Working Paper No. 30, Urban Institute
.
Hill
,
Heather C.
,
Charalambos Y.
Charalambous
, and
Matthew A.
Kraft
.
2012
.
When rater reliability is not enough: Teacher observation systems and a case for the generalizability study
.
Educational Researcher
41
(
2
):
56
64
. doi:10.3102/0013189X12437203
Jackson
,
Kirabo C.
, and
Elias
Bruegmann
.
2009
.
Teaching students and teaching each other: The importance of peer learning for teachers
.
American Economic Journal: Applied Economics
1
(
4
):
85
108
. doi:10.1257/app.1.4.85
Jacob
,
Brian A.
2005
.
Accountability, incentives and behavior: The impact of high-stakes testing in the Chicago public schools
.
Journal of Public Economics
89
(
5/6
):
761
796
. doi:10.1016/j.jpubeco.2004.08.004
Jacob
,
Brian A.
, and
Lars J.
Lefgren
.
2008
.
Can principals identify effective teachers? Evidence on subjective performance evaluations in education
.
Journal of Labor Economics
26
(
1
):
101
136
. doi:10.1086/522974
Jacob
,
Brian A.
, and
Steven D.
Levitt
.
2003
.
Rotten apples: An investigation of the prevalence and predictors of teacher cheating
.
Quarterly Journal of Economics
118
(
3
):
843
877
. doi:10.1162/00335530360698441
Jepsen
,
Christopher
, and
Steven
Rivkin
.
2009
.
Class size reduction and student achievement: The potential tradeoff between teacher quality and class size
.
Journal of Human Resources
44
(
1
):
223
250
. doi:10.1353/jhr.2009.0008
Kane
,
Thomas J.
, and
Douglas O.
Staiger
.
2008
.
Estimating teacher impacts on student achievement: An experimental evaluation
.
NBER Working Paper No. 14607
.
Koedel
,
Cory
, and
Julian R.
Betts
.
2011
.
Does student sorting invalidate value-added models of teacher effectiveness? An extended analysis of the Rothstein critique
.
Education Finance and Policy
6
(
1
):
18
42
. doi:10.1162/EDFP_a_00027
Luhby
,
Tami
.
2011
.
Detroit public school teachers get layoff notices
.
CNN Money
,
15
April
.
MacLeod
,
W. Bentley
.
2003
.
Optimal contracting with subjective evaluation
.
American Economic Review
93
(
1
):
216
240
. doi:10.1257/000282803321455232
Martinez
,
Barbara
.
2010
.
Teacher seniority rules challenged
.
The Wall Street Journal
,
19
February
.
McCaffrey
,
Daniel F.
,
J. R.
Lockwood
,
Daniel
Koretz
,
Thomas A.
Louis
, and
Laura
Hamilton
.
2004
.
Models for value-added modeling of teacher effects
.
Journal of Educational and Behavioral Statistics
29
(
1
):
67
101
. doi:10.3102/10769986029001067
Medina
,
Jennifer
.
2011
.
Teacher layoff plans in Los Angeles pose broad implications
.
The New York Times
,
4
March
.
National Council on Teacher Quality (NCTQ)
.
2010
.
Teacher layoffs: Rethinking “last-hired, first-fired” policies
.
Washington, DC
:
National Council on Teacher Quality
.
Ost
,
Ben
.
2014
.
How do teachers improve? The relative importance of specific and general human capital
.
American Economic Journal: Applied Economics
6
(
2
):
127
151
.
Papay
,
P. John
, and
Matthew A.
Kraft
(
forthcoming
).
Productivity returns to experience in the teacher labor market: Methodological challenges and new evidence of long-term career improvement
.
Journal of Public Economics
. In press. Available www.sciencedirect.com/science/article/pii/S0047272715000304.
Accessed 3 June 2015
.
Prendergast
,
Canice
.
1999
.
The provision of incentives in firms
.
Journal of Economic Literature
37
(
1
):
7
63
. doi:10.1257/jel.37.1.7
Prendergast
,
Canice
, and
Robert
Topel
.
1993
.
Discretion and bias in performance evaluation
.
European Economic Review
37
(
1
):
355
365
. doi:10.1016/0014-2921(93)90024-5
Rockoff
,
Jonah E.
, and
Ceceli
Speroni
.
2011
.
Subjective and objective evaluations of teacher effectiveness: Evidence from New York City
.
Labour Economics
18
(
5
):
687
696
. doi:10.1016/j.labeco.2011.02.004
Rockoff
,
Jonah E.
,
Douglas O.
Staiger
,
Thomas J.
Kane
, and
Eric S.
Taylor
.
2012
.
Information and employee evaluation: Evidence from a randomized intervention in public schools
.
American Economic Review
102
(
7
):
3184
3213
. doi:10.1257/aer.102.7.3184
Roza
,
Marguerite
.
2007
.
Frozen assets: Rethinking teacher contracts could free billions for school reform
. Available www.crpe.org/publications/frozen-assets-rethinking-teacher-contracts-could-free-billions-school-reform.
Accessed July 2015
.
Roza
,
Marguerite
.
2009
.
Seniority-based layoffs will exacerbate job loss in public education
. Available www.crpe.org/sites/default/files/rr_crpe_layoff_feb09_0.pdf.
Accessed 4 May 2015
.
Sepe
,
Cristina
, and
Marguerite
Roza
.
2010
.
The disproportionate impact of seniority-based layoffs on poor, minority students
. Available www.crpe.org/sites/default/files/rr_crpe_layoffs_rr9_may10_0.pdf.
Accessed 4 May 2015
.
The New Teacher Project
.
2010
.
A smarter teacher layoff system: How quality-based layoffs can help schools keep great teachers in tough economic times
.
Brooklyn, NY
:
The New Teacher Project
.
Todd
,
Petra E.
, and
Kenneth I.
Wolpin
.
2003
.
On the specification and estimation of the production function for cognitive achievement
.
Economic Journal
113
(
485
):
F3
F33
. doi:10.1111/146-80297.00097
USA Today
.
2011
.
Our view: Teacher layoffs based only on seniority fail kids
. Available http://usatoday30.usatoday.com/news/opinion/editorials/2011-03-09-editorial09_ST_N.htm.
Accessed 4 May 2015
.
Weisberg
,
Daniel
,
Susan
Sexton
,
Jennifer
Mulhern
, and
David
Keeling
.
2009
.
The widget effect
.
Brooklyn, NY
:
The New Teacher Project
.

Appendix A:  CMS School Board–Approved Reduction-in-Force Procedures

  1. The district estimates new Full Time Equivalent (FTE) allocations for schools using projected Average Daily Membership (ADM) and revised student-teacher ratio policies.

  2. Principals identify position categories to cut if their current teaching staff exceeds their allotted FTE for the next academic year.

  3. Human resource officials select which teacher(s) among those in the position(s) identified for elimination will be placed in the district-wide RIF pool in the following order:

    1. Non-Career Teachers

      • Any teacher who received an evaluation of below standard or unsatisfactory on the TPAI-R evaluation instrument

      • Any teachers with a licensure deficiency

      • Any teacher who is currently collecting pension benefits (i.e., returning retired teachers)

      • Any Part-time or Interim teacher

      • Any teacher on an End-of-Year contract

      • Seniority in the district

    2. Career Teachers

      • Based on comparative performance and comments in two most recent TPAI-R evaluations.

    3. All teachers in the RIF pool are informed of the district's intent to lay them off by 15 May.

  4. Principals with more FTE positions than current staff members must select among teachers in the district-wide RIF pool who are qualified for any open positions. Any career teacher must be selected first before noncareer teachers can be considered.

  5. Any teachers who remain in the district-wide RIF pool after the hiring process receive confirmation that they are laid off by 5 June.

Source: CMS memos entitled “Recommended Criteria for Reductions in Pay of Assistant Principals,” and “Reductions in Force of Classroom Teachers and Other Certified Employees Paid on the Teacher Pay Scale.”

Appendix B:  Value-Added Estimation

I fit the following model:
formula
B.1

where the outcome of interest, , is the end-of-year test score for student i in grade g, with teacher j in school s in year t. The outcome test score is modeled as a grade-specific cubic function of the student's prior year achievement, , in both mathematics and reading, vectors of controls for observable student characteristics , the characteristics of a student's peers with the same teacher (), and school characteristics (), as well as grade-by-year fixed effects (). I include indicators for the student's gender, race, limited English proficiency status, and special education status. For peer, and school-level characteristics, I include the means of all of these predictors as well as mean prior year achievement in mathematics and reading. I restrict the sample to exclude any teacher-year in which fewer than five students had valid test scores. I exclude any class with more than 90 percent of students requiring special educational services. Notably, I omit school fixed effects because I am interested in comparing the relative effectiveness of teachers across schools in the district.

I isolate teachers’ persistent effects following Kane and Staiger (2008). I accomplish this by predicting individual teacher random effects () while also including random effects for teacher-years () to account for any transitory class-specific shocks that would otherwise be attributed to the teacher. This approach rescales teacher effects by the reliability of these individual estimates, where reliability is the ratio of estimates of true teacher variance, over the sum of estimated true teacher variance, transitory teacher variance, and random variance in student scores (see Chetty, Friedman, and Rockoff 2014 for a full discussion).

Appendix C:  Tables

Table C.1. 
The Moderating Effect of Teacher Characteristics on the Relationship between Teacher Layoffs in the Previous Year and Student Achievement in Reading
(1)(2)(3)(4)(5)(6)(7)
LAYOFF 0.002 −0.007 −0.002 0.001 −0.016 −0.033 −0.043 
 (0.017) (0.018) (0.018) (0.029) (0.019) (0.033) (0.033) 
Seniority  0.001   0.002 0.003 0.003 
  (0.002)   (0.002) (0.002) (0.002) 
Evaluation   0.009  0.002  0.010 
   (0.013)  (0.014)  (0.014) 
Value-Added    −0.013  −0.026 −0.030+ 
    (0.016)  (0.017) (0.018) 
Observations 221,268 221,268 221,268 221,268 221,268 221,268 221,268 
(1)(2)(3)(4)(5)(6)(7)
LAYOFF 0.002 −0.007 −0.002 0.001 −0.016 −0.033 −0.043 
 (0.017) (0.018) (0.018) (0.029) (0.019) (0.033) (0.033) 
Seniority  0.001   0.002 0.003 0.003 
  (0.002)   (0.002) (0.002) (0.002) 
Evaluation   0.009  0.002  0.010 
   (0.013)  (0.014)  (0.014) 
Value-Added    −0.013  −0.026 −0.030+ 
    (0.016)  (0.017) (0.018) 
Observations 221,268 221,268 221,268 221,268 221,268 221,268 221,268 

Notes:+p < 0.10. Standard errors clustered by teacher are reported in parentheses. See table 6 notes for further details.

Table C.2. 
The Relationship between Domain Elements of the Teacher Performance Appraisal Instrument (Revised) and the Probability of Being Laid Off
(1)(2)(3)(4)(5)(6)(7)(8)(9)
Management of −0.035***        −0.007* 
instructional (0.003)        (0.004) 
time 
Management of  −0.037***       −0.022*** 
instructional  (0.003)       (0.003) 
behavior 
Instructional   −0.034***      −0.014*** 
presentation   (0.003)      (0.004) 
Instructional    −0.028***     0.003 
monitoring    (0.003)     (0.003) 
Instructional     −0.026***    0.005+ 
feedback     (0.003)    (0.003) 
Facilitating      −0.030***   −0.004 
instruction      (0.003)   (0.003) 
Communicating       −0.026***  −0.003 
within the       (0.002)  (0.003) 
education 
environment 
Performing non-        −0.025*** −0.008** 
instructional        (0.002) (0.003) 
duties 
RIF criteria 
controls 
Observations 16,206 16,207 16,203 16,209 16,203 16,190 16,190 16,182 16,160 
(1)(2)(3)(4)(5)(6)(7)(8)(9)
Management of −0.035***        −0.007* 
instructional (0.003)        (0.004) 
time 
Management of  −0.037***       −0.022*** 
instructional  (0.003)       (0.003) 
behavior 
Instructional   −0.034***      −0.014*** 
presentation   (0.003)      (0.004) 
Instructional    −0.028***     0.003 
monitoring    (0.003)     (0.003) 
Instructional     −0.026***    0.005+ 
feedback     (0.003)    (0.003) 
Facilitating      −0.030***   −0.004 
instruction      (0.003)   (0.003) 
Communicating       −0.026***  −0.003 
within the       (0.002)  (0.003) 
education 
environment 
Performing non-        −0.025*** −0.008** 
instructional        (0.002) (0.003) 
duties 
RIF criteria 
controls 
Observations 16,206 16,207 16,203 16,209 16,203 16,190 16,190 16,182 16,160 

Notes:*p < 0.05; **p < 0.01; ***p < 0.001; +p < 0.10. Cells represent average marginal effect estimates derived from logistic regression models. Robust standard errors reported in parentheses. All regressions include school-by-year fixed effects and controls for seniority, retired teachers, late hires, licensure deficiency, and licensure type fixed effects. Domain-specific scores are rolling average scores standardized in a teacher-year-level data set.

Table C.3. 
The Moderating Effect of Teacher Characteristics on the Relationship be-tween Teacher Layoffs in the Previous Year and Student Achievement in Mathematics
Evaluation and Value-Added Scores Constructed from Three Years of Data
(1)(2)(3)(4)(5)
LAYOFF −0.031+ 0.033 −0.027 0.027 0.008 
 (0.018) (0.043) (0.027) (0.057) (0.057) 
Seniority   −0.001 0.001 0.002 
   (0.002) (0.003) (0.003) 
Evaluation: 3-year −0.032+  −0.030  −0.027 
 (0.018)  (0.021)  (0.021) 
Value-Added: 3-year  −0.084+  −0.094* −0.090* 
  (0.043)  (0.044) (0.044) 
Observations 241,572 241,572 241,572 241,572 241,572 
Evaluation and Value-Added Scores Constructed from Three Years of Data
(1)(2)(3)(4)(5)
LAYOFF −0.031+ 0.033 −0.027 0.027 0.008 
 (0.018) (0.043) (0.027) (0.057) (0.057) 
Seniority   −0.001 0.001 0.002 
   (0.002) (0.003) (0.003) 
Evaluation: 3-year −0.032+  −0.030  −0.027 
 (0.018)  (0.021)  (0.021) 
Value-Added: 3-year  −0.084+  −0.094* −0.090* 
  (0.043)  (0.044) (0.044) 
Observations 241,572 241,572 241,572 241,572 241,572 

Notes:*p < 0.05; +p < 0.10. Standard errors clustered by teacher are reported in parentheses. See table 6 notes for further details.