Teacher Layoffs, Teacher Quality, and Student Achievement: Evidence from a Discretionary Layoff Policy

Personnel reductions are a common cost-cutting measure that firms use in response to changes in market demand and macroeconomic downturns. In the public education sector, where personnel costs represent between 60 and 80 percent of total expenditures, school districts can rarely avoid layoffs when faced with a major budget shortfall (Roza 2007). Beginning in 2008, public school districts across the country were forced to implement widespread teacher layoffs in response to large decreases in local and state tax revenues brought on by the Great Recession. The scale at which these layoffs occurred and the ways in which they were implemented have caused educators, policy makers, and researchers to reexamine these practices.

For the majority of districts, collectively bargained agreements, state laws, and legal precedent mandate that these layoffs are based on seniority. The implementation of these long-standing last-hired, first-fired layoff policies has generated considerable criticism among policy organizations and in the popular press because such policies eliminate the jobs of early-career teachers who may be more effective than some of their more experienced peers.¹ An analysis of seniority-based layoffs in Washington State confirmed that teachers who received reduction in force (RIF) notices were no less effective, on average, than those whose jobs were not threatened (Goldhaber and Theobald 2013). In recent years, several state legislatures and district superintendents have attempted to amend or eliminate last-in first-out policies, and advocacy groups have challenged their constitutionality under the equal protection clause in a widely publicized lawsuit (Vergara v. California).

The results of several simulation analyses suggest layoff policies that prioritize teacher effectiveness, as measured by valued-added scores, could result in the selection of less-effective teachers than those who would lose their jobs under inverse-seniority policies (Boyd et al. 2011; Goldhaber and Theobald 2013). Measures of effectiveness derived from student achievement, however, are controversial, only available for a minority of teachers, and, with high stakes attached, can lead to gaming behaviors such as teaching to the test (Jacob 2005) or even outright cheating (Jacob and Levitt 2003). Furthermore, proposals for layoffs based strictly on objective measures of effective teaching focus narrowly on teachers’ ability to raise achievement on standardized tests, while ignoring other educational priorities, organizational needs, and labor market constraints.

Subjective performance evaluations, such as ratings by principals, provide an alternative measure of effectiveness that consider multiple criteria, are widely available, and are capable of distinguishing among the very best and worst teachers (Jacob and Lefgren 2008). Administrators are often reluctant to give employees poor evaluations, however, given existing job protections and the costs negative evaluations can impose on workplace productivity, morale, and trust (Prendergast and Topel 1993; MacLeod 2003; Weisberg et al. 2009). Subjective ratings are also susceptible to rater biases (Prendergast 1999; Goldin and Rouse 2000) and contextual influences that can lower the reliability of scores (Hill, Charalambous, and Kraft 2012). Thus, open questions remain about the efficacy of performance-based layoffs that use either value-added measures or principal evaluations given the potential implementation challenges, unintended consequences, and moderate reliability of these measures.

In this study, I analyze the implementation and consequences of discretionary layoffs in the eighteenth largest public school district in the nation to provide some of the first empirical evidence on performance-based layoffs in education. In total, the Charlotte Mecklenburg Schools (CMS) in North Carolina eliminated almost 2,000 employees, including over 1,000 teaching positions, in the two years following the onset of the Great Recession. CMS identified candidates for layoffs using a set of five general criteria: duplicative positions, enrollment trends, job performance, job qualifications, and length of service. My administrative data set contains both principal evaluation scores, which directly informed the layoff selection process, as well as the necessary data to estimate value-added scores, which were not used by the district. These data also allow me to compare subjective and objective measures of teacher effectiveness by incorporating both measures throughout my analyses.²

Studying the implementation of layoffs in CMS provides a unique opportunity to shed light on several central questions surrounding discretionary layoff policies. Would administrators simply defer to seniority despite the flexibility afforded by such policies? Would they instead target the highest paid (i.e., most senior) teachers for layoffs in an effort to maximize costs savings? Or would they use their discretion to lay off teachers based on multiple factors? I explore these questions by comparing the predictive power of a variety of RIF criteria. I then estimate the grade-specific effects of teacher layoffs on student achievement in CMS with particular attention to the differential effects of layoffs based on teacher seniority and effectiveness. This is the first analysis, that I am aware of, which links the characteristics of teachers being laid off to changes in student achievement. I identify credible estimates by exploiting quasi-experimental variation where some rising cohorts of students in a school entered grades in which a teacher was laid off, while others did not. The key identifying assumption in my analysis requires that the grades in which layoffs occurred in a school were unrelated to the past achievement and characteristics of rising class cohorts in a school. Empirical tests lend strong support to this assumption. Finally, I simulate alternative layoff selection policies and compare the differences in average cost saving and teacher characteristics across layoff pools.

I find that layoffs in CMS were concentrated among probationary teachers with fewer than four years of seniority. Principal evaluations were also strong predictors of the probability of being laid off in CMS, as well as licensure status and licensure type. Layoffs were particularly concentrated among high school teachers as well as foreign language and arts teachers. I find suggestive but inconsistent evidence that, on average, layoffs had negative grade-specific effects on student achievement. These average estimates, however, mask wide variability in the impact of laying off individual teachers. Mathematics achievement in grades that lost an effective teacher (at the 75th percentile), as measured by subjective or objective metrics, decreased by between 0.05 and 0.11 standard deviations more than in grades that lost an ineffective teacher (at the 25th percentile). In contrast, I find that the marginal difference between laying off a senior versus early-career teacher is substantially smaller and statistically insignificant. Simulation analyses provide further evidence that seniority-based layoffs increase job losses but also demonstrate how inverse-layoff policies based on a single performance measure are suboptimal compared with a policy that considers both objective and subjective measures.

Together, these findings have important implications for teacher evaluation systems and layoff policies. They offer new evidence of the predictive validity of both principal evaluations and value-added scores. They provide evidence on the importance of prioritizing performance over seniority when districts are forced to implement teacher layoffs. They also illustrate the value of allowing principals and districts discretion when conducting layoffs rather than constraining these difficult decisions with inflexible policies based on any single measure.

Reductions in force in U.S. public schools broadly mirror the larger macroeconomic cycles of the U.S. economy. Just over three decades ago, teachers faced similar rounds of layoffs on the heels of the energy crisis and increasing stagflation. Districts are often able to forestall layoffs during less-severe economic downturns through a combination of hiring freezes, natural attrition, and incentives for early retirement. Sharp decreases in tax revenues during the Great Recession, however, left many states unable to maintain previous levels of funding for public education. The federal government responded in 2009 with the American Recovery and Reinvestment Act, which allocated over $53.6 billion for state education expenditures, and again in 2010 with the Education Jobs and Medicaid Assistance Act, which provided $10 billion for local school districts. Despite this federal aid, districts were unable to avoid reductions in force.

Estimates of the total number of teacher layoffs in this period range between the tens and hundreds of thousands annually (Martinez 2010). Ellerson (2010) reported that 37 percent of districts cut core classroom teaching positions in 2009–10, and over 60 percent planned cuts in 2010–11. The threat of potential job loss was real for many more teachers. Budget uncertainties and contract requirements to provide early notice of the possibility of termination led some districts to distribute “pink slips” to most teachers they employed.³

Layoff policies in most U.S. public school systems are governed by state legislation or collective bargaining agreements between districts and local teachers’ unions. Beginning in the 1970s, inverse-seniority layoff clauses became widely incorporated into teacher union contracts and adopted by over a dozen state legislatures in an effort to combat discriminatory and nepotistic employment practices. Today, the vast majority of districts are beholden to collective bargaining agreements or laws that continue to prioritize seniority over any other layoff selection criteria.⁴ Of the one hundred public school districts in the National Council on Teacher Quality's Teacher Contract Database (TR3) in 2010, seventy-five followed seniority-based layoff policies (NCTQ 2010).

Pure seniority-based policies maximize the number of jobs that need to be cut to reach a given budget reduction by requiring the least-experienced, and thus lowest-paid, teachers to be the first ones laid off (Roza 2009). These policies also erode districts’ recent efforts to recruit, select, and train highly qualified teachers and may undermine future initiatives to attract talented novice teachers. Furthermore, such policies can cause schools that are primarily staffed with inexperienced teachers to lose large portions of their faculty, some of whom must be replaced with veteran teachers who are forcibly transferred from other schools (Sepe and Roza 2010; Medina 2011).

Charlotte Mecklenburg Schools is the largest school district in North Carolina, and it serves over 137,000 pre-K to twelfth-grade students and employs over 9,000 teachers. Policies governing teacher contracts in CMS are determined by the state legislature, as North Carolina is one of five states where collective bargaining is explicitly illegal.⁵ Local education agencies such as CMS are allowed to determine RIF policies unilaterally, but only in accordance with federal and state fair labor practices, policies, and statutes. CMS School Board policy requires that superintendents make a recommendation to the school board regarding the need for a reduction in force and the criteria to be used in the reduction. Since 2003, CMS School Board policy has stipulated five broad criteria for selecting teachers as part of involuntary reductions: (1) structural considerations that allow for the elimination of duplicative or excess personnel and positions; (2) organizational considerations that permit the superintendent to account for future school enrollment projections; (3) job performance considerations broadly defined as employees’ recent performance on the job; (4) job qualifications, such as tenure status,⁶ education, licensure type, and licensure status, as well as more abstract qualities such as leadership abilities and future potential; and (5) length of continuous, full-time service with the district (i.e., seniority).⁷

In March 2009, then-Superintendent Gorman presented the CMS School Board with recommendations for employee reductions, including at least 456 classroom teaching positions, in order to make up an $87 million budgetary shortfall. The superintendent's proposal outlined three key steps that would be used to implement the layoffs: (1) the district would allocate layoffs across schools based on projected enrollment trends, (2) principals would identify position categories (such as grade levels or subjects) that would be reduced, and (3) district officials would select which teacher(s) among those in the identified position categories would be laid off (see Appendix  A for details). The CMS School Board ultimately approved the superintendent's proposal in 2009, as well as a similar proposal the following year for the elimination of approximately 600 classroom teachers. Notably, student performance was not included as a criterion to be used in the RIF process in either of the two years.

Although the board-approved RIF procedures specified a clear sequence of steps, the school board also granted the superintendent discretionary authority for “limited exceptions” to these procedures. In practice, principals commonly identified specific teachers rather than position categories to be considered for layoffs within their school. Given the discretionary nature of the RIF process, the significant implementation challenges inherent in executing the proposed procedures, and the flexibility principals had to determine which teachers could be considered for layoffs, it remained unclear which teachers would ultimately lose their jobs.

I use an administrative data set that links students, teachers, and test records across a 13-year panel of data from 1997–98 to 2009–10. Student data include demographic information and annual state test results in reading and mathematics. Data on teachers from human resource files include demographic information, tenure status, licensure type and status, title descriptions, as well as scores on a performance assessment rubric evaluated by principals. I combine this panel data set with RIF data files provided by CMS for the 2008–09 and 2009–10 school years that identify 654 laid-off teachers in the summer of 2009 and 433 in 2010.⁸ I am able to match 1,043 of these teachers (96 percent of all RIFed teachers) to my full panel of administrative data using unique identifiers. I complement these quantitative data with relevant state statutes, CMS School Board documents, newspaper articles, and interviews with CMS administrators familiar with the layoff process.

I construct a variety of measures for the teacher characteristics specified in the Board-approved RIF criteria. I create a measure of seniority using employment records from human resource files. For those teachers new to the district since 1997–98, I define seniority as the number of years in the data set a teacher was employed by CMS. For teachers who were hired prior to the 1997–98 school year, I define seniority as the minimum of either the difference between the current academic year and teachers’ district hire date or their level of salary experience credit.

Detailed data on teacher licensure status allow me to construct indicators for probationary (nontenured) teachers, retired teachers who have since been rehired (commonly referred to as “double dippers” because they collect a salary and pension simultaneously), and teachers with a licensure deficiency (teachers without a license or with a conditional or temporary license). I also create a set of indictors for licensure type that includes mathematics, English language arts, science, social studies, foreign language, arts, physical education, English as a second language, special education, elementary, and early education licenses. Finally, I create an indicator for late-hired teachers who were hired after the start of the academic year using detailed hire-date records.

I construct performance evaluation scores using principals’ ratings of teachers on a statewide evaluation protocol. Since 2001, North Carolina has required that public school teachers be evaluated using the Teacher Performance Appraisal Instrument–Revised (TPAI-R). Principals rate teachers on eight overall domains based on evidence collected through classroom observation, teaching artifacts, and discussions. The domains covered by the instrument include management of instructional time, management of student behavior, instructional presentation, instructional monitoring, instructional feedback, facilitating instruction, communicating within the education environment, and performing non-instructional duties. Domain scores are assigned using a four-point scale ranging from “Unsatisfactory” to “Above Standard.”

All probationary teachers and teachers with below-standard performance are evaluated on an annual basis, and tenured teachers are evaluated at least once every five years or at the recommendation of an administrator. In order to maximize the number of teachers with an evaluation score in my analytic sample, I construct a rolling average of all available evaluation scores for each teacher in each year. I first calculate year-specific evaluation scores by assigning values of 1 (“Unsatisfactory”) through 4 (“Above Standard”) to the four rating categories and averaging scores across the eight domains of the TPAI-R rubric. I then standardize scores in each year to be mean zero with unit variance such that a one unit difference can be interpreted as moving one standard deviation higher in the distribution of teacher effectiveness. Finally, I average all available scores in my panel of data up to and including a given year. I use these rolling average scores in all analyses unless otherwise noted.

I estimate teacher effects by attempting to isolate a teacher's value added to her students’ academic achievement on standardized tests for mathematics and English teachers in fourth through eighth grades—the grades and subjects in which the necessary baseline and outcome testing data are available. A large body of literature has examined the consequences of different value-added modeling approaches which attempt to account for the nonrandom sorting of students to teachers (Todd and Wolpin 2003; McCaffrey et al. 2004; Harris and Sass 2006; Kane and Staiger 2008; Koedel and Betts 2011; Chetty, Friedman, and Rockoff 2014; Guarino, Reckase, and Wooldridge 2015). I adopt the widely used covariate-adjustment model from the education production function literature as my preferred specification. My model controls flexibly for prior student achievement as well as a variety of student, classroom, and school characteristics (see Appendix  B for full description).

I isolate the permanent teacher effect separately from idiosyncratic class-year shocks by estimating effects across multiple years following the empirical Bayes approach described by Kane and Staiger (2008). Using this approach, I estimate a rolling average value-added score for each teacher in each year in mathematics and in reading by using all available data up to and including a given year. I use these rolling value-added estimates unless otherwise noted. I present value-added scores in test-score standard deviation units in all of my descriptive statistics, but restandardize these measures in a teacher-year-level data set for use in my regression analyses. This allows for a more meaningful comparison of the coefficients associated with value-added scores and principal evaluation scores.

Here, the parameters of interest are the coefficients in vector, ⁠, that capture the log odds that teachers with a given RIF criteria were selected for layoffs. I present these coefficients in subsequent tables as average marginal effects calculated across the analytic sample in order to facilitate interpretation.

I estimate the differential effect of reductions in force in 2008–09 on student achievement in 2009–10 across three measures central to the debate on layoff policy: seniority, principal evaluations scores, and value-added scores. I use a six-year student-level panel data set (2004–05 to 2009–10) to more accurately account for potential confounding trends in achievement over time between schools and within schools across grades.¹¹ I restrict these data to include only students who can be linked to their mathematics or English teacher in fourth through eighth grades, the grades for which both current and prior standardized test scores are available.¹² Research frequently finds that teachers have smaller effects on students’ achievement on standardized reading tests (Hanushek and Rivkin 2010) as is the case in the present analysis. Thus, I focus on student achievement in mathematics and present parallel results for reading in Appendix table C.1.

In order to isolate plausibly exogenous variation in layoffs, I focus my analyses on the differential impact of layoffs across grades within schools. My identification strategy approximates a natural experiment by comparing, within a school, the performance of students who entered a grade in which a teacher was laid off to the performance of students who entered a grade that did not experience a layoff. Layoffs were not chosen at random in CMS. Analyses in the following sections show that a teacher's performance and a variety of other characteristics are associated with being selected for layoffs. Within a school, however, the selection of a teacher in a given grade for layoffs was arguably unrelated to the achievement of the rising cohort of students in the grade below. I test for potential violations of this assumption by comparing the prior academic achievement of students who entered grades in which a teacher was laid off to the prior achievement of students who entered grades that did not lose a teacher due to layoffs. Results presented herein demonstrate that there is no evidence that principals selected teachers for layoffs based on the achievement of rising cohorts of students.

This approach provides a credible and policy-relevant estimate of the average effect of layoffs in CMS in affected grades within schools. More specifically, this estimate captures the net effect of multiple features that contribute to a layoff “treatment” including: (1) the loss of the effect of the laid-off teacher on the rising cohort of students; (2) the absence of a laid-off teacher's positive (or negative) peer effects on their grade-level colleagues; and (3) the efforts of principals to mitigate the grade-specific effects of layoffs by reassigning personnel, reallocating resources, adjusting class sizes, or hiring new teachers. My within-school grade-specific identification strategy, however, does not capture any schoolwide or district-wide effects of layoffs. For example, the loss of a teacher could affect the morale of the entire staff; a laid-off teacher might have influenced colleagues and students across her building; or principals might take steps to buffer their entire school from the effect of layoffs. These schoolwide impacts will not be reflected in my estimates. Further, the loss of key instructional support staff and student support personnel across both years likely had negative consequences for instructional quality and student achievement across the district.¹³ I adopt this narrow approach, which focuses on credibly exogenous variation, because comparing the relative effects of layoffs across teachers with different levels of seniority and performance is of first-order importance, while establishing the full effect of layoffs is secondary.

I implement this approach by constructing a dichotomous indicator for whether any classroom teachers were laid off in each grade of a school in the previous year, LAYOFF.¹⁴ I create corresponding measures of the average seniority, principal evaluation scores, and value-added scores of laid-off teachers in the previous school-year-grade. I calculate average seniority and principal evaluation scores of laid-off teachers using all data through the year in which they were laid off, 2008–09. I calculate subject-specific average value-added scores using all data through the year prior to the layoffs, 2007–08, to guard against the potential for correlated errors among individual students’ test scores over time that could bias my estimates.¹⁵ Furthermore, the current year standardized achievement results needed to calculate value-added scores are rarely, if ever, available to districts before they must notify teachers of layoffs. I include the average value-added scores in the same subject as the achievement outcome. I represent these three measures generically as C.¹⁶

Here, the outcome of interest, ⁠, is the standardized scaled score on a state end-of-grade test in a given subject for student i, in grade g, with teacher j, in school s, in year t. I include grade-specific cubic functions of students’ prior-year achievement, ⁠, in both mathematics and reading, as well as vectors of controls for observable student characteristics ⁠, the characteristics of a student's peers with the same teacher (⁠⁠), and grade-by-year fixed effects (⁠⁠).¹⁷ The inclusion of school-by-year fixed effects (⁠⁠) accounts for any schoolwide year-specific shocks to student achievement, such as the turnover of a principal or the introduction of a new curriculum, by restricting my comparison to students within the same school in the same year. The inclusion of school-by-grade fixed effects (⁠⁠) removes any permanent differences in average teacher effectiveness across grades within a school that could bias my estimates. Such differences might arise if, for example, less effective teachers were systematically assigned to teach lower grades. I estimate standard errors clustered at the teacher-level to account for the potential of correlated errors among students taught by the same teacher.

⁠, the coefficient associated with a given C, captures the estimated differential effect of laying off a teacher with one year more seniority (or one standard deviation higher evaluation or value-added score) in year t–1 on the academic achievement of students in year t in the grade and school in which the laid-off teacher taught.¹⁸ Interpreting the coefficients associated with these interaction terms as causal estimates imposes two important additional assumptions. First, principals’ ability to mitigate the effect of layoffs must be uncorrelated with the seniority and effectiveness of laid-off teachers. For example, if principals in schools where early-career teachers were laid off were more effective at buffering students from the consequences of layoffs compared to principals in schools where more senior teachers were laid off, my estimates would exaggerate any negative differential effect of laying off more senior teachers. Second, because my estimation strategy does not capture schoolwide effects of layoffs, these estimates may understate or overstate the total marginal effect of layoffs based on measures of seniority or effectiveness. If seniority or measures of teacher effectiveness are differentially predictive of the schoolwide effects of layoffs, it is possible that the relative magnitudes of the grade-specific marginal effects I estimate would be different from the total marginal effects.

In section 7, I test the robustness of my primary specifications, examine their identifying assumptions, and extend my analyses in several ways. I account for the potential threat of student sorting by adding student fixed effects. I also examine the effects of subject-specific layoffs and explore two potential mechanisms, increases in class-size and grade-specific turnover.

In table 1, I present averages of teacher characteristics across RIFed teachers and non-RIFed teachers in 2009 and 2010 for my full analytic sample, for a sample that excludes returning retired teachers, and for tenured teachers. These results show that the teachers selected for layoffs were likely to be nontenured teachers, returning retired teachers, teachers hired after the start of the school year, teachers with a licensure deficiency, and low-performing teachers. Over 84 percent of laid-off teachers in 2009 and 2010 were probationary teachers despite the flexibility afforded to the district by North Carolina's ban on collective bargaining. In follow-up interviews, administrators explained that the additional requirements of laying off a tenured teacher caused them to focus first on probationary teachers for whom they could simply not renew their contracts. They saw terminating the job of tenured teachers who met performance requirements as counterproductive because state law guarantees laid-off tenured teachers first rights to accept any open position for which they qualify, up to three years after being laid off.

Figure 1 illustrates how layoffs were heavily concentrated among nontenured teachers and, in 2009, teachers with thirty or more years of seniority. The increase in the probability of being laid off among teachers with thirty or more years of experience is driven by the nonrenewal of returning retired teachers who constituted over 21 percent of all RIFed teachers in 2009.¹⁹ These teachers who were collecting both a salary and a pension were among the very first the district targeted for layoffs (see Appendix  A), despite the fact that they were substantially more effective than the average CMS teacher. Their average evaluation scores were two thirds of a standard deviation higher than the district average and their average value-added scores were 0.048 standard deviations (SD) and 0.034 SD higher in mathematics and reading, respectively.

Overall, RIFed teachers in 2009 were rated 0.38 SD lower by principals (approximately one fifth of a point on a 4-point scale). These same teachers had slightly lower value-added scores in mathematics and reading compared with non-RIFed teachers. When returning retired teachers are excluded from these estimates (table 1, panel B), or the sample is restricted to tenured teachers (table 1, panel C), these differences become even greater. RIFed teachers who were not returning retired teachers had value-added scores that were, on average, 0.029 SD lower in mathematics and 0.017 SD lower in reading. In 2010, the differences in effectiveness between RIFed and non-RIFed teachers increased even further. RIFed teachers in 2010 received principal evaluation scores that were 0.93 SD lower than non-RIFed teachers. I also find that RIFed teachers in 2010 had significantly lower value-added scores, on average, than non-RIFed teachers (−0.068 SD lower in mathematics and −0.015 SD lower in reading).

Examining the distribution of unstandardized one-year principal evaluation scores for non-RIFed, 2009 RIFed, and 2010 RIFed teachers provides further insights into the layoff process. Figure 2 suggests that evaluation scores were used to directly inform the layoff process. Only 4 percent of teachers with an average rating of “At Standard” or above (an average of 3 on the 4-point scale) were laid off, whereas 58 percent of all teachers whose average evaluation scores were below “At Standard” were laid off.²⁰ In contrast, the continuous distributions of value-added scores in figure 3 illustrate that although RIFed teachers, particularly 2010 RIFed teachers, had lower value-added scores on average, the layoff selection process did not operate directly through these scores. The differences in the average value-added scores of RIFed and non-RIFed teachers are not surprising given the weak-to-modest positive correlations between principal evaluation and value-added scores in my data. Similar to previous studies, I find that principal evaluation scores are correlated 0.27 and 0.19 with math and reading value-added scores, respectively (Jacob and Lefgren 2008; Harris and Sass 2009).²¹

Overall, teacher layoffs were widely distributed among the 165 K–12 schools in CMS. Across both years, 64 percent of schools laid off between 1 percent and 10 percent of their teaching staff, and another 16 percent of schools laid off more than 10 percent but less than 20 percent of their classroom teachers. Only nine small elementary schools and one small high school avoided layoffs for classroom teachers in both years. The highest concentration of staff layoffs was 29 percent. In table 2, I present the probability that a teacher was laid off across school levels and by quartiles of school characteristics. In 2009, 9.6 percent of high school teachers were laid off, and 7.4 percent lost their jobs in 2010. High school teachers were approximately twice as likely to be laid off as elementary school teachers, and 40 percent more likely to be laid off than middle school teachers.

Teachers working in schools that served larger proportions of students who were African American, who scored lower on achievement tests, and who attended school less frequently were more likely to lose their jobs. Teachers in schools in the top quartile of African American student enrollment were approximately 3 percentage points more likely to be laid off compared with teachers at schools in the bottom quartile. Teachers in schools where students’ scores were in the top quartile of average achievement in both mathematics and reading were also approximately 3 percentage points less likely to be laid off compared with teachers in schools in the bottom quartile. There were no consistent differences in the distribution of layoffs across schools by their state performance rating or by the proportion of students who were Hispanic, limited English proficient, or receiving special education services.²²

I fit a series of logistic regression models to better understand the relative importance CMS administrators and principals placed on different school board–approved RIF criteria. In table 3, I begin with a specification that includes indicators for each year a teacher has probationary status within CMS (column 1). Notably, being a novice teacher was associated with a 22 percentage point higher probability of being laid off as compared with tenured teachers, but drops precipitously to between 2.2 and 6.3 percentage points for teachers with between one and three years of seniority. When I include other school board–approved RIF criteria in column 2, the relative probability a novice teacher was laid off is reduced by 13.5 percentage points, even without accounting for principal evaluation scores. Comparing these conditional average marginal probabilities that a CMS teacher was laid off given her seniority to estimates from Washington State suggests that districts in North Carolina had more flexibility to lay off teachers based on criteria other than seniority. For Washington State teachers, the marginal effect of having between zero and three years of seniority ranges from 9.2 to 11.6 percentage points (Goldhaber and Theobald 2013; table 4, p. 512), whereas corresponding marginal effect estimates for CMS in column 2 of table 3 are half as large. Other criteria used to identify teachers for layoffs, such as if a teacher was a returning retired teacher or was hired after the start of the school year, are much stronger predictors than seniority. These indicators are associated with 89 and 40 percentage point increases in the probability of layoffs, respectively.

Results from these analyses are consistent with those from a sample of teachers for whom evaluation scores are available. When all RIF criteria are added to the model (column 5), I estimate that being evaluated as one standard deviation lower by a principal is associated with a 4 percentage point increase in the probability of being laid off. Nevertheless, this average marginal effect masks the differential relationship between teachers’ evaluation scores and the probability of layoffs. In figure 4, I plot the predicted probability of layoffs across the sample distribution of standardized rolling evaluation scores for teachers new to the district, teachers with three years of seniority (i.e., teachers currently in their fourth year of teaching who would receive tenure if they are rehired), and for tenured teachers. This figure illustrates two key findings. First, the probability of being laid off increases precipitously in all three plots as evaluation scores drop below two standard deviations, suggesting that the district targeted the very lowest performing teachers across all levels of seniority. Second, lower-performing tenured teachers were more likely to be laid off than higher-performing untenured teachers. For example, the probability that a tenured teacher with an evaluation score of −1 SD was laid off was 4.4 percentage points, which is greater than the corresponding probability for newly hired teachers with at least an average evaluation score (probability <3.5 percentage points).

I extend these analyses by refitting the full model in column 5 and substituting in each of the eight evaluation domains on the TPAI-R rubric. In Appendix table C.2, I find that Management of Instructional Behavior, Management of Instructional Time, and Instructional Presentation are the strongest predictors of the probability of being laid off. Further analyses also suggest that the higher rates of layoffs in schools with more African American and lower-achieving students are driven by teacher sorting across schools. Conditioning on the full set of RIF criteria, the relationships between these individual school characteristics and the probability of being laid off, as seen in table 2, reverse, but are of extremely small magnitude (less than a 0.7 percentage point difference per SD change in a given school characteristic).

These descriptive analyses also reveal that among all teachers, foreign language and arts teachers were the most likely to be laid off. In table 4, I present estimates of the relationship between licensure type and the probability of being laid off relative to teachers with elementary licensures. Column 1 shows how foreign language, mathematics, and arts teachers were all significantly more likely to be laid off. When I include controls for the full set of RIF criteria both with and without evaluation scores, foreign language and arts teachers have the highest probabilities of being laid off. The reduction in the relative probability of being laid off for mathematics teachers suggests that many of these teachers were laid off based on other RIF criteria. Conditional on all RIF criteria, including evaluation scores, foreign language and arts teachers had approximately a 3 percentage point greater probability of being laid off compared with teachers with elementary licensures.

I examine the moderating effect of the characteristics of laid-off teachers in 2008–09 on mathematics achievement in the following year by isolating plausibly exogenous variation in layoffs within a school across grade levels. I formally test the core assumption of my modeling approach—that principals did not choose specific grades for layoffs based on the relative achievement of the rising cohorts of students in a school—by fitting a modified version of model 2. I regress prior test scores of the rising cohorts of students on LAYOFF and my full set of two-way fixed effects. In table 5, I present estimates of the coefficient associated with LAYOFF that are both near zero and statistically insignificant. These estimates provide little evidence that principals selected layoffs based on the achievement of rising cohorts of students.²³ Parallel tests based on student demographic characteristics produce estimates equal to or less than 0.023, none of which is statistically significant.

In table 6, I fit a series of models to estimate the direct effect of layoffs in CMS, and to compare the moderating effects of three teacher characteristics on the relationship between layoffs and student achievement in the following year.²⁴ In my baseline model, I find a small negative and statistically insignificant relationship between layoffs and student achievement. I then expand the model to examine whether this average effect masks important variation in the relationship between layoffs and student achievement related to the characteristics of laid-off teachers. In columns 2 through 4, I present estimates of the differential effect of layoffs based on the average seniority, principal evaluation scores, and value-added scores of laid-off teachers in the previous school-grade-year as described earlier.

I find that seniority has limited predictive power on the effect of teacher layoffs. The estimated coefficient associated with laying off a teacher with one more year of experience is near zero and not statistically significant (−0.002 SD, p = .242). Even if we ignored the lack of statistical significance and scaled the point estimate to compare laying off a teacher with seven years of seniority versus a novice teacher (a one standard deviation difference), the magnitude would only increase to −0.014 SD. Supplemental analyses where seniority is specified as a binary variable produce similar results. In contrast, laying off a teacher one standard deviation higher in the distribution of teacher effectiveness as judged by principals or measured by value-added scores lowered student achievement by 0.034 SD (p = .050) and 0.083 SD (p = .017), respectively. Thus, the differential effect on student achievement between laying off a teacher ranked at the 75th percentile of the distribution of evaluation scores and one at the 25th percentile is 0.046 SD. The corresponding difference for value-added scores is 0.112 SD. These estimates show that the effect of reductions in force depends primarily on the effectiveness of those teachers selected for layoffs.

Next, I fit models that allow for the effect of teacher layoffs on future student achievement to differ by both seniority and measures of teacher effectiveness simultaneously. Results presented in columns 5–7 illustrate that seniority contains little information about the future effect of laying off a teacher that is not captured by performance measures. When I include principal evaluation scores or value-added scores in the model, the relationship between the seniority of laid-off teachers and mathematics achievement is even further reduced and switches signs. In contrast, the coefficients associated with measures of principal evaluation scores and value-added score remain nearly identical. In my full model that includes all three teacher characteristics, I find that both principal evaluation and value-added scores maintain their predictive power, conditional on the other, suggesting these measures are capturing somewhat different aspects of teacher effectiveness. Estimates from models that use evaluation scores or value-added measures constructed from the most recent three years of data produce nearly identical results (see Appendix table C.3).

Finally, these results suggest that laying off a low-performing teacher raised student achievement the following year in the grade taught relative to other grades. Using estimates from my full model in column 7, I find that the linear combination of the main effect of LAYOFF and the marginal effect of laying off a teacher who received an evaluation score of −0.5 SD or lower is positive, holding all else constant. The same linear combination for value-added scores becomes positive when a teacher has a score of −0.15 teacher-level SD or lower. It is possible, however, that negative schoolwide effects of layoffs not captured by these estimates could partially offset or dominate these positive within-school, across-grade effects.

One potential threat to identifying credible estimates is the possibility that students and their families responded strategically to layoffs by switching schools. Model 2 addresses this threat by including sets of controls for student and peer characteristics. Nevertheless, student sorting on unobserved characteristics related to layoffs in the previous year could bias my results. I examine this potential threat by testing the sensitivity of my estimates to the inclusion of student fixed effects in place of prior achievement scores and student characteristics. This approach further restricts my estimates to within-student differences in achievement across time, greatly limiting the endogenous student selection process as a threat. Thus, these estimates are identified using only a subsample of students with two or more observations. This effectively removes 13 percent of the student-year observations in the analytic sample and over 27 percent of the student observations in 2010.²⁵

Estimates from models that include student fixed effects are consistent with the previous results although somewhat less precise given the limited remaining variation for identifying parameters. As shown in table 7, the most notable difference is the increase in the estimated average effect of layoffs on student achievement in math, which becomes −0.071 (p = .031). Further analyses suggest this difference is largely driven by the sample restriction imposed by including student fixed effects. An estimate from my preferred specification in model 2 using a restricted sample of students with at least two years of data is nearly as large (–0.047, p = .210), but is not statistically significant.²⁶ This suggests that student sorting on unobserved characteristics may mask a negative main effect of layoffs that operates independently of any compositional changes in teacher quality.

Estimates of the differential effect of layoffs based on seniority, evaluation scores, and value-added scores are practically unchanged. The coefficient associated with average evaluation scores in column 3 becomes slightly more negative (–0.037, p = .115) but is no longer statistically significant. The marginal effect of layoffs across value-added scores also becomes slightly more negative (–0.113, p = .077) and remains significant at the 10 percent level. Estimates from the full model in column 7 continue to show that measures of teacher effectiveness dominate seniority as a predictor of the effect of layoffs.

Here I explore the direction, and degree to which, potential violations of the identifying assumptions for estimates of the marginal effect of layoffs across teacher characteristics could bias my estimates. Estimating the correlation between principals’ ability to mitigate the effect of layoffs and teachers’ seniority or effectiveness is largely intractable because this ability of principals is unobserved. One plausible violation would be a negative correlation where principals who were more effective at mitigating layoff impacts were also more successful at identifying and securing the layoff of low-performing teachers, whereas principals who were less effective at mitigating the impacts of layoffs had teachers laid off who were relatively higher performing. This scenario would bias my estimates toward zero. It seems likely, however, that the layoff selection process in CMS, where central office officials, not principals, made the final layoff decisions, would prevent any strong systematic relationship between principals’ ability to mitigate layoff impacts and teachers’ characteristics. Furthermore, teacher sorting does not appear to have constrained the option to lay off low-performing or novice teachers in some schools but not others, potentially inducing a mechanical correlation between principal ability and laid-off teachers’ characteristics. Analyses of variance show that over 84 percent of the variation in seniority, evaluation ratings, and value-added scores exists within schools.

Next, I consider whether seniority or measures of teacher effectiveness are differentially predictive of the schoolwide effects of layoff. My previous estimates suggest that measures of effectiveness are much stronger predictors of the grade-specific effects of layoffs than seniority. One possibility is that schoolwide peer effects follow the same pattern as grade-level peer effects. Jackson and Bruegmann (2009) provide direct evidence of the relative magnitude of grade-level peer effects based on measures of seniority and value added. They found that when the average seniority and value added of a teacher's grade-level peers are both included in a model, only value added is a strong and significant predictor of student achievement. These findings mirror my results from model 2 when both measures are included (table 6, column 6). If schoolwide peer effects follow this same pattern, then my results on the total effect of layoffs will understate the relative difference in the predicative power of teacher effectiveness over seniority.

I explore this assumption in my own data by estimating the relationship between layoffs and schoolwide achievement using a difference-in-differences design. To implement this approach, I modify model 2 by replacing grade-specific predictors for LAYOFF and the average characteristics of laid-off teachers, C, with corresponding schoolwide measures. I also replace school-by-grade and school-by-year fixed effects with school fixed effects in order to contrast the average achievement in a school prior to layoffs with achievement in 2010, the first year after layoffs. The coefficients associated with C capture the conditional correlation between first-difference estimates of schoolwide layoff effects and the seniority or effectiveness of laid-off teachers. I find no evidence for differential schoolwide effects of layoffs based on the seniority or value-added scores of laid-off teachers. However, I estimate that laying off a teacher that is rated 1 SD higher by her principal is associated with a −0.026 SD (p = .000) difference in student achievement. This makes sense given that principals can at least partially observe teachers’ schoolwide peer effects and the TPAI-R evaluation rubric explicitly incorporates teachers’ performance outside of their classroom, such as in dimension 7.2: “Teacher participates in the development of a broad vision of the school.” These findings suggest that my within-school across-grade modeling approach may underestimate the marginal negative effect of laying off a teacher with a high evaluation score relative to a teacher with a low score.

The primary analyses presented here pool layoffs across all classroom teachers and examine their effect on students’ achievement in mathematics. This broad approach helps to increase the precision of my estimates and reflects the joint production process of education, although it obscures any potential differences of layoffs based on the subject taught by teachers. We might expect the loss of an effective math teacher to have a larger impact on student achievement in mathematics than the loss of an effective English teacher. I explore this possibility by fitting models that compare the effect on mathematics achievement of laying off a math teacher compared with laying off only non-math teachers in the prior school-grade-year.

I find suggestive evidence that laying off math teachers has a larger effect on mathematics achievement than laying off non-math teachers. As shown in table 8 column 1, the coefficient associated with math teacher layoffs is over 3.5 times that of nonmath teacher layoffs, although these estimates are not statistically significantly different from zero or each other. I extend this model by allowing the differential effect of seniority and performance to vary across math and nonmath teacher layoffs. The results suggest that teacher characteristics matter more for math teachers than nonmath teachers when examining their effect on mathematics achievement. Across all three measures, the characteristics of laid-off math teachers are significant predictors, whereas those of nonmath teachers are not. This contrast is particularly large for value-added scores where the coefficient associated with math value-added scores is −0.077, and the coefficient associated with reading value added is −0.006. However, I am unable to reject the null of equivalence between math and non-math teachers for the pairs of coefficients associated with these characteristics.

Increases in class size are one primary mechanism through which layoffs could have affected student achievement in the following year. Across the district, average class sizes increased by less than one student across all grades from 2008–09 to 2009–10.²⁷ I estimate whether grade-specific layoffs in a given school year caused larger increases in class size in the affected grade by re-estimating model 2 using average class size as the outcome. As shown in table 9, I find no statistically significant relationship between layoffs in a given school-grade and average class size in the same school-grade the following year, and am able to reject differential changes as small as one student, on average.

A second possible mechanism could be the degree to which teachers were reassigned to different grade levels as a result of layoffs. Research has documented that grade-specific experience constitutes a large portion of the total returns to experience that teachers accrue over their careers (Ost 2014). I examine this potential mechanism by calculating the proportion of teachers in a given school-grade-year who taught in that same school and grade in the previous year. Not surprisingly, I find the rate of teachers returning to the same grade and school jumped from 52 percent in 2008–09 to 68 percent in 2009–10. This large increase likely reflects the rapid decrease in alternative labor market opportunities for teachers during the recession. I then examine whether there was any differential degree of grade-switching across grades within a school by refitting model 2 using this school and grade specific measure of retention. Again, I find no statistically significant effect, although I cannot rule out differential effects as large as a 4 percentage point increase in grade-specific turnover. Together, these exploratory analyses suggest that the district was able to distribute any effect of the layoffs on class-size and grade-switching evenly across grades within a school.

Comparing the characteristics of laid-off teachers selected under alternative layoff policies highlights the differential consequences these policies can have for district budgets and teacher effectiveness. In table 10, I contrast the average characteristics of teachers laid off in CMS in 2009 and 2010 to the characteristics of teachers who would have been selected under six alternative policies: inverse seniority, inverse evaluation scores (1-year and average), inverse value-added scores²⁸ (1-year and average), and inverse composite performance scores.²⁹ I obtain these estimates by replicating the within-school selection process used by CMS where the number of layoffs per school was predetermined by the district based on enrollment projections and current staffing levels. Thus, I maintain the same number of teachers for layoffs within each school as the actual policy (as well as the same total number), but change the decision rule that determines which teacher(s) in a school were selected. I also conduct simulations of a district-wide selection process holding constant the total reduction in salary as opposed to the total number of layoffs. Selecting teachers across the district for layoffs allows me to estimate the number of layoffs needed under each alternative selection rule to reach an equivalent level of savings achieved by the actual layoffs.

As shown in panel A of table 10, discretionary teacher layoffs in 2009 reduced the budget of CMS by almost $30.5 million in annual salaries, over $5 million more than a comparable inverse seniority policy.³⁰ It would have required an additional 136 teacher layoffs to achieve an equivalent reduction in salary expenditures under a seniority-based policy because of the high savings the district realized by targeting returning retired teachers. In fact, layoffs under each of the performance-based policies would also result in less total savings than the district realized in 2009. An inverse seniority policy in 2010 would have again required more layoffs (27) to achieve the same payroll savings, whereas inverse performance policies would have required at least 26 fewer layoffs to reach the same level of savings.

In 2009, it appears as though the district selection process produced teachers who were, on average, no more or less effective than those who would have been selected under an inverse-seniority policy. However, these 2009 averages mask the process whereby the district almost uniformly laid off all returning retired teachers who were collecting both salaries and pension benefits. In the second row of panel A, I present the average characteristics of laid-off teachers excluding these 140 returning retired teachers. Excluding these veteran teachers reveals how the district selected teachers for layoffs who were, on average, less effective than those identified under an inverse-seniority policy. Non-retired laid-off teachers in 2009 received evaluation scores that were 0.3 SD lower, on average, than teachers who would have been laid off under an inverse-seniority policy. In 2010, CMS selected teachers for layoffs who were rated, on average, 0.17 SD lower by their principals and had 0.045 SD lower value-added scores in mathematics than those teachers who would have been laid off under an inverse-seniority policy.

Results in table 10 also show that all of the inverse performance-based policies would have resulted in laying off substantially less-effective teachers, on average, than an inverse-seniority layoff procedure. Across both years, inverse value-added policies would have resulted in the lowest average value-added scores among laid-off teachers but higher average principal evaluation scores than under either the discretionary policy or an inverse-seniority process. These results suggest that a layoff policy based exclusively on objective measures of teacher effectiveness would fail to consider all aspects of teacher performance that principals value. These simulations also show how a policy that considers both evaluation and value-added scores results in the selection of teachers with nearly as low value-added scores as a pure inverse-value-added score policy, but substantially lower evaluation scores. Of course, the value of subjective performance measures depends on principals’ capacity to assess teachers’ effectiveness and willingness to differentiate among them on formal evaluations, both of which may vary across districts.

In 2009, and again in 2010, Charlotte Mecklenburg Schools laid off hundreds of teachers following sharp cuts in local and state funding for public education. CMS implemented layoffs with broad discretion afforded by North Carolina's ban on collective bargaining. In addition to concentrating layoffs among nontenured teachers, returning retired teachers, late-hired teachers, and teachers with licensure deficiencies, the district also targeted underperforming teachers. Fifty-eight percent of all teachers who received a “Below Standard” or “Unsatisfactory” rating on any of the eight evaluation rubric domains were laid off.

Although these findings are specific to one district, they provide the first evidence of a district's revealed preferences when implementing discretionary layoffs at scale. CMS principals and administrators appear to have considered multiple teacher characteristics rather than defaulting to an inverse-seniority process or targeting the highest paid teachers as some have claimed would happen. This example also provides an additional case study for examining trends in layoffs across districts. Similar to districts in Washington State, CMS schools were most likely to lay off teachers who taught electives, such as foreign language and the arts (Goldhaber and Theobald 2013).

Layoffs in CMS also provide a compelling policy context in which to examine the validity of objective and subjective measures of teacher effectiveness. I find evidence that laying off a more effective teacher, as measured by either subjective or objective performance metrics, decreased mathematics achievement in the following year compared with laying off an ineffective teacher. In contrast, laying off a more senior teacher resulted in at most a substantially smaller decrease in achievement when compared with laying off an early-career teacher. When compared simultaneously, measures of teacher effectiveness strictly dominate seniority as predictors of the effect of teacher layoffs on future achievement. Teacher experience matters; experienced teachers are, on average, more effective than their early-career colleagues and make important contributions to their schools outside of the classroom. However, performance measures are better indicators of the likely effect of layoffs on achievement compared to seniority.

Simulation analyses illustrate that the district was able to use its discretion to lay off less-effective teachers in both 2009 and 2010 compared to an inverse-seniority policy. An inverse layoff policy using a combination of principal evaluations and value-added scores would have lowered the effectiveness of laid-off teachers by even more in each year. Rockoff et al. (2012) found that principals randomly assigned to receive value-added scores for their teachers were able to use these data to improve their personnel decisions, which led to small improvements in teacher effectiveness. These results, combined with the findings herein, suggest that CMS could have further reduced the negative effect of layoffs had they also used value-added scores to inform the layoff process.

Laying off teachers based on their seniority in the district, rather than their performance in the classroom, results in greater job losses and exacerbates the negative effects of layoffs on student achievement. Layoff policies that do not incorporate increasingly available measures of teacher effectiveness fail to consider all the best available information when making high-stakes decisions. However, exchanging one inflexible inverse layoff criterion for another will not provide districts with any discretion in navigating a complex process aimed at preventing a variety of negative consequences. Going forward, it will be important to examine the long-run effects of layoffs and to obtain a more detailed understanding of an optimal selection process should districts and unions decide that layoffs are unavoidable.

I thank Andrew Baxter, Thomas Tomberlin, Kim Brazzell, and the Charlotte-Mecklenburg Schools for their support of this research. I am grateful to Thomas Bailey, Sarah Cohodes, David Deming, Susan Moore Johnson, Lawrence Katz, Richard Murnane, Jonah Rockoff, and Martin West for their helpful comments on earlier drafts of this manuscript.

1. The district estimates new Full Time Equivalent (FTE) allocations for schools using projected Average Daily Membership (ADM) and revised student-teacher ratio policies.

2. Principals identify position categories to cut if their current teaching staff exceeds their allotted FTE for the next academic year.

3. Human resource officials select which teacher(s) among those in the position(s) identified for elimination will be placed in the district-wide RIF pool in the following order:

   1. Non-Career Teachers

      • Any teacher who received an evaluation of below standard or unsatisfactory on the TPAI-R evaluation instrument

      • Any teachers with a licensure deficiency

      • Any teacher who is currently collecting pension benefits (i.e., returning retired teachers)

      • Any Part-time or Interim teacher

      • Any teacher on an End-of-Year contract

      • Seniority in the district

   2. Career Teachers

      • Based on comparative performance and comments in two most recent TPAI-R evaluations.

   3. All teachers in the RIF pool are informed of the district's intent to lay them off by 15 May.

4. Principals with more FTE positions than current staff members must select among teachers in the district-wide RIF pool who are qualified for any open positions. Any career teacher must be selected first before noncareer teachers can be considered.

5. Any teachers who remain in the district-wide RIF pool after the hiring process receive confirmation that they are laid off by 5 June.

Source: CMS memos entitled “Recommended Criteria for Reductions in Pay of Assistant Principals,” and “Reductions in Force of Classroom Teachers and Other Certified Employees Paid on the Teacher Pay Scale.”

where the outcome of interest, ⁠, is the end-of-year test score for student i in grade g, with teacher j in school s in year t. The outcome test score is modeled as a grade-specific cubic function of the student's prior year achievement, ⁠, in both mathematics and reading, vectors of controls for observable student characteristics ⁠, the characteristics of a student's peers with the same teacher (⁠⁠), and school characteristics (⁠⁠), as well as grade-by-year fixed effects (⁠⁠). I include indicators for the student's gender, race, limited English proficiency status, and special education status. For peer, and school-level characteristics, I include the means of all of these predictors as well as mean prior year achievement in mathematics and reading. I restrict the sample to exclude any teacher-year in which fewer than five students had valid test scores. I exclude any class with more than 90 percent of students requiring special educational services. Notably, I omit school fixed effects because I am interested in comparing the relative effectiveness of teachers across schools in the district.

I isolate teachers’ persistent effects following Kane and Staiger (2008). I accomplish this by predicting individual teacher random effects (⁠⁠) while also including random effects for teacher-years (⁠⁠) to account for any transitory class-specific shocks that would otherwise be attributed to the teacher. This approach rescales teacher effects by the reliability of these individual estimates, where reliability is the ratio of estimates of true teacher variance, over the sum of estimated true teacher variance, transitory teacher variance, and random variance in student scores (see Chetty, Friedman, and Rockoff 2014 for a full discussion).