Abstract

The No Child Left Behind Act of 2001 required states to set cutoffs to determine which schools were subject to accountability for their racial/ethnic subgroups. Using a regression discontinuity design and data from North Carolina, this study examines the effects of this policy on teacher turnover and attrition. Subgroup-specific accountability had no overall effects on teacher turnover or attrition, but the policy caused black teachers who taught in schools that were held accountable for the black student subgroup to leave teaching at significantly lower rates, compared with black teachers who taught in schools not accountable for the black subgroup's performance. The policy also caused shifts in the students assigned to black teachers, with schools that were held accountable for the black subgroup less likely to assign black students to black teachers the following year. These findings demonstrate that subgroup-focused policies—particularly those that use cutoffs to determine subgroup accountability—can shape the composition of the teacher labor force in unintended ways, and have implications for the design of future accountability systems that aim to close racial/ethnic gaps in achievement.

1.  Introduction

Significant gaps persist between the academic achievement of students of various races and socioeconomic levels (Reardon and Robinson 2008). In recent decades, policies have attempted to close these achievement gaps in a variety of ways, including by implementing school desegregation orders, encouraging students and their families to move to different neighborhoods or schools, and instituting promotion or graduation exams (Harris 2008). Despite these interventions, these achievement gaps have remained sizeable, and perhaps have even widened in recent years (Reardon and Robinson 2008; Reardon 2011).

The No Child Left Behind Act of 2001 (NCLB) attempted to close these achievement gaps in a novel way: by requiring schools to make yearly improvements not only in overall student achievement but also in the achievement of students of various subgroups, including racial/ethnic minority subgroups. NCLB required that students in each of these subgroups in each school make yearly gains in achievement (what the law termed Adequate Yearly Progress, or AYP), or the entire school would be labeled as failing. This “subgroup-specific accountability” had major consequences for schools across the nation, as subgroups’ passage or failure was a major contributor to whether schools passed or failed the accountability targets set by NCLB, and thus the degree of sanction they faced under the law (Reback, Rockoff, and Schwartz 2014; Davidson et al. 2015).

Recent research has found mixed impacts of subgroup-specific accountability on student achievement. Whereas some work finds the policy had the intended effects of raising minority student achievement (Lauen and Gaddis 2012), and of narrowing at least some racial/ethnic achievement gaps (Reardon et al. 2013), other work finds that the policy increased schools’ failure rates and lowered schools’ overall performance and the performance of minority students in particular (Sims 2013). Despite these advances, however, we know little about the policy's effects on teachers, which is important to gain a more complete picture of the policy's overall impact. If the policy increased student achievement but resulted in the loss of many teachers, for example, these benefits and costs should be weighed together when assessing the policy's overall effects.

There are a variety of reasons to believe that subgroup-specific accountability might have affected teachers. State-level accountability policy affects teachers’ decisions about whether to remain in or leave their schools or teaching (Clotfelter et al. 2004; Boyd et al. 2008; Feng, Figlio, and Sass 2010), and NCLB altered teachers’ perceptions of their job security (Reback et al. 2014) and affected principals’ decisions about whether to remain in or leave their schools (Li 2015). NCLB also implemented subgroup-specific accountability using cutoff-based rules that divided otherwise similar schools into those that were accountable for their racial/ethnic subgroups and those that were not; prior studies demonstrate that setting cutoffs can affect individuals whose scores fall either just above or, in some cases, just below those cutoffs (Papay, Murnane, and Willett 2010; Domina, Penner, and Penner 2016). The implementation of NCLB's subgroup-specific accountability might thus have shaped teachers’ decisions to remain in or leave their schools or teaching, as well as administrators’ decisions about which teachers to retain, particularly in schools that fell near these cutoffs.

This study investigates whether the initial implementation of NCLB's subgroup-specific accountability affected the likelihood that teachers left their schools or teaching. Using data on elementary school teachers in North Carolina, this study finds that when first implemented, subgroup-specific accountability that targeted the black or white student subgroups had no overall effects on teacher turnover or attrition. Accountability for the black student subgroup, however, caused black teachers in schools that were held accountable for the performance of the black subgroup to leave teaching at significantly lower rates than black teachers who taught in schools not accountable for the black subgroup's performance. It is unclear, however, whether this discontinuity is driven by a decrease in black teacher attrition in schools just above the cutoff or an increase in attrition of black teachers in schools just below. Further analysis suggests that the policy caused changes in the teaching assignments of black teachers, particularly the percentages of black students they were assigned. These results suggest that the implementation of subgroup-specific accountability altered the composition of the teacher labor force in unintended ways.

2.  Background

Subgroup-Specific Accountability

A central goal of NCLB was to close racial/ethnic achievement gaps. To do so, the law required that the students in each racial/ethnic subgroup in each school make yearly gains in achievement, or the entire school would be labeled as failing. NCLB did not hold all schools accountable for the performance of their subgroups, however. To address concerns about the reliability of AYP determinations, as well as about student privacy, NCLB stipulated that a subgroup's performance would not “count” toward AYP determinations for schools with very small numbers of students in that subgroup. NCLB required states to set a “minimum subgroup size,” which was the minimum number of students in a subgroup who were required for that subgroup to count in a school's AYP determination (Davidson et al. 2015). Schools whose numbers of students in a subgroup were equal to or greater than their state's minimum subgroup size were held accountable for that subgroup's performance, whereas schools whose numbers of students in a subgroup fell below that cutoff were not held accountable for that subgroup's scores. In the first year of NCLB, these minimum subgroup sizes ranged from a low of 10 to as high as 50 (Pierce 2003; Davidson et al. 2015). The state examined in this study, North Carolina, set its minimum subgroup size at 40 students, and did not allow this number to vary with the size of the school (Pierce 2003; Fulton 2006).

One consequence of these minimum subgroup size rules was that the difference of a single student in a subgroup could affect whether that subgroup's test scores counted in determining a school's AYP. In North Carolina, for example, the black student subgroup's performance did not count in AYP determinations for schools with 39 black students, but counted for schools with 40 black students. NCLB thus required states to draw arbitrary lines that divided otherwise similar schools, holding schools on one side of the line accountable for the performance of a subgroup while leaving schools on the other side unaccountable. This study investigates whether implementing such a policy caused teachers who worked in schools near these cutoffs to leave their schools or teaching.

North Carolina is an ideal place to examine the effects of the initial implementation of subgroup-specific accountability, since prior to NCLB, North Carolina had a strong state-level accountability system but no system of subgroup-specific accountability (North Carolina Department of Public Instruction 2003; Lauen and Gaddis 2012). The effects of subgroup-specific accountability in NCLB's first year were therefore undiluted by a similar state-level policy. Because North Carolina's state-level accountability system remained in effect when NCLB was implemented in 2002–03, the counterfactual to subgroup-specific accountability in this study is a strong state-level accountability system that lacked a subgroup-specific component. The rewards and sanctions faced by North Carolina schools did not change in NCLB's first year—under North Carolina's state-level accountability system, staff at high-performing schools received monetary awards, and low-performing schools were required to work with state assistance teams and notify parents of their low performance (North Carolina Department of Public Instruction 2003). These rewards and sanctions remained in place in 2002–03, but schools additionally reported their overall AYP status in mandatory reports to parents. NCLB sanctioned Title I schools if they did not make AYP, but only after schools failed to do so for two consecutive years (North Carolina Department of Public Instruction 2003).

Holding Schools Accountable Versus Low Performance

Most prior studies of the effects of NCLB's subgroup-specific accountability have explored the effects of low subgroup performance, or the threat of such low performance, on students and teachers (Lauen and Gaddis 2012; Reardon et al. 2013; Reback, Rockoff, and Schwartz 2014). Lauen and Gaddis (2012), the only previous study to explore the effects of subgroup-specific accountability in North Carolina, found that accountability pressure for the low-income, black, and Hispanic subgroups increased the subsequent achievement of those students. The “treatment” in the current study, however, differs from this prior work. Instead of investigating the effects of pressure generated by the implementation of subgroup-specific accountability, this study explores the effects of simply being held accountable for subgroups during NCLB's first year. More precisely, the treatment in this study can be thought of as a school being held accountable for a subgroup while otherwise similar schools were not. Sims (2013) similarly examined the impacts of being held accountable for an additional subgroup on both the probability of schools’ failing to make AYP and their subsequent student achievement. He found that schools held accountable for an additional subgroup were more likely to fail to make AYP, and such failure led to decreased future achievement for both the school overall as well as the subgroup in question (Sims 2013).

Subgroup-specific accountability could have affected different groups of teachers in different ways. Because subgroup-specific accountability was a policy that focused explicitly on student race, it is particularly plausible that the policy had different effects on teachers of various races. Prior research shows, for example, that black and white teachers respond differently to an influx of black students into their schools (Jackson 2009). Research also suggests an organization's orientation toward diversity is a key predictor of workers’ commitment to that organization, particularly for minority workers (Foley, Kidder, and Powell 2002; Chrobot-Mason 2003), and minority workers’ perceptions of their workplaces’ “diversity climates” are key predictors of their turnover (Griffeth and Hom 2001). The policy could also have impacted teachers directly, by affecting teachers’ decisions about whether or not to remain in their schools or teaching, or indirectly, by altering administrators’ decisions about teachers’ class assignments, outside-of-classroom responsibilities, and which teachers to remove or keep.

There are four main ways that being held accountable for a subgroup might affect teacher turnover or attrition. One is by altering the pressure that teachers felt to improve their students’ achievement. Because schools that were accountable for more subgroups were more likely to fail to make AYP (Kane and Staiger 2002; Sims 2013), schools whose numbers of students in a subgroup were just above the minimum subgroup size cutoff might have felt more accountability pressure than teachers in schools just below the cutoff. Accountability pressure has been shown to decrease teachers’ perceptions of their job security (Crocco and Costigan 2007; Reback, Rockoff, and Schwartz 2014), and the decreased job security for teachers in above-cutoff schools could have driven these teachers from their schools or even from teaching. Changes in such pressure could also have altered administrators’ decisions about classroom assignments or about which teachers to keep or remove. If black teachers were more effective teachers of black students, for example, administrators of schools that were newly accountable for the black subgroup could have reassigned black students to those teachers in response to the increased pressure associated with those students’ performance.

A second way that subgroup-specific accountability might have affected teachers is by altering the allocation of resources either within or between schools. Prior evidence suggests that increased resources often accompany accountability pressure, and that these resources can potentially entice teachers to stay in their schools (Feng, Figlio, and Sass 2010). Subgroup-specific accountability could have led to increased resources—including materials, planning time, and other supports—being devoted to schools just above the cutoff as compared to those just below; these additional resources could have made teaching more appealing and enticed teachers to stay in their schools or in teaching. Within schools, administrators may have made particular efforts to direct resources to teachers who taught the students for which schools were newly accountable, making these teachers particularly likely to remain. Students could also have been assigned to teachers differently after the policy took effect. Again, this change could have affected teachers of different races differently. If black teachers were more likely to teach black students, for example, and increased resources were directed to black students (and their teachers) in schools that were newly accountable for the black subgroup, these resources could have been particularly beneficial to black teachers.

Third, subgroup-specific accountability could have affected teachers’ motivation to remain in their schools or teaching. Again, these effects could have been different for teachers of different races. For example, research suggests that some black teachers enter teaching because of a “calling” prompted by their sense of duty to improve educational outcomes for black students (Casey 1993; Irvine 2002; Dixson and Dingus 2008). In schools above the subgroup accountability cutoff, black students’ performance “counted” toward AYP, reinforcing the message that black students’ achievement was important, which may have been particularly congruent with some black teachers’ motivations for entering and remaining in teaching. In below-cutoff schools, however, black students’ performance did not “count,” a message that could have conflicted with these teachers’ motivation to teach, perhaps driving them from their schools or even from teaching.

Finally, subgroup-specific accountability could have impacted teachers by causing school leaders to strategically manipulate their schools’ numbers of students in various subgroups. Prior work finds that schools indeed manipulate their numbers of students in subgroups in order to avoid NCLB accountability for those subgroups (Jennings and Crosta 2011). This finding corroborates findings from other studies suggesting that some schools use suspensions and classification into special education to remove potentially low-performing students from the testing pool (Cullen and Reback 2006; Figlio 2006; Figlio and Getzler 2006). Teachers could have directly observed this manipulation or heard of it second-hand, and knowledge of this manipulation could have affected their motivation to remain in their schools or even teaching. Such manipulation in response to NCLB could also have been correlated with other practices in which some administrators engaged, making schools particularly bad places to work.

The impacts of subgroup-specific accountability on teachers also may not have been confined to the schools above the cutoffs—recent research in education suggests that cutoffs that divide otherwise similar individuals into groups can affect behavior for those who fall just short of those cutoffs. Domina, Penner, and Penner (2016), for example, examined the effects of a policy that gave color-coded identification cards to high school students based on their test scores; these cards were publicly displayed and conferred a number of advantages to students. The authors found that students whose scores fell just short of the cutoff for a more prestigious identification card had lower test scores and grades the following year (Domina, Penner, and Penner 2016). Papay, Murnane, and Willett (2010) similarly found that low-income urban high school students who barely failed a tenth-grade math exit exam were significantly more likely to drop out of school the following year than similar students who just passed the exam. In both cases, falling just short of the cutoff appeared to significantly affect behavior. Subgroup-specific accountability could have similarly affected the turnover or attrition of teachers in schools that fell just short of the cutoffs, perhaps by affecting teachers’ motivation, as discussed previously.

Although the four explanations outlined in this section have some theoretical and empirical support, they are largely speculative. However, they each lead to some testable predictions. If changes in pressure explained any effects of subgroup-specific accountability on teachers, we would expect teachers to leave their schools or teaching at higher rates from schools that faced subgroup-specific accountability, compared with those that did not. We also might observe that these effects were greatest in the schools where those subgroups were most likely to fail. If resource allocation were most important, we would expect to see lower turnover and attrition from schools that faced accountability for their subgroups. We might also expect to see those teachers who worked most often with the focal students leave their schools or teaching at particularly low rates. If motivation were a main means by which subgroup-specific accountability affected teachers, we might expect to see minority teachers remaining in their schools or teaching at higher rates when their same-race subgroup “counted” in their schools, or perhaps leaving their schools or teaching when they fell just short of those cutoffs. If strategic manipulation took place, we might see particular effects for same-race teachers in schools that fell just short of the subgroup accountability cutoff. After describing the data and methods, this study examines the overall effects of subgroup-specific accountability on teachers, then turns to testing these potential explanations.

3.  Methods

Data and Sample

Data for this study were taken from administrative data on North Carolina students, teachers, and schools from the school years 1999–2000 through 2003–04. This five-year period spanned the initial implementation of NCLB in 2002–03, allowing a comparison of teacher outcomes in schools whose numbers of students in their subgroups fell just above or just below the state's minimum subgroup-size cutoff in the periods before and after the law took effect. These data include information on every public school teacher in the state, including information on teachers’ race/ethnicity, education, and the racial/ethnic composition and achievement of their students. For the purposes of these analyses, the sample was limited to elementary school teachers, who had direct responsibility for specific students and therefore might be expected to respond particularly strongly to subgroup-specific accountability.1 The sample was also limited to black and white teachers, as these teachers constituted the vast majority of North Carolina elementary school teachers during the study period (14 percent and 84 percent, respectively, in 2001–02), and these analyses sought to explore the effects of accountability for the same-race subgroup on these teachers.

Using this administrative data, measures of one-year and two-year turnover and attrition were constructed for both the pre- and post-NCLB periods. In the pre-NCLB period, each public elementary school teacher in North Carolina in 1999–2000 was searched for in the state data from either 2000–01 or 2001–02 (depending on the particular outcome) to determine whether that teacher (1) still taught in the same North Carolina public school, (2) taught in a different North Carolina public school, or (3) had left public school teaching in North Carolina. Similar measures were constructed for the post-NCLB period, beginning with all North Carolina teachers in 2001–02 (the year before NCLB went into effect), and examining outcomes for these teachers in either 2002–03 or 2003–04.

Both one-year and two-year outcomes were examined because the implementation of subgroup-specific accountability took place over two years, and there was ambiguity about when North Carolina schools and districts became aware of the state minimum subgroup size of 40 students. NCLB was signed into law in January 2002, but North Carolina's minimum subgroup size was not approved until April 2003 (Hickok 2003). Within this large window of time, it is unclear exactly when North Carolina schools and districts became aware of the state minimum subgroup size of 40 students. It is also unclear when teachers learned where their schools’ subgroups fell in relation to that cutoff. NCLB based its subgroup-specific accountability on the number of tested students but required students to have been enrolled in a school for nearly the entire school year to count as a member of a subgroup.2 Preliminary test results were released beginning in June 2003, and press coverage in North Carolina at that time suggested that NCLB had had positive impacts on student achievement, particularly for minority students (North Carolina Department of Public Instruction 2003; Silberman 2003). Teachers, therefore, could have been aware of where their schools’ subgroups fell in relation to the cutoff by this time, if not earlier. Given the uncertainty about the timeline of the policy implementation, it is also possible that school administrators knew of the cutoff of 40 earlier than their teachers, and were able to respond to that cutoff, perhaps by engaging in strategic manipulation (as described earlier). Because of the ambiguity in the timeline of the policy's implementation, as well as when exactly teachers and administrators knew where they fell in relation to the minimum subgroup size cutoff, analyses for this study examined both one-year and two-year outcomes. Two-year attrition was the preferred outcome, however, since it captured attrition from the year before NCLB through the full initial implementation of the policy.

Data on the implementation of subgroup-specific accountability were taken from the Barnard/Columbia NCLB database, a state-by-state compendium of information on states’ implementation of NCLB (Reback et al. 2011). These data, which were compiled from publicly available records and personal correspondence with state officials, provided the number of “continuously enrolled” and tested students by subgroup in each North Carolina school in the first two years of NCLB (2002–03 and 2003–04), and were available for 94 percent of the elementary schools in North Carolina during the study period.

Along with the numbers of tested students in each subgroup in each school, the Barnard/Columbia NCLB data also contained the AYP outcomes (pass or fail) for each subgroup in each school in both reading and math in 2002–03 and 2003–04, the first two years of NCLB. If a school had an AYP outcome listed for a subgroup in both subjects, that school was deemed to have been held accountable for that subgroup that year; any school without an AYP outcome for a subgroup in both subjects was deemed to have not been held accountable for that subgroup. North Carolina elementary schools were held accountable for the black and white subgroups far more often than for any other racial ethnic groups—in the first year of NCLB, 64 percent of schools were held accountable for the black subgroup, and 82 percent of schools for the white subgroup. (In contrast, only 8 percent of schools were held accountable for the Hispanic subgroup, and 1 percent of schools for the Asian subgroup.) Because large numbers of schools were held accountable for the black and white subgroups, and because nearly all elementary school teachers in North Carolina at the time of NCLB's initial implementation were either black or white, analyses for this study focused on the effects of being held accountable for these two subgroups.

In the first year of NCLB, North Carolina perfectly assigned schools to treatment or nontreatment status based on the numbers of students in their black or white subgroups, holding every school with 40 or more black or white students accountable for that subgroup, and not holding any schools accountable for the black or white subgroup when the number of such students was below 40. Ninety-five percent of schools that were held accountable for the black or white subgroups in the first year of NCLB were also held accountable for that subgroup in the law's second year, making it impossible to examine separate effects of subgroup-specific accountability in NCLB's first two years.

Table 1 presents descriptive statistics on the sample of elementary schools in North Carolina in 2001–02, the year before NCLB took effect. Column 1 shows descriptive statistics for all North Carolina elementary schools; column 2 for North Carolina elementary schools whose numbers of black students fell close to the state minimum subgroup size (between 35 and 44 tested black students) during the first year of NCLB; and column 3 for elementary schools whose numbers of white students were close to the state minimum subgroup size (between 35 and 44 tested white students). Schools whose numbers of black students were close to 40 were generally more advantaged than other schools in the state. These schools had smaller percentages of low-income students and higher state-level accountability performance than other schools. These schools also had, on average, greater percentages of white students and smaller percentages of black students than other North Carolina schools, and were more likely to be located in rural areas and suburbs. Schools near the cutoff for the number of tested white students, in contrast, were generally less-advantaged than other schools in the state.

Table 1.
North Carolina Elementary School Characteristics, 2001—02
All North Carolina Elementary Schools (1)Elementary Schools with 35—44 Tested Black Studentsa (2)Elementary Schools with 35—44 Tested White Studentsa (3)
Fraction free or reduced-price lunch students 0.50 0.41 0.72 
 (0.24) (0.23) (0.21) 
Fraction Asian students 0.02 0.02 0.02 
 (0.03) (0.03) (0.02) 
Fraction black students 0.35 0.24 0.56 
 (0.26) (0.14) (0.17) 
Fraction Hispanic students 0.07 0.07 0.08 
 (0.07) (0.06) (0.08) 
Fraction Native American students 0.01 0.01 0.01 
 (0.05) (0.01) (0.02) 
Fraction white students 0.55 0.66 0.33 
 (0.28) (0.16) (0.18) 
Located in city 0.31 0.20 0.58 
 (0.46) (0.40) (0.50) 
Located in suburbs 0.17 0.26 0.00 
 (0.37) (0.44) (0.00) 
Located in town 0.14 0.10 0.25 
 (0.35) (0.30) (0.44) 
Located in rural area 0.39 0.45 0.17 
 (0.49) (0.50) (0.38) 
Charter school 0.01 0.02 0.03 
 (0.12) (0.15) (0.17) 
Number of students 511 521 414 
 (197) (200) (163) 
Exemplary (ABC Accountability)b 0.42 0.49 0.28 
 (0.50) (0.50) (0.45) 
Expected (ABC Accountability)b 0.37 0.34 0.19 
 (0.48) (0.48) (0.40) 
No recognition (ABC Accountability)b 0.20 0.17 0.53 
 (0.40) (0.38) (0.51) 
Predicted probability black subgroup made AYP 0.86 0.86 0.85 
 (0.03) (0.02) (0.02) 
Predicted probability white subgroup made AYP 0.98 0.99 0.98 
 (0.09) (0.04) (0.04) 
Schools, N 1,060 94 36 
All North Carolina Elementary Schools (1)Elementary Schools with 35—44 Tested Black Studentsa (2)Elementary Schools with 35—44 Tested White Studentsa (3)
Fraction free or reduced-price lunch students 0.50 0.41 0.72 
 (0.24) (0.23) (0.21) 
Fraction Asian students 0.02 0.02 0.02 
 (0.03) (0.03) (0.02) 
Fraction black students 0.35 0.24 0.56 
 (0.26) (0.14) (0.17) 
Fraction Hispanic students 0.07 0.07 0.08 
 (0.07) (0.06) (0.08) 
Fraction Native American students 0.01 0.01 0.01 
 (0.05) (0.01) (0.02) 
Fraction white students 0.55 0.66 0.33 
 (0.28) (0.16) (0.18) 
Located in city 0.31 0.20 0.58 
 (0.46) (0.40) (0.50) 
Located in suburbs 0.17 0.26 0.00 
 (0.37) (0.44) (0.00) 
Located in town 0.14 0.10 0.25 
 (0.35) (0.30) (0.44) 
Located in rural area 0.39 0.45 0.17 
 (0.49) (0.50) (0.38) 
Charter school 0.01 0.02 0.03 
 (0.12) (0.15) (0.17) 
Number of students 511 521 414 
 (197) (200) (163) 
Exemplary (ABC Accountability)b 0.42 0.49 0.28 
 (0.50) (0.50) (0.45) 
Expected (ABC Accountability)b 0.37 0.34 0.19 
 (0.48) (0.48) (0.40) 
No recognition (ABC Accountability)b 0.20 0.17 0.53 
 (0.40) (0.38) (0.51) 
Predicted probability black subgroup made AYP 0.86 0.86 0.85 
 (0.03) (0.02) (0.02) 
Predicted probability white subgroup made AYP 0.98 0.99 0.98 
 (0.09) (0.04) (0.04) 
Schools, N 1,060 94 36 

Notes: AYP: Adequate Yearly Progress.

aIn 2002—03, the first year of NCLB.

bNorth Carolina's state-level accountability system.

Table 2 shows information on whether or not North Carolina elementary schools were held accountable for various subgroups during the first year of NCLB. In order to give an overall sense of the implementation of subgroup-specific accountability during NCLB's first year, this table is not limited to the black or white subgroups. In NCLB's first year, there was a great deal of overlap in accountability for various subgroups. For example, 92 percent of schools near the cutoff for the number of black students were held accountable for the white subgroup, and 86 percent of schools near the cutoff for the number of white students were held accountable for the performance of the black subgroup. Because of these overlaps in accountability for various subgroups, some analyses controlled for whether schools were accountable for subgroups other than the focal subgroup.

Table 2.
Subgroup-specific Accountability in North Carolina Elementary Schools, 2002—03
All North Carolina Elementary Schools (1)Elementary Schools with 35—44 Tested Black Students (2)Elementary Schools with 35—44 Tested White Students (3)
Accountable for Asian subgroup 0.01 0.03 0.00 
 (0.11) (0.18) (0.00) 
Accountable for black subgroup 0.64 0.50 0.86 
 (0.48) (0.50) (0.35) 
Accountable for Hispanic subgroup 0.08 0.04 0.11 
 (0.28) (0.20) (0.32) 
Accountable for multiracial subgroup 0.00 0.00 0.00 
 (0.00) (0.00) (0.00) 
Accountable for Native American subgroup 0.01 0.00 0.00 
 (0.11) (0.00) (0.00) 
Accountable for white subgroup 0.82 0.92 0.47 
 (0.38) (0.28) (0.51) 
Accountable for economically disadvantaged subgroup 0.91 0.88 0.89 
 (0.29) (0.32) (0.32) 
Accountable for limited English subgroup 0.04 0.02 0.06 
 (0.19) (0.15) (0.23) 
Accountable for special education subgroup 0.32 0.28 0.14 
 (0.47) (0.45) (0.35) 
Schools, N 1,060 94 36 
All North Carolina Elementary Schools (1)Elementary Schools with 35—44 Tested Black Students (2)Elementary Schools with 35—44 Tested White Students (3)
Accountable for Asian subgroup 0.01 0.03 0.00 
 (0.11) (0.18) (0.00) 
Accountable for black subgroup 0.64 0.50 0.86 
 (0.48) (0.50) (0.35) 
Accountable for Hispanic subgroup 0.08 0.04 0.11 
 (0.28) (0.20) (0.32) 
Accountable for multiracial subgroup 0.00 0.00 0.00 
 (0.00) (0.00) (0.00) 
Accountable for Native American subgroup 0.01 0.00 0.00 
 (0.11) (0.00) (0.00) 
Accountable for white subgroup 0.82 0.92 0.47 
 (0.38) (0.28) (0.51) 
Accountable for economically disadvantaged subgroup 0.91 0.88 0.89 
 (0.29) (0.32) (0.32) 
Accountable for limited English subgroup 0.04 0.02 0.06 
 (0.19) (0.15) (0.23) 
Accountable for special education subgroup 0.32 0.28 0.14 
 (0.47) (0.45) (0.35) 
Schools, N 1,060 94 36 

Some analyses examined whether the effects of subgroup-specific accountability differed depending on the level of pressure schools faced for their subgroups. For these analyses, schools were divided into groups based on the predicted probability their black or white subgroups would make AYP, and indicators for each of these groups were interacted with each variable included in the analyses.3 Other analyses explored whether the policy's effects differed depending on the percentages of black or white students in teachers’ classes. To accomplish this, the percentage of teachers’ students who were black or white was interacted with each variable; in another analysis, each variable was interacted with indicators for whether teachers’ percentages of black or white students were more than a standard deviation above the mean for the entire sample, within a standard deviation of the mean, or more than a standard deviation below the mean.

Table 3 presents descriptive statistics on the sample of North Carolina elementary teachers used in this study. Teachers in schools with close to 40 tested black students were more likely to be white than teachers in schools with close to 40 tested white students. They were also more likely to have higher scores on licensure exams, standard teaching certification, and National Board Certification. Teachers in schools with close to 40 tested white students were also more likely to leave their schools or teaching within one or two years.

Table 3.
North Carolina Elementary Teacher Characteristics, 2001—02
All North Carolina Elementary Schools (1)Elementary Schools with 35—44 Tested Black Studentsa (2)Elementary Schools with 35—44 Tested White Studentsa (3)
MeanSDMeanSDMeanSD
Asian 0.00 (0.05) 0.00 (0.06) 0.00 (0.07) 
Black 0.14 (0.34) 0.08 (0.27) 0.23 (0.42) 
Hispanic 0.01 (0.09) 0.01 (0.09) 0.01 (0.09) 
Multiracial 0.01 (0.07) 0.00 (0.05) 0.00 (0.07) 
Native American 0.01 (0.09) 0.00 (0.06) 0.01 (0.09) 
White 0.84 (0.37) 0.91 (0.29) 0.75 (0.44) 
Female 0.94 (0.23) 0.95 (0.22) 0.92 (0.27) 
Entering first year in education 0.06 (0.24) 0.06 (0.24) 0.08 (0.27) 
Very competitive college 0.07 (0.26) 0.09 (0.28) 0.06 (0.23) 
Competitive college 0.47 (0.50) 0.51 (0.50) 0.41 (0.49) 
Less competitive college 0.39 (0.49) 0.35 (0.48) 0.44 (0.50) 
Holds advanced degree 0.27 (0.45) 0.28 (0.45) 0.24 (0.43) 
National Board certified 0.04 (0.20) 0.05 (0.22) 0.02 (0.13) 
High-scorer on licensure exam 0.10 (0.30) 0.10 (0.30) 0.09 (0.28) 
Middle-scorer on licensure exam 0.78 (0.42) 0.82 (0.39) 0.75 (0.43) 
Low-scorer on licensure exam 0.12 (0.33) 0.08 (0.27) 0.16 (0.37) 
Holds standard license 0.93 (0.26) 0.93 (0.25) 0.90 (0.30) 
Taught self-contained class 0.84 (0.37) 0.85 (0.36) 0.87 (0.33) 
% students taught that are black 0.32 (0.27) 0.21 (0.15) 0.56 (0.19) 
% students taught that are white 0.54 (0.30) 0.65 (0.22) 0.28 (0.17) 
Left school within 1 year 0.22 (0.41) 0.22 (0.41) 0.26 (0.44) 
Left school within 2 years 0.35 (0.48) 0.36 (0.48) 0.44 (0.50) 
Left teaching within 1 yearb 0.17 (0.37) 0.16 (0.37) 0.19 (0.40) 
Left teaching within 2 yearsb 0.26 (0.44) 0.28 (0.45) 0.30 (0.46) 
Teachers, N 31, 211 2,695 930 
All North Carolina Elementary Schools (1)Elementary Schools with 35—44 Tested Black Studentsa (2)Elementary Schools with 35—44 Tested White Studentsa (3)
MeanSDMeanSDMeanSD
Asian 0.00 (0.05) 0.00 (0.06) 0.00 (0.07) 
Black 0.14 (0.34) 0.08 (0.27) 0.23 (0.42) 
Hispanic 0.01 (0.09) 0.01 (0.09) 0.01 (0.09) 
Multiracial 0.01 (0.07) 0.00 (0.05) 0.00 (0.07) 
Native American 0.01 (0.09) 0.00 (0.06) 0.01 (0.09) 
White 0.84 (0.37) 0.91 (0.29) 0.75 (0.44) 
Female 0.94 (0.23) 0.95 (0.22) 0.92 (0.27) 
Entering first year in education 0.06 (0.24) 0.06 (0.24) 0.08 (0.27) 
Very competitive college 0.07 (0.26) 0.09 (0.28) 0.06 (0.23) 
Competitive college 0.47 (0.50) 0.51 (0.50) 0.41 (0.49) 
Less competitive college 0.39 (0.49) 0.35 (0.48) 0.44 (0.50) 
Holds advanced degree 0.27 (0.45) 0.28 (0.45) 0.24 (0.43) 
National Board certified 0.04 (0.20) 0.05 (0.22) 0.02 (0.13) 
High-scorer on licensure exam 0.10 (0.30) 0.10 (0.30) 0.09 (0.28) 
Middle-scorer on licensure exam 0.78 (0.42) 0.82 (0.39) 0.75 (0.43) 
Low-scorer on licensure exam 0.12 (0.33) 0.08 (0.27) 0.16 (0.37) 
Holds standard license 0.93 (0.26) 0.93 (0.25) 0.90 (0.30) 
Taught self-contained class 0.84 (0.37) 0.85 (0.36) 0.87 (0.33) 
% students taught that are black 0.32 (0.27) 0.21 (0.15) 0.56 (0.19) 
% students taught that are white 0.54 (0.30) 0.65 (0.22) 0.28 (0.17) 
Left school within 1 year 0.22 (0.41) 0.22 (0.41) 0.26 (0.44) 
Left school within 2 years 0.35 (0.48) 0.36 (0.48) 0.44 (0.50) 
Left teaching within 1 yearb 0.17 (0.37) 0.16 (0.37) 0.19 (0.40) 
Left teaching within 2 yearsb 0.26 (0.44) 0.28 (0.45) 0.30 (0.46) 
Teachers, N 31, 211 2,695 930 

aIn 2002—03, the first year of NCLB.

bPublic school teaching in North Carolina.

Research Design

This study used a regression discontinuity (RD) design to determine the effects of subgroup-specific accountability on teachers. The identifying assumption of the analysis was that schools with 39 tested students in a subgroup were identical to schools with 40 tested students, except for the fact that NCLB held schools with 40 students in the subgroup accountable for that subgroup's performance. Any discontinuity in teacher turnover or attrition at the cutoff of 40 tested black or white students, therefore, was directly attributable to the policy.

Because the likelihood of receiving the treatment jumped from zero to one at the cutoff, this study used a “sharp” RD design (Lee and Lemieux 2010; Bloom 2012). Linear probability models were estimated for ease of interpretation, with standard errors clustered at the school level to reflect teacher grouping in schools and the likely correlation of teacher outcomes within schools. The bandwidth within which the analyses were conducted varied between 10, 20, and 30 tested students around the cutoff. Except for in a few cases (noted below), the assignment variable—the number of tested students in the black or white subgroups—was always from the first year of NCLB (2002–03).4

The model for the RD analyses in the post-NCLB period was as follows:
Yijt=β0+β1Zjt-1+β2Xjt-1+β3Xjt-12+β4Xjt-13+β5Xjt-1Zjt-1+β6Xjt-12Zjt-1+β7Xjt-13Zjt-1+ϵijt,
(1)
where the outcome variable Yijt was an indicator for whether teacher i in school j left the base year school, or public school teaching in North Carolina, in year t (2003–04); Zjt-1 was an indicator for whether the number of tested black or white students in school j met or exceeded North Carolina's minimum subgroup size of 40 in year t − 1, the first year of NCLB (2002–03); Xjt-1 was the assignment variable, the number of tested black or white students in school j in year t − 1; and Xjt-12 and Xjt-13 were quadratic and cubic powers of the assignment variable. Xjt-1Zjt-1, Xjt-12Zjt-1, and Xjt-13Zjt-1 were interactions between Xjt-1, Xjt-12, and Xjt-13 and the treatment dummy Zjt-1. Values of Xjt-1 and its higher-order polynomials were centered at the cutoff of 40, which made the intercept of the equation (β0) the estimate of teacher turnover or attrition at the cutoff in the absence of subgroup-specific accountability. β1 was the treatment effect, and estimated the effect of subgroup-specific accountability for the black or white subgroup on teacher turnover or attrition in schools with exactly 40 tested students in that subgroup in 2002–03, the first year of NCLB.5 Pre-NCLB analyses used 2001–02 as the base year.
Because the data spanned the pre- and post-NCLB periods, some analyses pooled pre- and post-NCLB data and estimated a difference-in-differences RD model. This model not only estimated the discontinuity in teacher turnover or attrition at the cutoff in both the pre- and post-NCLB periods, but also estimated the difference in those discontinuities. This model was estimated as follows:
Yijt=β0+β1Zjt-1+β2Tt+β3Zjt-1Tt+β4Xj-1+β5Xjt-12+β6Xjt-13+β7Xjt-1Zjt-1+β8Xjt-12Zjt-1+β9Xjt-13Zt-1+β10Xjt-1Tt+β11Xjt-12Tt+β12Xjt-13Tt+β13Xjt-1Zjt-1Tt+β14Xjt-12Zjt-1Tt+β15Xjt-13Zjt-1Tt+ϵijt,
(2)
where the difference between this equation and equation 1 was the addition of Tt, a dummy variable that indicated whether an observation was from the post-NCLB period, and the interaction of this variable with each variable in equation 1 to differentiate their effects in the pre- and post-NCLB periods. The coefficient of interest was now β3, which estimated the difference in the discontinuities in teacher turnover or attrition between the pre- and post-NCLB periods at the cutoff of 40 tested black or white students. The assignment variable, Xjt-1, was still measured in the first year of NCLB (2002–03).

In addition to the parametric analyses discussed above, nonparametric RD models were also estimated, with bandwidths selected using the bandwidth optimization procedure of Imbens and Kalyanaraman (2012). Applying this procedure, however, often produced optimal bandwidths that were very small—usually between two and three, and sometimes less than two. Nonparametric analyses were therefore conducted using bandwidths of two, three, and four tested students around the cutoff, which are each very close to these optimal bandwidths, using a triangle kernel to weigh observations within each bandwidth (McCrary 2008).

Internal Validity of Research Design

Because the cutoff of 40 tested students was unique to NCLB, estimates of the policy's effects were not confounded with the effects of other policies. The institutional and statistical evidence also suggested that schools did not manipulate their numbers of tested black or white students in NCLB's first year, nor did teachers sort themselves to schools on either side of the cutoff in the period before the policy took effect.

Institutional Evidence Against Manipulation of the Assignment Variable.

Manipulation of schools’ numbers of tested students was unlikely to be substantial for several reasons. One was that schools lacked precise control over their student enrollments. In most public schools, additional students can enroll or withdraw at any time, giving schools only imprecise control over the student enrollments of various subgroups. Schools could have manipulated their numbers of tested students in the black or white subgroups by reclassifying student race; however, student race/ethnicity is recorded when students enroll in a particular North Carolina school district, and this information remains in the district's computer system when students transfer public schools within districts. The only substantial opportunity to reclassify student race, therefore, was when students were just entering a new district, and the numbers of such students in any given year were unlikely to be great.

Another way that the number of tested students in subgroups could have been manipulated was through the continuous enrollment provision of NCLB. Schools could have manipulated their numbers of tested students in the black or white subgroups by enrolling students just after the cutoff date for continuous enrollment, a practice which research suggests some schools indeed attempted (Jennings and Crosta 2011). Such manipulation, however, could only have affected students who were newly enrolling in a school near the cutoff date for continuous enrollment, and the numbers of such students were again unlikely to have been great.

Schools could also have classified black or white students into special education, or used suspensions, to push their numbers of tested students in these subgroups below the cutoffs. Prior research suggests that schools indeed use suspensions and classification into special education to remove potentially low-performing students from the testing pool (Cullen and Reback 2006; Figlio 2006; Figlio and Getzler 2006). NCLB, however, required that 95 percent of the continuously enrolled students in each subgroup be tested, or the entire school would fail to make AYP. Thus, the only schools that could have excluded enough students from testing to push themselves below the minimum subgroup size cutoff were those whose true numbers of continuously enrolled students were just above the cutoff. In North Carolina, only schools with 40 or 41 continuously enrolled students in a subgroup could have excluded enough students from test-taking to push that subgroup's numbers of tested students below 40 and still meet the 95 percent participation requirement. Similarly, only schools with 38 or 39 tested students in a subgroup could actually have had 40 or more continuously enrolled students in that subgroup. This form of manipulation was therefore only plausible in a very small number of schools. The following sections explore whether the results presented are robust to excluding schools where such manipulation was possible.

Statistical Evidence Against Manipulation of the Assignment Variable.

A common test for manipulation of the assignment variable in RD is to look for a discontinuity in the density of cases at the cutoff (Lee and Lemieux 2010; Bloom 2012). Graphs of the results from such a test are presented in figure 1, using data from 2002–03. Results showed no statistically significant discontinuities in the density of schools at the cutoff for either the number of tested black students (p = 0.15) or white students (p = 0.32). For the number of tested black students, the figure shows a greater density of schools just above the cutoff than just below, contrary to what might be expected had manipulation indeed occurred.

Figure 1.

Discontinuities in Density of Assignment Variable. a. Number of Tested Black Students. b. Number of Tested White Students.

Figure 1.

Discontinuities in Density of Assignment Variable. a. Number of Tested Black Students. b. Number of Tested White Students.

Another test for manipulation is to examine the smoothness of pretreatment covariates across the cutoff (Lee and Lemieux 2010; Bloom 2012). Figure 2 shows the smoothness of several pretreatment covariates across the cutoff for the number of tested black students. At the school level (panel A), these included school size, the proportion of students qualifying for free or reduced-price school lunch, the proportion of schools located in an urban area, and the proportion of schools rated “exemplary” under North Carolina's state-level accountability system. At the teacher level (panel B), these included whether a teacher was National Board–certified, a high-scorer on teacher licensure exams, entering his/her first year in education, or attended a less competitive college. Each of these covariates was from 2001–02, the year before NCLB went into effect, and the x variable (the number of tested black students) was from 2002–03, the first year of NCLB. All analyses were conducted at the school level. The graphs each include a scatterplot, a local linear (nonparametric) regression line, and cubic, quadratic, and linear fits, each fitted separately on both sides of the cutoff. Results for the white subgroup were similar and are not presented here.

Figure 2.

Estimated Discontinuities in Pretreatment School and Teacher Covariates a. School Covariates. b. Teacher Covariates.

Figure 2.

Estimated Discontinuities in Pretreatment School and Teacher Covariates a. School Covariates. b. Teacher Covariates.

Panel A shows there were no discontinuities in any pretreatment school covariates at the cutoff for the number of tested black students. Panel B similarly shows no discontinuities in teacher characteristics at the cutoff of 40 tested black students in the pretreatment year, 2001–02. The findings of panels A and B were confirmed in regressions that are not presented here, which showed no pattern of significant discontinuities in any pretreatment covariates at any of the bandwidths tested.6

Inclusion of Covariates

Prior research suggests that teacher turnover and attrition are associated with the percentage of minority students in a school (Borman and Dowling 2008), therefore, one might expect an increase in teacher turnover and attrition as the number of tested black students in schools increases. This was indeed the case, even in a relatively small window around the cutoff. Limiting the sample to teachers in the pre-NCLB period in schools that would later be within a 30-student bandwidth of the cutoff, an increase of one tested black student was associated with an increased probability of leaving teaching of a tenth of a percentage point (p = 0.02). Also consistent with prior research (see Jackson 2009), this was true for white, but not black, teachers. The number of tested white students was not associated with the likelihood of attrition for either black or white teachers.

The addition of school covariates, however, accounted for this trend. After including the full set of school covariates in table 1, schools’ numbers of tested black students were no longer a statistically significant predictor of teacher attrition. This finding provided further reassurance that there were not large, underlying differences in teacher attrition across the cutoff in the period before NCLB took effect, and those differences that were present could be accounted for by the inclusion of school-level covariates. Section 4, therefore, reports results of models that both include and exclude covariates, to assess whether the inclusion of these school and teacher characteristics had significant effects on the results.

4.  Results

Subgroup-specific accountability that focused on the black or white student subgroups had no overall effects on teacher turnover or attrition. This was true regardless of the predicted probability that the black subgroup would make AYP, or the percentage of teachers’ students who were black or white. At some bandwidths around the cutoff for the number of tested black students, there was a significant negative discontinuity in the pre-NCLB period in teacher turnover among teachers who taught the highest percentages of black students. Because there were relatively few teachers who taught high percentages of black students in schools near the cutoff for the black subgroup, however, this finding was likely to have been driven by random variation in turnover among these teachers.7

Separate analyses for black and white teachers, however, found that accountability for the black subgroup in the first year of NCLB had significant effects on the likelihood that black teachers left public school teaching in North Carolina. Table 4 shows RD estimates of the effects of accountability for the black subgroup on two-year teacher attrition, separating teachers into black teachers (top panel) and white teachers (bottom panel). The table contains estimates of the discontinuity in teacher attrition at the cutoff in the pre- and post-NCLB periods, as well as estimates of the difference in these discontinuities. The columns of the table correspond to various methods of analyses (nonparametric or parametric), bandwidths around the cutoff within which the analyses were conducted (2, 3, 4, 10, 20, or 30 tested students), and functional forms used to model the outcome-assignment variable relationship (cubic, quadratic, or linear). The “mean attrition” in the leftmost column is the mean attrition in either the pre- or post-NCLB period, depending on the row, and is provided to give a sense of the magnitude of the discontinuities estimated across that row of the table.

Table 4.
Effects of Accountability for Black Subgroup on Two-Year Attrition for Black and White Teachers
NonparametricParametric (cubic)Parametric (quadratic)Parametric (linear)
Panel A: Black Teachers
Mean attrition234102030102030102030
Post-NCLB 0.258 −0.360** −0.495* −0.602*** −0.715** −0.390* −0.311* −0.431* −0.238* −0.221* −0.213* −0.141* −0.062 
  (0.144) (0.270) (0.189) (0.243) (0.163) (0.132) (0.173) (0.114) (0.096) (0.110) (0.076) (0.070) 
Pre-NCLB 0.259 −0.056 −0.270 −0.162 −0.088 −0.020 −0.090 −0.020 −0.037 −0.024 −0.019 −0.003 0.067 
  (0.141) (0.259) (0.184) (0.369) (0.211) (0.152) (0.243) (0.023) (0.101) (0.118) (0.074) (0.067) 
Difference     −0.628** −0.370** −0.222* −0.411** −0.201* −0.197** −0.195* −0.138* −0.129* 
     (0.255) (0.160) (0.131) (0.175) (0.114) (0.098) (0.111) (0.080) (0.071) 
N, post-NCLB  109 136 195 336 707 971 336 707 971 336 707 971 
N, pre-NCLB  116 153 216 363 764 1,037 363 764 1,037 363 764 1,037 
Panel B: White Teachers 
Post-NCLB 0.254 0.004 0.065 0.013 0.006 −0.074 −0.009 −0.019 0.005 0.014 −0.014 0.011 0.003 
  (0.040) (0.075) (0.056) (0.084) (0.060) (0.046) (0.063) (0.042) (0.034) (0.039) (0.027) (0.023) 
Pre-NCLB 0.259 −0.018 −0.073 −0.029 0.036 0.028 −0.017 0.014 −0.007 −0.014 0.003 −0.018 −0.007 
  (0.038) (0.072) (0.055) (0.071) (0.046) (0.037) (0.049) (0.034) (0.028) (0.032) (0.023) (0.020) 
Difference     −0.030 −0.102 0.008 −0.033 0.012 0.028 −0.018 0.029 0.010 
     (0.092) (0.063) (0.051) (0.064) (0.046) (0.039) (0.044) (0.032) (0.027) 
N, post-NCLB  1,384 1,668 2,285 4,292 8,448 11,316 4,292 8,448 11,316 4,292 8,448 11,316 
N, pre-NCLB  1,357 1,640 2,113 3,987 7,914 10,618 3,987 7,914 10,618 3,987 7,914 10,618 
NonparametricParametric (cubic)Parametric (quadratic)Parametric (linear)
Panel A: Black Teachers
Mean attrition234102030102030102030
Post-NCLB 0.258 −0.360** −0.495* −0.602*** −0.715** −0.390* −0.311* −0.431* −0.238* −0.221* −0.213* −0.141* −0.062 
  (0.144) (0.270) (0.189) (0.243) (0.163) (0.132) (0.173) (0.114) (0.096) (0.110) (0.076) (0.070) 
Pre-NCLB 0.259 −0.056 −0.270 −0.162 −0.088 −0.020 −0.090 −0.020 −0.037 −0.024 −0.019 −0.003 0.067 
  (0.141) (0.259) (0.184) (0.369) (0.211) (0.152) (0.243) (0.023) (0.101) (0.118) (0.074) (0.067) 
Difference     −0.628** −0.370** −0.222* −0.411** −0.201* −0.197** −0.195* −0.138* −0.129* 
     (0.255) (0.160) (0.131) (0.175) (0.114) (0.098) (0.111) (0.080) (0.071) 
N, post-NCLB  109 136 195 336 707 971 336 707 971 336 707 971 
N, pre-NCLB  116 153 216 363 764 1,037 363 764 1,037 363 764 1,037 
Panel B: White Teachers 
Post-NCLB 0.254 0.004 0.065 0.013 0.006 −0.074 −0.009 −0.019 0.005 0.014 −0.014 0.011 0.003 
  (0.040) (0.075) (0.056) (0.084) (0.060) (0.046) (0.063) (0.042) (0.034) (0.039) (0.027) (0.023) 
Pre-NCLB 0.259 −0.018 −0.073 −0.029 0.036 0.028 −0.017 0.014 −0.007 −0.014 0.003 −0.018 −0.007 
  (0.038) (0.072) (0.055) (0.071) (0.046) (0.037) (0.049) (0.034) (0.028) (0.032) (0.023) (0.020) 
Difference     −0.030 −0.102 0.008 −0.033 0.012 0.028 −0.018 0.029 0.010 
     (0.092) (0.063) (0.051) (0.064) (0.046) (0.039) (0.044) (0.032) (0.027) 
N, post-NCLB  1,384 1,668 2,285 4,292 8,448 11,316 4,292 8,448 11,316 4,292 8,448 11,316 
N, pre-NCLB  1,357 1,640 2,113 3,987 7,914 10,618 3,987 7,914 10,618 3,987 7,914 10,618 

Notes: Each cell reports the estimated discontinuity in the probability that a teacher in a school with 40 black students (North Carolina's minimum subgroup size) left public school teaching in North Carolina. Each cell reports the results from a separate regression. Nonparametric estimates are from local linear regressions in the designated bandwidth around the cutoff, using a triangle kernel. Parametric analyses are linear probability models that include the given terms of the assignment variable (the number of tested black students), as well as the interaction of each polynomial with the treatment dummy. “Mean attrition” is the teacher attrition of black or white teachers in the entire sample, and is provided for reference. Nonparametric bandwidths of 2, 3, and 4 are near the optimal bandwidth given by the Imbens and Kalyanaraman (2012) procedure, which ranges between 2 and 3 for these regressions. Standard errors for parametric analyses are clustered at the school level. Analyses do not include covariates.

*p < 0.10; **p < 0.05; ***p < 0.01.

Subgroup-specific accountability that focused on the black subgroup caused a significant, negative discontinuity in black teacher attrition at the cutoff of 40 tested black students. The estimates in panel A of table 4 show that this discontinuity in teacher attrition was consistent in magnitude and direction across both nonparametric and parametric analyses and a variety of functional forms and bandwidths. (The exception is the linear model, where estimates are smaller in magnitude and only marginally statistically significant; this model, however, generally provides a poor fit to the data.) The magnitude of these estimates suggests that subgroup-specific accountability for the black subgroup had a large negative effect on the likelihood that black teachers left teaching. Using a cubic functional form and a bandwidth of 20, for example, there is a negative discontinuity of 0.39 in black teacher attrition at the cutoff, representing a 68 percent drop in teacher attrition when compared with 0.57, the model's estimate of black teacher attrition at the cutoff in the absence of subgroup-specific accountability. This negative discontinuity could be the result of a decrease in black teacher attrition in above-cutoff schools, an increase in attrition in below-cutoff schools, or some combination of the two.

This discontinuity in black teacher attrition at the cutoff did not exist before NCLB took effect, as can be seen in the second row of panel A of table 4. This suggests that the post-NCLB discontinuity in black teacher attrition was not simply an artifact of an underlying difference between schools on either side of the cutoff that predated the policy, but that this discontinuity arose when NCLB took effect. Even after differencing out this small and statistically insignificant pre-NCLB discontinuity in black teacher attrition, the results of the difference-in-differences RD analyses (presented in the third row of panel A of table 4) show the discontinuity in teacher attrition caused by accountability for the black subgroup remained negative and was generally statistically significant, although at some bandwidths only marginally so. These findings again suggest that subgroup-specific accountability for the black subgroup significantly affected the attrition of black teachers from North Carolina public schools in the period immediately after NCLB took effect. Panel B of table 4 shows that accountability for the black subgroup did not affect the likelihood that white teachers left teaching, and analyses not presented here found that accountability for the white subgroup had no effects on either black or white teachers.

Figure 3 presents a graphical version of these analyses, showing black teacher attrition in the pre-NCLB period (top panel) and post-NCLB period (bottom panel), with the number of tested black students in teachers’ schools in 2002–03 on the x-axis. The top panel of the figure demonstrates that before NCLB went into effect there was no discontinuity in black teacher attrition in schools that would later have 40 tested black students. The bottom panel of the figure, however, shows a large, negative discontinuity in black teacher attrition at the cutoff in the post-NCLB period.

Figure 3.

Two-Year Black Teacher Attrition, by Number of Tested Black Students in School.

Figure 3.

Two-Year Black Teacher Attrition, by Number of Tested Black Students in School.

The post-NCLB discontinuity in black teacher attrition appears to be driven, at least in part, by an increase in black teacher attrition in schools with just fewer than 40 tested black students. To further explore the changes in black teacher attrition on both sides of the cutoff, figure 4 shows the difference in average black teacher attrition between the pre- and post-NCLB periods, by the number of tested black students in the school in the first year of NCLB. For the purposes of this figure, teacher attrition is calculated at the school level and then averaged for all schools with each number of tested black students. The figure shows, relative to the pre-NCLB period, black teacher attrition increased in the post-NCLB period in schools whose numbers of tested black students fell just below the minimum subgroup-size cutoff. At the same time, the figure also shows black teacher attrition decreased slightly, compared with the pre-NCLB period, in schools just above this cutoff, as shown in the graph by the slight dip below zero just to the right of the cutoff. Given the research design, it is not possible to determine whether changes on one side of the cutoff or the other (or both) led to the discontinuity observed here.

Figure 4.

Pre–Post-NCLB Difference in Two-Year Black Teacher Attrition, by Number of Tested Black Students in School.

Figure 4.

Pre–Post-NCLB Difference in Two-Year Black Teacher Attrition, by Number of Tested Black Students in School.

To determine how much of this discontinuity in two-year attrition occurred during the first year of the policy (between 2001–02 and 2002–03) and how much occurred in the second year (between 2002–03 and 2003–04), similar analyses were conducted using one-year attrition as the outcome, as well as for an additional year after the policy took effect (between 2003–04 and 2004–05), and three years prior to NCLB. Results of these analyses are presented in table 5 and figure 5, and show no discontinuity at the cutoff throughout the pre-NCLB period. Between the year that NCLB was signed into law and the first year the policy took effect (i.e., between 2001–02 and 2002–03), there was a large, negative discontinuity at the cutoff in the likelihood that black teachers left teaching, although this discontinuity could not be statistically distinguished from zero. This discontinuity persisted, and in most specifications grew in magnitude, between the first year that subgroup-specific accountability was implemented in schools (2002–03) and the second year of the policy (2003–04), although it could generally not be statistically distinguished from zero, before shrinking again the following year.8

Table 5.
Effects of Accountability for Black Subgroup on One-Year Black Teacher Attrition
NonparametricParametric (cubic)Parametric (quadratic)Parametric (linear)
234102030102030102030
Pre-NCLB
98—99 to 99—00 −0.151 −0.066 0.044 0.006 −0.066 −0.067 −0.096 −0.102* −0.013 −0.062 −0.035 
 (0.204) (0.146) (0.176) (0.101) (0.077) (0.107) (0.067) (0.059) (0.069) (0.048) (0.046) 
99—00 to 00—01 −0.002 −0.151 −0.024 0.118 0.079 −0.025 0.142 −0.008 −0.019 −0.020 −0.013 0.032 
 (0.108) (0.181) (0.140) (0.235) (0.129) (0.093) (0.160) (0.078) (0.066) (0.072) (0.052) (0.048) 
00—01 to 01—02 −0.020 −0.129 −0.096 −0.109 −0.050 −0.057 −0.085 −0.022 −0.047 −0.022 −0.031 0.024 
 (0.141) (0.258) (0.182) (0.263) (0.163) (0.127) (0.180) (0.107) (0.086) (0.102) (0.067) (0.058) 
Post-NCLB 
01—02 to 02—03 −0.205* −0.107 −0.259 −0.340* −0.193 −0.177* −0.156 −0.121 −0.115 −0.097 −0.078 −0.041 
 (0.124) (0.248) (0.172) (0.184) (0.127) (0.103) (0.134) (0.092) (0.078) (0.089) (0.062) (0.056) 
02—03 to 03—04 −0.274* −0.478 −0.499** −0.522 −0.266 −0.139 −0.422 −0.095 −0.129 −0.088 −0.074 −0.009 
 (0.154) (0.299) (0.202) (0.429) (0.259) (0.191) (0.271) (0.154) (0.117) (0.146) (0.084) (0.071) 
03—04 to 04—05 0.004 0.293 −0.063 −0.124 −0.063 −0.020 −0.043 0.032 −0.036 −0.007 −0.019 0.045 
 (0.102) (0.228) (0.136) (0.202) (0.131) (0.111) (0.139) (0.088) (0.074) (0.080) (0.057) (0.054) 
NonparametricParametric (cubic)Parametric (quadratic)Parametric (linear)
234102030102030102030
Pre-NCLB
98—99 to 99—00 −0.151 −0.066 0.044 0.006 −0.066 −0.067 −0.096 −0.102* −0.013 −0.062 −0.035 
 (0.204) (0.146) (0.176) (0.101) (0.077) (0.107) (0.067) (0.059) (0.069) (0.048) (0.046) 
99—00 to 00—01 −0.002 −0.151 −0.024 0.118 0.079 −0.025 0.142 −0.008 −0.019 −0.020 −0.013 0.032 
 (0.108) (0.181) (0.140) (0.235) (0.129) (0.093) (0.160) (0.078) (0.066) (0.072) (0.052) (0.048) 
00—01 to 01—02 −0.020 −0.129 −0.096 −0.109 −0.050 −0.057 −0.085 −0.022 −0.047 −0.022 −0.031 0.024 
 (0.141) (0.258) (0.182) (0.263) (0.163) (0.127) (0.180) (0.107) (0.086) (0.102) (0.067) (0.058) 
Post-NCLB 
01—02 to 02—03 −0.205* −0.107 −0.259 −0.340* −0.193 −0.177* −0.156 −0.121 −0.115 −0.097 −0.078 −0.041 
 (0.124) (0.248) (0.172) (0.184) (0.127) (0.103) (0.134) (0.092) (0.078) (0.089) (0.062) (0.056) 
02—03 to 03—04 −0.274* −0.478 −0.499** −0.522 −0.266 −0.139 −0.422 −0.095 −0.129 −0.088 −0.074 −0.009 
 (0.154) (0.299) (0.202) (0.429) (0.259) (0.191) (0.271) (0.154) (0.117) (0.146) (0.084) (0.071) 
03—04 to 04—05 0.004 0.293 −0.063 −0.124 −0.063 −0.020 −0.043 0.032 −0.036 −0.007 −0.019 0.045 
 (0.102) (0.228) (0.136) (0.202) (0.131) (0.111) (0.139) (0.088) (0.074) (0.080) (0.057) (0.054) 

Notes: Each cell reports the estimated discontinuity in the probability that a teacher in a school with 40 black students (North Carolina's minimum subgroup size) left public school teaching in North Carolina. Each cell reports the results from a separate regression. Nonparametric estimates are from local linear regressions in the designated bandwidth around the cutoff, using a triangle kernel. Parametric analyses are linear probability models that include the given terms of the assignment variable (the number of tested black students), as well as the interaction of each polynomial with the treatment dummy. Nonparametric bandwidths of 2, 3, and 4 are near the optimal bandwidth given by Imbens and Kalyanaraman (2012) procedure, which ranges between 2 and 3 for these regressions. Standard errors for parametric analyses are clustered at the school level. Analyses do not include covariates.

*p < 0.10; **p < 0.05.

Figure 5.

Discontinuities in One-Year Black Teacher Attrition at the Subgroup-Specific Accountability Cutoff Across the Pre- and Post-NCLB Periods.

Figure 5.

Discontinuities in One-Year Black Teacher Attrition at the Subgroup-Specific Accountability Cutoff Across the Pre- and Post-NCLB Periods.

Results for black teacher turnover (i.e., leaving the school, as opposed to leaving teaching) show a similar pattern as those for black teacher attrition—after NCLB took effect there was a large, negative discontinuity in the likelihood that black teachers left their schools at the cutoff of 40 tested black students. Analyses limited to black teachers who remained in public school teaching in North Carolina the following year, however, found no discontinuity in black teacher turnover at the cutoff, suggesting that the turnover findings were driven by black teachers who left teaching altogether.

Because schools received their AYP results in the summer of 2003 (North Carolina Department of Public Instruction 2003; Silberman 2003), it was also possible that the black subgroup's contribution to the AYP status of the school may have impacted black teacher attrition. To examine this possibility, a series of difference-in-differences analyses comparing the attrition of black and white teachers in the pre- and post-NCLB periods was conducted. These analyses were conducted separately for schools on each side of the cutoff, and for schools that made or did not make AYP, both overall and for their black subgroups. These analyses found no significant differences in black teacher attrition, except in schools that were held accountable for their black subgroups and failed to make overall AYP. In those schools, black teacher attrition increased in the post-NCLB period compared with white teachers, but only in schools where the black subgroup did not contribute to the overall AYP failure (see table 6). The results of these difference-in-differences analyses are suggestive, and should not be interpreted causally, but do suggest black students’ performance—at least in schools that failed to make overall AYP—could have played a role in black teacher attrition.

Table 6.
Difference-in-Differences Analyses of Teacher Attrition in North Carolina Schools Accountable for Black Subgroup that Failed to Make Overall AYP in 2002—03
Pre-NCLB (99—00 to 01—02)Post-NCLB (01—02 to 03—04)Differences in Differences
Black subgroup did not make AYPa    
Black teacher attrition 0.274 0.276  
White teacher attrition 0.289 0.302  
Difference (black — white) −0.015 −0.026 −0.012 
    
Black subgroup made AYPa    
Black teacher attrition 0.205 0.258  
White teacher attrition 0.265 0.245  
Difference (black — white) −0.061*** 0.013 0.073*** 
Pre-NCLB (99—00 to 01—02)Post-NCLB (01—02 to 03—04)Differences in Differences
Black subgroup did not make AYPa    
Black teacher attrition 0.274 0.276  
White teacher attrition 0.289 0.302  
Difference (black — white) −0.015 −0.026 −0.012 
    
Black subgroup made AYPa    
Black teacher attrition 0.205 0.258  
White teacher attrition 0.265 0.245  
Difference (black — white) −0.061*** 0.013 0.073*** 

Notes: Standard errors clustered at school level. AYP: Adequate Yearly Progress.

aIn 2002—03, the first year of NCLB.

***p < 0.01.

Robustness Checks

Excluding Schools Just Below the Cutoff

The effects on black teachers of subgroup-specific accountability for the black subgroup appeared to be driven by teachers in schools with 38 or 39 tested black students. Because prior research has found effects of falling just short of cutoffs (Papay, Murnane, and Willett 2010; Domina, Penner, and Penner 2016), this could represent the effect on black teachers of their schools falling just short of the cutoff for accountability for the black subgroup. However, as noted earlier, these schools are also those that could have potentially manipulated their numbers of tested black students to push themselves just below the cutoff. To test the robustness of the findings to the exclusion of teachers in these schools, these teachers were excluded from some analyses to determine their effects on the results. When teachers in schools with 39 tested black students were excluded, and the discontinuity in teacher attrition between black teachers in schools with 38 tested black students and black teachers in schools with 40 tested black students was estimated, the estimates of the discontinuity decreased in magnitude and were no longer statistically significant, although they generally retained the negative sign (in parametric analyses using a bandwidth of 30, e.g., β = −0.158, SE = 0.145). When schools with 38 tested black students were additionally excluded from the analysis, estimating the discontinuity in attrition between black teachers in schools with 37 tested black students and schools with 40, the discontinuity decreased in magnitude to nearly zero and was not statistically significant (in parametric analyses using a bandwidth of 30, β = −0.009, SE = 0.150).

Testing Pseudo-cutoffs

Because there were relatively few black teachers in the data, these teachers’ attrition varied a great deal across schools with various numbers of tested black students, making it possible that this study's findings were simply an artifact of random variation. A common falsification test to examine whether RD findings are unique to the cutoff is to conduct RD analyses using false cutoffs, and examine whether discontinuities exist at these values of the assignment variable. Following the procedure outlined by Imbens and Lemieux (2008), separate analyses were conducted above and below the true cutoffs—so that the discontinuity at the true cutoff would not influence the estimates—using the median value of the assignment variable in the above-cutoff and below-cutoff samples as the false cutoffs. The same full range of parametric and nonparametric analyses described earlier were conducted, and these analyses found no evidence of discontinuities at any of the false cutoffs in either the pre- or post-NCLB periods.9

Including Controls

The inclusion of covariates, although not necessary to obtain unbiased regression discontinuity estimates (Lee and Lemieux 2010), can increase the precision of the estimated discontinuity, and can also provide a useful robustness test. If the estimated discontinuity decreases in magnitude or becomes statistically insignificant with the addition of covariates, the observed discontinuity may simply be an artifact of selection of units to one side or another of the cutoff (Bloom 2012). To assess this possibility, table 7 presents results of RD models testing the effects of accountability for the black student subgroup that include the full set of controls from tables 1, 2, and 3. For simplicity of presentation, these models focus on wider bandwidths around the cutoff (20 and 30 tested black students) and the quadratic and linear forms of the running variable. The first model in each panel reproduces the analyses from table 4, which do not include controls; the second model includes the control variables from tables 1, 2, and 3; and the third model in each section includes these controls as well as their interactions with a post-NCLB dummy. Only difference-in-differences coefficients are presented.

Table 7.
Effects of Accountability for Black Subgroup on Two-Year Attrition for Black and White Teachers, with Controls
20, Quadratic30, Quadratic20, Linear30, Linear
Panel A: Black Teachers
Difference −0.201* −0.237** −0.345*** −0.197** −0.209** −0.299*** −0.138* −0.154* −0.210** −0.129* −0.113 −0.144* 
 (0.114) (0.115) (0.122) (0.098) (0.102) (0.109) (0.080) (0.082) (0.089) (0.071) (0.073) (0.076) 
Controls     
Controls × post         
N observations (teacher × year) 1,471 1,471 1,471 2,008 2,008 2,008 1,471 1,471 1,471 2,008 2,008 2,008 
Panel B: White Teachers 
Difference 0.012 0.028 0.029 0.028 0.029 0.040 0.029 0.031 0.036 0.010 0.017 0.019 
 (0.046) (0.045) (0.047) (0.039) (0.037) (0.040) (0.032) (0.030) (0.032) (0.027) (0.025) (0.025) 
Controls     
Controls × post         
N observations (teacher × year) 16,362 16,362 16,362 21,934 21,934 21,934 16,362 16,362 16,362 21,934 21,934 21,934 
20, Quadratic30, Quadratic20, Linear30, Linear
Panel A: Black Teachers
Difference −0.201* −0.237** −0.345*** −0.197** −0.209** −0.299*** −0.138* −0.154* −0.210** −0.129* −0.113 −0.144* 
 (0.114) (0.115) (0.122) (0.098) (0.102) (0.109) (0.080) (0.082) (0.089) (0.071) (0.073) (0.076) 
Controls     
Controls × post         
N observations (teacher × year) 1,471 1,471 1,471 2,008 2,008 2,008 1,471 1,471 1,471 2,008 2,008 2,008 
Panel B: White Teachers 
Difference 0.012 0.028 0.029 0.028 0.029 0.040 0.029 0.031 0.036 0.010 0.017 0.019 
 (0.046) (0.045) (0.047) (0.039) (0.037) (0.040) (0.032) (0.030) (0.032) (0.027) (0.025) (0.025) 
Controls     
Controls × post         
N observations (teacher × year) 16,362 16,362 16,362 21,934 21,934 21,934 16,362 16,362 16,362 21,934 21,934 21,934 

Notes: Each cell reports the estimated discontinuity in the probability that a teacher in a school with 40 black students (North Carolina's minimum subgroup size) left public school teaching in North Carolina. Each cell reports the results from a separate regression. Controls are those listed in tables 1, 2, and 3. Analyses are linear probability models that include the given terms of the assignment variable (the number of tested black students), as well as the interaction of each polynomial with the treatment dummy. Standard errors are clustered at the school level.

*p < 0.10; **p < 0.05; ***p < 0.01.

Results presented in table 7 show that the inclusion of control variables did not significantly affect the estimates of the discontinuity at the cutoff. In the models with controls, the magnitude of the estimated discontinuity was generally larger than that in the models without controls, although the estimates from all three sets of models could not be statistically distinguished from one another with 95 percent confidence. (The same was true for the year-by-year analyses of black teacher attrition.) This suggests the discontinuity at the cutoff in the main analyses was not driven by differences in the covariates between schools just above and just below the cutoff.

The inclusion of controls, however, did not diminish the dependence of the results on schools with 38 and 39 tested black students. With the full set of controls included in the models, the exclusion of schools with 39 tested black students (or of schools with 38 or 39 tested black students) decreased the magnitude of the discontinuity in black teacher attrition at the cutoff and rendered it statistically insignificant, as in prior analyses. The fact that these schools remain highly influential to the results suggests that unique circumstances in these schools may account for the results. The following examination of potential explanations for these findings, therefore, both includes and excludes this group of schools in order to determine their impact on the results.

5.  Exploratory Analyses of Potential Explanations

Because black elementary school teachers taught greater percentages of black students than white teachers in their schools, it was possible that black teachers responded strongly to accountability for the black subgroup simply because they worked with these students more than other teachers.10 To explore this possibility, the RD analyses for black teachers in equations 1 and 2 were interacted with the percentage of black students in black teachers’ classes (both as a continuous and three-level categorical variable). Results showed no evidence of differential effects of the policy depending on the percentage of black students in black teachers’ classes.

Other analyses examined whether the policy caused changes in black teachers’ teaching assignments. Although these analyses did not shed light on the aspects of subgroup-specific accountability that caused black teachers to leave teaching, they did enable an exploration of the changes in black teachers’ working conditions that were caused by the policy, providing suggestive evidence of what might have caused these teachers to leave. These analyses examined black teachers’ class sizes, whether they taught self-contained classes or a tested grade, and the percentage of their students who were black; they also compared black teachers’ classes in each year to their prior year's classes, examining whether they taught fewer students, a different grade, were moved into a tested grade, or taught a greater percentage of black students compared to the prior year. Each of these characteristics of black teachers’ teaching assignments was examined using a full set of nonparametric and parametric RD analyses, using the bandwidths and functional forms outlined earlier, for two years prior to NCLB and two years after the policy took effect.

In 2003–04, the year after schools were first held accountable for their black students’ performance, black teachers in schools that fell just above the cutoff for the number of tested black students the prior year taught significantly smaller percentages of black students than black teachers in schools that fell just below the cutoff. Results from this analysis are presented in the top row of table 8, where the estimates are consistently negative and—particularly in the parametric RD models—statistically significant. The magnitude of these estimates suggests that in above-cutoff schools, black teachers’ classes were composed of about 12 percentage points fewer black students than the classes of black teachers in schools just below the cutoff. In prior years, including the first year of NCLB (2002–03), the discontinuities in the percentage of black students in black teachers’ classes at the cutoff were imprecisely measured and not generally statistically significant. This suggests that in 2003–04 there were changes in black teachers’ teaching assignments caused by accountability for the black subgroup the prior year. These findings are robust to the inclusion of the full set of teacher and school covariates shown in tables 1, 2, and 3. Similarly, table 9 shows that when schools with 38 or 39 tested black students were excluded from the analyses, the results remained consistent in magnitude and direction.

Table 8.
Effects of Accountability for Black Subgroup on Classroom Characteristics for Black Teachers
NonparametricParametric (cubic)Parametric (quadratic)Parametric (linear)
234102030102030102030
Percent Black Students in Teachers’ Class(es)
2003—04 −0.101* −0.074 −0.106* −0.110 −0.164** −0.183** −0.161** −0.138** −0.107* −0.163*** −0.071 −0.025 
 (0.043) (0.136) (0.062) (0.114) (0.083) (0.082) (0.080) (0.066) (0.058) (0.057) (0.047) (0.053) 
2002—03 −0.385*** −0.759*** −0.560*** −0.634* −0.341 −0.231 −0.418* −0.185 −0.145 −0.190 −0.090 −0.055 
 (0.093) (0.193) (0.132) (0.351) (0.223) (0.171) (0.244) (0.140) (0.105) (0.123) (0.075) (0.067) 
2001—02 −0.449*** −0.374*** −0.410 −0.205 −0.125 −0.276 −0.103 −0.082 −0.097 −0.042 −0.021 
 (0.166) (0.120) (0.314) (0.187) (0.144) (0.206) (0.116) (0.091) (0.105) (0.066) (0.060) 
2000—01 −0.102 −0.385** −0.178 −0.145 −0.051 −0.031 −0.036 −0.030 −0.027 −0.037 −0.015 −0.013 
 (0.102) (0.177) (0.134) (0.344) (0.198) (0.144) (0.229) (0.118) (0.093) (0.109) (0.068) (0.060) 
N (2003—04) 91 127 169 322 702 986 322 702 986 322 702 986 
Taught Greater Percentage Black Students than Prior Year 
2003—04 −0.321 −0.230 −0.234 −0.387 −0.304 −0.280** −0.258 −0.246** −0.187* −0.199* −0.144* −0.158** 
 (0.218) (0.426) (0.283) (0.335) (0.196) (0.140) (0.226) (0.120) (0.098) (0.114) (0.079) (0.069) 
2002—03 0.177 −0.039 −0.012 −0.037 −0.161 0.015 −0.155 −0.147 −0.132 −0.140 −0.132* 
 (0.339) (0.239) (0.281) (0.179) (0.151) (0.184) (0.133) (0.109) (0.127) (0.088) (0.074) 
2001—02 −0.081 −0.200 −0.153 −0.354 −0.301* −0.101 −0.402** −0.096 −0.066 −0.098 −0.010 −0.015 
 (0.193) (0.360) (0.248) (0.248) (0.167) (0.141) (0.180) (0.125) (0.104) (0.116) (0.086) (0.074) 
2000—01 0.470 0.492** 0.582** 0.392** 0.101 0.418** 0.074 0.069 0.134 0.042 0.042 
 (0.289) (0.202) (0.236) (0.160) (0.143) (0.165) (0.126) (0.106) (0.114) (0.084) (0.073) 
N (2003—04) 78 109 146 274 587 822 274 587 822 274 587 822 
NonparametricParametric (cubic)Parametric (quadratic)Parametric (linear)
234102030102030102030
Percent Black Students in Teachers’ Class(es)
2003—04 −0.101* −0.074 −0.106* −0.110 −0.164** −0.183** −0.161** −0.138** −0.107* −0.163*** −0.071 −0.025 
 (0.043) (0.136) (0.062) (0.114) (0.083) (0.082) (0.080) (0.066) (0.058) (0.057) (0.047) (0.053) 
2002—03 −0.385*** −0.759*** −0.560*** −0.634* −0.341 −0.231 −0.418* −0.185 −0.145 −0.190 −0.090 −0.055 
 (0.093) (0.193) (0.132) (0.351) (0.223) (0.171) (0.244) (0.140) (0.105) (0.123) (0.075) (0.067) 
2001—02 −0.449*** −0.374*** −0.410 −0.205 −0.125 −0.276 −0.103 −0.082 −0.097 −0.042 −0.021 
 (0.166) (0.120) (0.314) (0.187) (0.144) (0.206) (0.116) (0.091) (0.105) (0.066) (0.060) 
2000—01 −0.102 −0.385** −0.178 −0.145 −0.051 −0.031 −0.036 −0.030 −0.027 −0.037 −0.015 −0.013 
 (0.102) (0.177) (0.134) (0.344) (0.198) (0.144) (0.229) (0.118) (0.093) (0.109) (0.068) (0.060) 
N (2003—04) 91 127 169 322 702 986 322 702 986 322 702 986 
Taught Greater Percentage Black Students than Prior Year 
2003—04 −0.321 −0.230 −0.234 −0.387 −0.304 −0.280** −0.258 −0.246** −0.187* −0.199* −0.144* −0.158** 
 (0.218) (0.426) (0.283) (0.335) (0.196) (0.140) (0.226) (0.120) (0.098) (0.114) (0.079) (0.069) 
2002—03 0.177 −0.039 −0.012 −0.037 −0.161 0.015 −0.155 −0.147 −0.132 −0.140 −0.132* 
 (0.339) (0.239) (0.281) (0.179) (0.151) (0.184) (0.133) (0.109) (0.127) (0.088) (0.074) 
2001—02 −0.081 −0.200 −0.153 −0.354 −0.301* −0.101 −0.402** −0.096 −0.066 −0.098 −0.010 −0.015 
 (0.193) (0.360) (0.248) (0.248) (0.167) (0.141) (0.180) (0.125) (0.104) (0.116) (0.086) (0.074) 
2000—01 0.470 0.492** 0.582** 0.392** 0.101 0.418** 0.074 0.069 0.134 0.042 0.042 
 (0.289) (0.202) (0.236) (0.160) (0.143) (0.165) (0.126) (0.106) (0.114) (0.084) (0.073) 
N (2003—04) 78 109 146 274 587 822 274 587 822 274 587 822 

Notes: Each cell reports the estimated discontinuity in the outcome for black teachers in a school with 40 black students (North Carolina's minimum subgroup size) in 2002—03, the first year of NCLB. Each cell reports the results from a separate regression. Nonparametric estimates are from local linear regressions in the designated bandwidth around the cutoff, using a triangle kernel. Parametric analyses are linear probability models that include the given terms of the assignment variable (the number of tested black students), as well as the interaction of each polynomial with the treatment dummy. Nonparametric bandwidths of 2, 3, and 4 are near the optimal bandwidth given by Imbens and Kalyanaraman (2012) procedure, which ranges between 2 and 3 for these regressions. Standard errors for parametric analyses are clustered at the school level. Analyses do not include covariates.

*p < 0.10; **p < 0.05; ***p < 0.01.

Table 9.
Effects of Accountability for Black Subgroup on Classroom Characteristics for Black Teachers, Excluding Schools Just Below the Cutoff
QuadraticLinear
Bandwidth 20Bandwidth 30Bandwidth 20Bandwidth 30
Allw/o 39w/o 38, 39Allw/o 39w/o 38, 39Allw/o 39w/o 38, 39Allw/o 39w/o 38, 39
Percent Black Students in Teachers’ Class(es)
2003—04 −0.138** −0.134 −0.165* −0.107* −0.111* −0.135* −0.071 −0.071 −0.085 −0.025 −0.020 −0.027 
 (0.066) (0.082) (0.095) (0.058) (0.066) (0.073) (0.047) (0.051) (0.055) (0.053) (0.056) (0.058) 
2002—03 −0.185 −0.199 −0.230 −0.145 −0.156 −0.179 −0.090 −0.095 −0.108 −0.055 −0.057 −0.066 
 (0.140) (0.158) (0.166) (0.105) (0.115) (0.121) (0.075) (0.080) (0.083) (0.067) (0.070) (0.072) 
2001—02 −0.103 −0.118 −0.139 −0.082 −0.093 −0.109 −0.042 −0.048 −0.057 −0.021 −0.025 −0.031 
 (0.116) (0.128) (0.135) (0.091) (0.099) (0.103) (0.066) (0.069) (0.072) (0.060) (0.063) (0.065) 
2000—01 −0.030 −0.043 −0.052 −0.027 −0.038 −0.045 −0.015 −0.021 −0.026 −0.013 −0.017 −0.021 
 (0.118) (0.129) (0.134) (0.093) (0.100) (0.103) (0.068) (0.071) (0.072) (0.060) (0.063) (0.064) 
N (2003—04) 702 692 685 986 976 969 702 692 685 986 976 969 
Taught Greater Percentage Black Students than Prior Year 
2003—04 −0.246** −0.282** −0.313** −0.187* −0.201** −0.214* −0.144* −0.147* −0.149* −0.158** −0.158** −0.157** 
 (0.120) (0.124) (0.136) (0.098) (0.101) (0.109) (0.079) (0.081) (0.085) (0.069) (0.071) (0.074) 
2002—03 −0.155 −0.163 −0.169 −0.147 −0.150 −0.153 −0.140 −0.143 −0.145 −0.132* −0.134* −0.136* 
 (0.133) (0.142) (0.154) (0.109) (0.114) (0.122) (0.088) (0.090) (0.095) (0.074) (0.076) (0.079) 
2001—02 −0.096 −0.058 −0.093 −0.066 −0.037 −0.062 −0.010 0.013 0.001 −0.015 0.004 −0.006 
 (0.125) (0.140) (0.152) (0.104) (0.113) (0.122) (0.086) (0.090) (0.095) (0.074) (0.077) (0.080) 
2000—01 0.074 0.056 0.058 0.069 0.056 0.058 0.042 0.034 0.036 0.042 0.037 0.039 
 (0.126) (0.144) (0.151) (0.106) (0.117) (0.122) (0.084) (0.090) (0.093) (0.073) (0.077) (0.079) 
N (2003—04) 587 581 574 822 816 809 587 581 574 822 816 809 
QuadraticLinear
Bandwidth 20Bandwidth 30Bandwidth 20Bandwidth 30
Allw/o 39w/o 38, 39Allw/o 39w/o 38, 39Allw/o 39w/o 38, 39Allw/o 39w/o 38, 39
Percent Black Students in Teachers’ Class(es)
2003—04 −0.138** −0.134 −0.165* −0.107* −0.111* −0.135* −0.071 −0.071 −0.085 −0.025 −0.020 −0.027 
 (0.066) (0.082) (0.095) (0.058) (0.066) (0.073) (0.047) (0.051) (0.055) (0.053) (0.056) (0.058) 
2002—03 −0.185 −0.199 −0.230 −0.145 −0.156 −0.179 −0.090 −0.095 −0.108 −0.055 −0.057 −0.066 
 (0.140) (0.158) (0.166) (0.105) (0.115) (0.121) (0.075) (0.080) (0.083) (0.067) (0.070) (0.072) 
2001—02 −0.103 −0.118 −0.139 −0.082 −0.093 −0.109 −0.042 −0.048 −0.057 −0.021 −0.025 −0.031 
 (0.116) (0.128) (0.135) (0.091) (0.099) (0.103) (0.066) (0.069) (0.072) (0.060) (0.063) (0.065) 
2000—01 −0.030 −0.043 −0.052 −0.027 −0.038 −0.045 −0.015 −0.021 −0.026 −0.013 −0.017 −0.021 
 (0.118) (0.129) (0.134) (0.093) (0.100) (0.103) (0.068) (0.071) (0.072) (0.060) (0.063) (0.064) 
N (2003—04) 702 692 685 986 976 969 702 692 685 986 976 969 
Taught Greater Percentage Black Students than Prior Year 
2003—04 −0.246** −0.282** −0.313** −0.187* −0.201** −0.214* −0.144* −0.147* −0.149* −0.158** −0.158** −0.157** 
 (0.120) (0.124) (0.136) (0.098) (0.101) (0.109) (0.079) (0.081) (0.085) (0.069) (0.071) (0.074) 
2002—03 −0.155 −0.163 −0.169 −0.147 −0.150 −0.153 −0.140 −0.143 −0.145 −0.132* −0.134* −0.136* 
 (0.133) (0.142) (0.154) (0.109) (0.114) (0.122) (0.088) (0.090) (0.095) (0.074) (0.076) (0.079) 
2001—02 −0.096 −0.058 −0.093 −0.066 −0.037 −0.062 −0.010 0.013 0.001 −0.015 0.004 −0.006 
 (0.125) (0.140) (0.152) (0.104) (0.113) (0.122) (0.086) (0.090) (0.095) (0.074) (0.077) (0.080) 
2000—01 0.074 0.056 0.058 0.069 0.056 0.058 0.042 0.034 0.036 0.042 0.037 0.039 
 (0.126) (0.144) (0.151) (0.106) (0.117) (0.122) (0.084) (0.090) (0.093) (0.073) (0.077) (0.079) 
N (2003—04) 587 581 574 822 816 809 587 581 574 822 816 809 

Notes: Each cell reports the estimated discontinuity in the outcome for black teachers in a school with 40 black students (North Carolina's minimum subgroup size) in 2002—03, the first year of NCLB. Each cell reports the results from a separate regression. Analyses are linear probability models that include the given terms of the assignment variable (the number of tested black students), as well as the interaction of each polynomial with the treatment dummy. Standard errors are clustered at the school level. Analyses do not include covariates.

*p < 0.10; **p < 0.05.

Results from analyses including all schools are depicted visually in figure 6. In the two years prior to NCLB, there are no discontinuities in the percentage of black students in black teachers’ classes at the cutoff. The year that NCLB first took effect (2002–03), the imprecisely estimated discontinuity at the cutoff appears to be driven by a particularly high value for schools with 39 tested black students. In the first year after schools faced accountability for their black subgroups (2003–04), however, the figure shows a clear discontinuity at the cutoff in the percentage of black students assigned to black teachers. Subgroup-specific accountability clearly caused changes in the way that black students were assigned to black teachers.

Figure 6.

Discontinuities in the Percentage of Black Students in Black Teachers’ Classes.

Figure 6.

Discontinuities in the Percentage of Black Students in Black Teachers’ Classes.

There was also a sizeable discontinuity in the likelihood that black teachers taught a greater percentage of black students than the prior year at the cutoff, as can be seen in the bottom panel of table 8. This discontinuity was not consistently statistically significant across all bandwidths and functional forms, but was large and negative in 2003–04, and increased a great deal relative to the prior year (2002–03). Along with the results for the percentage of black students in black teachers’ classes, this suggests that subgroup-specific accountability for the black subgroup led to a shifting of black students away from black teachers in above-cutoff schools, or a shifting of black students to black teachers in below-cutoff schools, in the year after the policy took effect. It is important to note that these shifts in the teaching assignments of black teachers happened after the policy caused the attrition of black teachers outlined previously. For this reason, these changes in black teachers’ teaching assignments are not necessarily explanations for the effects on black teacher attrition; instead, they can be seen as evidence that the dynamics of black teachers’ teaching assignments were different in above-cutoff versus below-cutoff schools after the policy took effect.

Other analyses examined whether schools that were accountable for the black subgroup in the first year of NCLB subsequently hired more or fewer black teachers than schools that were not accountable for the black subgroup's performance. These analyses found no pattern of significant discontinuities in the number of newly hired black teachers, or the percentage of newly hired teachers who were black, at the cutoff in the pre-NCLB period or in any of the three years after the policy took effect.

6.  Discussion

The first section of this study proposed four mechanisms that might explain any effects of subgroup-specific accountability on teacher turnover and attrition: job pressure, resource allocation, teacher motivation, and strategic manipulation. Of these mechanisms, job pressure is the least consistent with the results presented here. The job pressure explanation predicts that accountability for the black subgroup would increase the pressure on teachers in above-cutoff schools and increase those teachers’ turnover or attrition; instead, these analyses find that accountability for the black subgroup caused black teachers in above-cutoff schools to leave teaching at significantly lower rates than black teachers in below-cutoff schools. The resource allocation explanation is more consistent with these results, since it predicts that teacher turnover and attrition would decrease in above-cutoff schools; however, this explanation also predicts that teachers who work most closely with the focal subgroup would be the most affected by the policy, which these analyses do not find to be the case.

The motivation explanation is an intriguing possibility, particularly given that black teachers were affected by accountability for the same-race subgroup. Seeing that the black students “counted” in their schools, and that their schools were taking action to address the achievement gap between black and white students, may have caused black teachers who might otherwise have left to remain in teaching. In below-cutoff schools, black teachers might have been discouraged by their schools falling just short of those cutoffs, and chosen to leave teaching. Prior research has documented effects of falling just short of strict cutoffs for various policies (Papay, Murnane, and Willett 2010; Domina, Penner, and Penner 2016), and it is possible that the increased attrition of black teachers from schools that fell just short of accountability for their black subgroups represents exactly such an effect. Prior research also suggests that minority workers’ perceptions of the “diversity climates” of their workplaces strongly predict both their commitment to their organizations and their turnover (Griffeth and Hom 2001; Foley, Kidder, and Powell 2002; Chrobot-Mason 2003). The specific mechanisms by which accountability for the black subgroup affected black teachers is challenging to investigate using administrative data, however, and surveys of teachers or qualitative research will likely be necessary to more fully explore these effects.

The final potential mechanism discussed at the outset of this study, strategic manipulation, is also consistent with the results presented here. The effects of accountability for the black subgroup appear concentrated in schools with 38 or 39 tested black students, the very schools where such manipulation could have occurred. Although teachers may have not had knowledge of where their schools fell in relation to the cutoff, their principals could have had knowledge of both this cutoff and their schools’ numbers of black students eligible for testing, as well as the opportunity to manipulate their schools’ numbers of tested black students, and teachers may have observed this manipulation.

This study's findings on changes in black teachers’ class assignments, however, demonstrate that the policy had significant effects on aspects of black teachers’ working conditions. The year after subgroup-specific accountability was first implemented, black teachers in schools that had been held accountable for the black subgroup were assigned significantly smaller percentages of black students than black teachers in schools just below the cutoff. This is consistent with black students being shifted away from black teachers in schools where the black subgroup suddenly counted toward AYP. These results, in contrast to the results for turnover and attrition, are not sensitive to the exclusion of schools with 38 or 39 tested black students.

The effects of accountability for the black subgroup on black teacher attrition was cumulative, with some of the effect occurring between the announcement of the policy and its first year of implementation, and these effects continuing in the year between the policy's first and second years. The small, insignificant, negative discontinuity in black teacher attrition at the cutoff between 2001–02 and 2002–03 took place well before the testing that ultimately determined which schools fell above the cutoff and which did not. Given the ambiguities about the timeline of the policy's implementation, however, it is impossible to rule out that schools and teachers had some knowledge of where they would fall in relation to the cutoff in the summer of 2002, and this information could have affected black teacher attrition during that time.

One interesting question raised by these analyses is whether black teacher attrition was voluntary or involuntary. Black teacher attrition could have been driven by teachers’ decisions to remain in or leave public school teaching in North Carolina; alternatively, accountability for the black subgroup could have affected the likelihood that black teachers were let go from their jobs. Analyses not presented here find that black teachers who left teaching from below-cutoff schools after the policy took effect were more likely to be first-year teachers than black teachers who left teaching from above cutoff schools (24 percent versus 4 percent, respectively, p < 0.05). This is consistent with below-cutoff schools letting go of their most inexperienced black teachers or with above-cutoff schools holding onto those teachers.

This study suggests that policies that impose cutoffs and hold some schools accountable for subgroup performance and others not may have unintended consequences for teacher labor markets. Minority teacher attrition is a significant concern for schools and school districts (Ingersoll and May 2011), and understanding the factors driving that attrition—particularly the role of large-scale policies such as NCLB—is an important direction for future research.

Acknowledgments

Work on this paper was primarily completed while the author was at Northwestern University. The research was supported by funding from the Albert Shanker Institute, the National Academy of Education/Spencer Foundation, and the Institute of Education Sciences, U.S. Department of Education (through grant R305B080027 to Northwestern University). The opinions expressed herein do not represent the views of the funders. The author thanks the North Carolina Education Research Data Center at Duke University for providing access to the data; David Figlio, Kirabo Jackson, Randall Reback, Jim Spillane, and two anonymous reviewers for their valuable input; and Kara Bonneau and Bruce Foster for their help with data.

REFERENCES

Bloom
,
Howard S.
2012
.
Modern regression discontinuity analysis
.
Journal of Research on Educational Effectiveness
5
(
1
):
43
82
. doi:10.1080/19345747.2011.578707.
Borman
,
Geoffrey D.
, and
N.
Maritza Dowling
.
2008
.
Teacher attrition and retention: A meta-analytic and narrative review of the research
.
Review of Educational Research
78
(
3
):
367
409
. doi:10.3102/0034654308321455.
Boyd
,
Donald
,
Hamilton
Lankford
,
Susanna
Loeb
, and
James
Wyckoff
.
2008
.
The impact of assessment and accountability on teacher recruitment and retention
.
Public Finance Review
36
(
1
):
88
111
. doi:10.1177/1091142106293446.
Casey
,
Kathleen
.
1993
.
I answer with my life: Life histories of women working for social change
.
New York
:
Routledge
.
Chrobot-Mason
,
Donna L.
2003
.
Keeping the promise: Psychological contract violations for minority employees
.
Journal of Managerial Psychology
18
(
1
):
22
45
. doi:10.1108/02683940310459574.
Clotfelter
,
Charles T.
,
Helen F.
Ladd
,
Jacob L.
Vigdor
, and
Roger Aliaga
Diaz
.
2004
.
Do school accountability systems make it more difficult for low-performing schools to attract and retain high-quality teachers
?
Journal of Policy Analysis and Management
23
(
2
):
251
271
. doi:10.1002/pam.20003.
Crocco
,
Margaret S.
, and
Arthur T.
Costigan
.
2007
.
The narrowing of curriculum and pedagogy in the age of accountability: Urban educators speak out
.
Urban Education
42
(
6
):
512
535
. doi:10.1177/0042085907304964.
Cullen
,
Julie B.
, and
Randall
Reback
.
2006
. Tinkering towards accolades: School gaming under a performance accountability system. In
Improving school accountability (Advances in applied microeconomics, vol. 14)
, edited by
Timothy J.
Gronberg
and
Dennis W.
Jansen
, pp.
1
34
.
Somerville, MA
:
Emerald Group Publishing
.
Davidson
,
Elizabeth
,
Randall
Reback
,
Jonah E
.
Rockoff
, and
Heather L.
Schwartz
.
2015
.
Fifty ways to leave a child behind: Idiosyncrasies and discrepancies in states’ implementation of NCLB
.
Educational Researcher
44
(
6
):
347
358
. doi:10.3102/0013189X15601426.
Dixson
,
Adrienne D.
, and
Jeannine E.
Dingus
.
2008
.
In search of our mother's gardens: Black women teachers and professional socialization
.
Teachers College Record
110
(
4
):
805
837
.
Domina
,
Thurston
,
Andrew M.
Penner
, and
Emily K. Penner
.
2016
.
“Membership has its privileges”: Status incentives and categorical inequality in education
.
Sociological Science
3
(
13
):
264
295
. doi:10.15195/v3.a13.
Feng
,
Li
,
David N.
Figlio
, and
Tim
Sass
.
2010
.
School accountability and teacher mobility
.
NBER Working Paper No. 16070.
Figlio
,
David N.
2006
.
Testing, crime and punishment
.
Journal of Public Economics
90
(
4–5
):
837
851
. doi:10.1016/j.jpubeco.2005.01.003.
Figlio
,
David N.
, and
Lawrence S.
Getzler
.
2006
.
Accountability, ability and disability: Gaming the system?
In
Improving school accountability (Advances in applied microeconomics
, vol.
14
)
edited by
Timothy J.
Gronberg
and
Dennis W.
Jansen
, pp.
35
49
.
Somerville, MA
:
Emerald Group Publishing
.
Foley
,
Sharon
,
Deborah L.
Kidder
, and
Gary N.
Powell
.
2002
.
The perceived glass ceiling and justice perceptions: An investigation of Hispanic law associates
.
Journal of Management
28
(
4
):
471
496
. doi:10.1177/014920630202800401.
Fulton
,
Mary
.
2006
.
Minimum size of subgroups for Adequate Yearly Progress (AYP): State trends and highlights
.
Denver, CO
:
Education Commission of the States
.
Griffeth
,
Rodger W.
, and
Peter W.
Hom
.
2001
.
Retaining valued employees
.
Thousand Oaks, CA
:
Sage
.
Harris
,
Douglas N.
2008
.
Educational outcomes of disadvantaged students: From desegregation to accountability
. In
Handbook of research in education finance and policy
,
edited by
David N.
Figlio
and
Helen F.
Ladd
, pp.
551
570
.
New York
:
Routledge
.
Hickok
,
Eugene
.
2003
.
Decision letters—State accountability plans
.
Available
www2.ed.gov/admins/lead/account/letters/index.html#nc.
Accessed 17 October 2017
.
Imbens
,
Guido
, and
Karthik
Kalyanaraman
.
2012
.
Optimal bandwidth choice for the regression discontinuity estimator
.
Review of Economic Studies
79
(
3
):
933
959
. doi:10.1093/restud/rdr043.
Imbens
,
Guido
, and
Thomas
Lemieux
.
2008
.
Regression discontinuity designs: A guide to practice
.
Journal of Econometrics
142
(
2
):
615
635
. doi:10.1016/j.jeconom.2007.05.001.
Ingersoll
,
Richard
, and
Henry
May
.
2011
.
Recruitment, retention, and the minority teacher shortage
.
CPRE Research Report No. RR-69
.
Philadelphia, PA
:
Consortium for Policy Research in Education
.
Irvine
,
Jacqueline Jordan
.
2002
.
African American teachers’ culturally specific pedagogy
. In
In search of wholeness: African American teachers and their culturally specific classroom practices
,
edited by
Jacqueline Jordan
Irvine,
pp.
139
146
.
New York
:
Palgrave
. doi:10.1057/9780230107182_8.
Jackson
,
C. Kirabo
.
2009
.
Student demographics, teacher sorting, and teacher quality: Evidence from the end of school desegregation
.
Journal of Labor Economics
27
(
2
):
213
256
. doi:10.1086/599334.
Jennings
,
Jennifer L.
, and
Peter M.
Crosta
.
2011
.
The unaccountables
.
Paper presented at the 26th Annual American Education Finance Association Conference, Seattle, WA, March
.
Kane
,
Thomas J.
, and
Douglas O.
Staiger
.
2002
.
Volatility in school test scores: Implications for test-based accountability systems
. In
Brookings papers on education policy
,
edited by
Diane
Ravitch
, pp.
235
283
.
Washington, DC
:
Brookings Institution Press
. doi:10.1353/pep.2002.0010.
Lauen
,
Douglas Lee
, and
S. Michael
Gaddis
.
2012
.
Shining a light or fumbling in the dark? The effects of NCLB's subgroup-specific accountability on student achievement
.
Educational Evaluation and Policy Analysis
34
(
2
):
185
208
. doi:10.3102/0162373711429989.
Lee
,
David S.
, and
Thomas
Lemieux
.
2010
.
Regression discontinuity designs in economics
.
Journal of Economic Literature
48
(
2
):
281
355
. doi:10.1257/jel.48.2.281.
Li
,
Danielle
.
2015
.
School accountability and principal mobility: How No Child Left Behind affects the allocation of school leaders
.
Harvard Business School Working Paper No. 16–052
.
McCrary
,
Justin
.
2008
.
Manipulation of the running variable in the regression discontinuity design: A density test
.
Journal of Econometrics
142
(
2
):
698
714
. doi:10.1016/j.jeconom.2007.05.005.
North Carolina Department of Public Instruction
.
2003
.
ABCs 2003 accountability report background packet
.
Available
www.ncpublicschools.org/abcs/02-03/03mediapacket.pdf.
Retrieved 1 December 2015
.
Papay
,
John P.
,
Richard J.
Murnane
, and
John B.
Willett
.
2010
.
The consequences of high school exit examinations for low-performing urban students: Evidence from Massachusetts
.
Educational Evaluation and Policy Analysis
32
(
1
):
5
23
. doi:10.3102/0162373709352530.
Pierce
,
Jason
.
2003
.
Minimum size of subgroups for Adequate Yearly Progress (AYP)
.
Denver, CO
:
Education Commission of the States
.
Reardon
,
Sean F.
2011
.
The widening academic achievement gap between the rich and the poor: New evidence and possible explanations
. In
Whither opportunity? Rising inequality, schools, and children's life chances
,
edited by
Greg J.
Duncan
and
Richard J.
Murnane
, pp.
91
115
.
New York
:
Russell Sage
.
Reardon
,
Sean F.
,
Erica H.
Greenberg
,
Demetra
Kalogrides
,
Kenneth A.
Shores
, and
Rachel A.
Valentino
.
2013
.
Left behind? The effect of No Child Left Behind on academic achievement gaps
.
Unpublished paper, Stanford University
.
Reardon
,
Sean F.
, and
Joseph P.
Robinson
.
2008
.
Patterns and trends in racial/ethnic and socioeconomic academic achievement gaps
. In
Handbook of research in education finance and policy
,
edited by
David N.
Figlio
and
Helen F.
Ladd
, pp.
497
516
.
New York
:
Routledge
.
Reback
,
Randall
,
Jonah E.
Rockoff
, and
Heather L.
Schwartz
.
2014
.
Under pressure: Job security, resource allocation, and productivity in schools under No Child Left Behind
.
American Economic Journal: Economic Policy
6
(
3
):
207
241
. doi:10.1257/pol.6.3.207.
Reback
,
Randall
,
Jonah E.
Rockoff
,
Heather L.
Schwartz
, and
Elizabeth
Davidson
.
2011
.
Barnard/Columbia No Child Left Behind database, 2002–03 and 2003–04
.
Available
www.gsb.columbia.edu/nclb.
Accessed 8 February 2013
.
Silberman
,
Todd
.
2003
.
Test scores leaping in N.C.
Available
archives.republicans.edlabor.house.gov/archive/issues/108th/education/nclb/dc061903.htm.
Accessed 19 October 2017
.
Sims
,
David P.
2013
.
Can failure succeed? Using racial subgroup rules to analyze the effect of school accountability failure on student performance
.
Economics of Education Review
32
:
262
274
. doi:10.1016/j.econedurev.2012.12.003.

Notes

1. 

The sample was not limited to self-contained teachers but instead included teachers of art, music, and other non–self-contained classes. Eighty-four percent of teachers in the sample, however, taught self-contained classes.

2. 

NCLB allowed each state to define “continuous enrollment” (Davidson et al. 2015), and North Carolina required students to be continuously enrolled in their schools for 140 days at the time of spring testing (Lauen and Gaddis 2012). In the remainder of this study, references to the numbers of students in subgroups refer to the number of such continuously enrolled and tested students.

3. 

For these analyses, the predicted probability of a subgroup making AYP in reading, and (separately) for making AYP in math, was computed by estimating school-by-subgroup logistic regression models where the dependent variable was an indicator for whether the black or white subgroup passed its accountability target in that subject during the first year of NCLB, and the independent variable was the prior year's school-average reading or math test score for that subgroup. Results reported in the following section were robust to using the likelihood of making AYP in either subject.

4. 

Some analyses tested whether accountability for subgroups in the second year of NCLB (2003–04) caused changes in teacher turnover and attrition. These analyses are discussed later.

5. 

For the pre-NCLB analyses, schools’ numbers of tested black or white students from 2002–03 were used as the assignment variable; these analyses were therefore limited to those pre-NCLB schools for which such information was available. Seventy-seven schools (7 percent of all North Carolina elementary schools) and 1,594 teachers (5 percent of elementary teachers) were lost from the pre-NCLB analyses because of the lack of NCLB-related data on their schools.

6. 

Results of these analyses are available upon request.

7. 

Results of these analyses are available upon request.

8. 

Other analyses examined whether accountability for the black student subgroup in 2003–04 (the second year of NCLB) affected black teacher turnover or attrition between 2001–02 and 2004–05, and between 2002–03 and 2004–05. In neither case did accountability in the second year of the policy have a significant effect on outcomes for black teachers.

9. 

Results of these analyses are available upon request.

10. 

The results of regressions using school fixed effects showed that black teachers’ classes had approximately 4 percentage points more black students than the classes of white teachers in their schools. In a class of twenty-five students, this translated to black teachers having, on average, nine black students in their classes, as compared to eight black students for their white colleagues.

Appendix A:  Description of Data

Data used in this study were drawn from two sources. The first was a series of data files on North Carolina teachers and schools, which are available to researchers through the North Carolina Education Research Data Center (NCERDC) at Duke University. The specific data files used in this study were the following (all from years 1999–2005):

  • School Activity Report (SAR) personnel files

  • School Activity Report (SAR) student directory files

  • Student End-of-Grade (EOG) exam files

  • ABC accountability growth files

  • Teacher education files

  • Teacher licensure files

  • Teacher testing files

  • Teacher National Board Certification files

  • Common Core of Data (CCD) Public School Universe (PSU) files

The second source of data for this study was the Barnard/Columbia No Child Left Behind Database. These data are publicly available for download at the following Web site: www8.gsb.columbia.edu/nclb/files#North%20Carolina. For this study, two data files were used: NorthCarolina_2003.dta and NorthCarolina_2004.dta.

Stata do-files used to prepare the data and conduct the analyses are available from the author upon request.