Hold Back To Move Forward? Early Grade Retention And Student Misbehavior:

Accountability remains at the forefront of the education policy debate more than a decade after the No Child Left Behind Act of 2001 was signed into law. The last decade has witnessed the nationwide implementation of an educational system where student test performance plays an important role in high-stakes decisions, such as school closures and teacher retention and compensation. All states have established test-based performance benchmarks for students and meeting these standards is a prerequisite for high school graduation and grade promotion in many states.

Grade retention has been a long-standing and highly debated intervention for low-performing students. The biggest point of contention is the academic benefits of grade retention—that is, whether holding back students who are not ready for more challenging course content translates into higher achievement in the following years.¹ The overarching conclusion of the earlier literature is that retained students perform significantly worse than their promoted peers in the years that follow. On the other hand, more recent studies, which better address the identification challenges by using the nonlinearities in retention policies, show that grade retention, especially in early grades, has a positive impact on test scores in the short term.²

Equally contentious are the possible adverse effects of grade retention policies. Critics of these policies commonly argue that grade retention imposes significant emotional burdens on students because they are stigmatized as failing and they face the challenges of adjusting to new peers, which might in turn lead to student disengagement from schooling. In fact, two recent studies have found evidence that grade retention in eighth grade reduces the likelihood of high school graduation under some conditions (Jacob and Lefgren 2009), whereas early grade retention has no significant impact on student attendance in the following years (Schwerdt and West 2012).

This study explores another way this emotional burden might manifest itself, by examining the possible adverse effects of early grade retention on student disruptive behavior in the years that follow. Using the test-based third-grade promotion policy in Florida, regression discontinuity estimates suggest that “just-retained” students are significantly more likely to have disciplinary problems and receive suspensions in the two years that follow, yet these effects vanish in the long run. Further, subgroup analyses reveal this adverse effect is mostly concentrated among economically-disadvantaged and male students. These findings might help better assess the costs and benefits of grade retention in an era where test-based accountability is becoming the new norm.

The remainder of the paper is organized as follows. Section 2 provides background on the Florida early grade retention policy. Section 3 describes the data and details the empirical strategy, section 4 presents the findings, and section 5 concludes.

The Just Read, Florida! Initiative, enacted in 2001, uses frequent progress monitoring, intensive instructional assistance, and grade retention to ensure all students meet the reading benchmarks described in Florida's Sunshine States Standards before they reach the fourth grade, when students traditionally begin to “read to learn” rather than “learn to read.” Since 2002, all third graders in Florida are categorized into “achievement levels” based on their reading performance in the curriculum standards-based Florida Curriculum Assessment Test (FCAT-SSS). If a student fails to perform at achievement level 2 or higher, the law requires that they should not be promoted to the fourth grade. This discontinuity in grade promotion is the key element of my identification strategy described subsequently.

The legislation requires that schools provide development strategies for retained students. These include proven effective teaching strategies, assigning retained students to high-performing teachers, participation in summer reading camps, and at least ninety minutes of reading instruction each day. If the retained student can demonstrate the required reading level before the beginning of the following school year or during the school year, they might be eligible for mid-year grade promotion.

There are several “good cause exemptions” under which students can be promoted to fourth grade even though they fail the high-stakes reading test. For instance, if a student can demonstrate an acceptable level of performance on an alternative standardized test approved by the State Board of Education, the student is promoted to the next grade. Further, limited English proficiency (LEP) students with less than two years in the English for Speakers of Other Languages program, special education students with certain disabilities, students who show through a teacher-developed portfolio that they can read at grade level, and students who have received intensive reading remediation for two years and who have already been retained twice between kindergarten and third grade are granted the good cause exemption.³

To examine the consequences of grade retention, I utilize student-level administrative data and track seven cohorts of students in Florida entering third grade for the first time between 2003–04 and 2009–10. The data set includes demographic information on students such as race, gender, free or reduced-price lunch (FRPL) eligibility, LEP status, LEP program entry and exit dates, exceptional student education status, and FCAT-SSS scores in reading and math.

More importantly for the purposes of this study, the data set contains detailed information about student disciplinary incidents. In particular, for each incident, I observe the type of disciplinary/referral action taken and the duration of the suspension (if applicable) for at least two years after the students in the sample first enter third grade. These incidents can be triggered by a wide array of student misbehavior, ranging from disruptive behavior in the classroom to gang involvement.⁴ Based on the severity of the incident, teachers and principals have full discretion over the type of action taken, which may include corporal punishment, in-school or out-of-school suspension, placement in a different program, and expulsion.

Table 1 breaks down by grade the incident rates, types of disciplinary action, and days suspended. There are several findings worth highlighting. First, incident rates increase monotonically as grade increases, with a significant jump between elementary and middle school, which might be driven by differences in teacher and principal tolerance toward student misbehavior between grade levels. This trend is also observed in the average suspension days. Second, punishment types differ considerably across grade levels, with more frequent use of corporal punishment and out-of-school suspensions in earlier grades. Finally, suspension is the most frequent form of student punishment, with almost all students involved in disciplinary incidents (80 to 90 percent) receiving in-school or out-of-school suspensions. In the analyses that follow, I am interested in the likelihood of student misbehavior (as measured by the incident indicator), and the severity of the misbehavior (as measured by whether the student received an in-school or out-of-school suspension), noting that the results are similar when the number of suspended days is used as an indicator of severity.

Table 2 presents the descriptive statistics for the entire sample (first column) along with the promoted students with scores below cutoff (second column) and retained students below cutoff (third column). During this time frame, roughly 16 percent of all third graders scored below the retention cutoff and 8 percent were retained, with significantly higher retention rates in the earlier cohorts (roughly 11 percent in the first cohort versus 6 percent in the last). Compared with their promoted peers, retained students are significantly more likely to have disciplinary issues during the third grade, more likely to come from economically disadvantaged families, more likely to belong to a racial/ethnic minority group (other than Asian), less likely to have English as their native language, and more likely to be first generation immigrants. Conditional on low performance on the high-stakes reading test in the third grade, these differences seem to subside considerably, yet the promoted low performers are significantly more likely to be LEP and/or special education students because of the exemption clauses in the policy.

The biggest challenge in revealing the causal impact of grade retention is that the retention decisions are typically made by teachers and principals based on student attributes that are not necessarily observable to the researcher, such as parental involvement and student motivation, which in turn affect future student outcomes. Therefore, regression-adjusted differences based on observable student attributes between promoted and retained students are likely to yield biased inferences. In this study, I utilize the non-linearity created by the retention policy and compare students who scored right below and right above the promotion cutoff in a regression discontinuity framework. In what follows, I detail this empirical approach.

I first check to make sure there is a significant discontinuity in the treatment variable at the cutoff. Figure 1 presents the local linear smoothing of the retention indicator on the relative reading score, calculated separately for each side of the cutoff using the triangle kernel and the bandwidth of 5 points, with the solid circles representing the retention rate for each test score. This figure shows that students who score just below the retention cutoff are approximately 30 percentage points more likely to be retained compared with their peers who scored right at the cutoff. This is true for each cohort with slightly larger discontinuities observed for the third graders in the earlier cohorts.

Figure 2 presents a graphical inspection of the effects of retention on student discipline, replacing the retention indicator in figure 1 with whether the student was involved in a disciplinary incident in the next two years (in the upper panel) or in the past two years (lower panel). Whereas the third graders who scored right below the promotion cutoff were no more likely to have disciplinary issues during the previous two school years than their peers on the other side, they are significantly more likely to be involved in incidents in the following two years. Using the jump in the retention rate at the cutoff displayed in figure 1, the simple Wald estimator given in equation 2 indicates the magnitude of this difference is roughly 4–5 percentage points. This gap approximately corresponds to one fourth of the control mean at the cutoff.

Table 3 presents the short-term effects of grade retention on disciplinary incidents and suspensions in the years following the retention. In the first two columns, I estimate equations 4a and 4b using a bandwidth of 5 points and a linear ⁠, and the last two columns use 20 points and a quartic polynomial. In all specifications, I include cohort fixed-effects to take differences between cohorts into account, although the results are robust to the exclusion of these fixed effects.

The estimated effects reported in columns labeled as (I) align well with the earlier graphical analysis. Grade retention increases the likelihood of disciplinary incidents by about 3 to 5 percentage points (30 to 50 percent of the control mean of 0.107 at the cutoff) in the following year, and roughly by 5 to 6 percentage points (40 to 50 percent of the control mean of 0.132 at the cutoff) in the second year that follows. Just-retained students are also significantly more likely to receive suspensions in the following two years. The estimated effects are positive for both in-school and out-of-school suspensions, but the point estimates are only statistically significant for out-of-school suspensions in the first year and in-school suspensions in the second year. One possible explanation behind this discrepancy is that the retained students are involved in more severe incidents in the first year after they are retained, and thus receive out-of-school suspensions, compared with the second year.

Table 4 explores the effects of grade retention beyond the first two years. In this analysis, I restrict the sample to earlier cohorts (first-time third graders between 2003 and 2006) that are observed for at least six years after third grade. The estimates presented in columns (I) indicate that there are no significant discontinuities at the retention cutoff in the long-run, except for the significant negative discontinuity during the third year after retention. Important to note here, however, is that during that year the majority of the just-promoted students attend middle school whereas their just-retained peers are in elementary school. It is possible, therefore, that the observed discontinuities are reflections of the aforementioned jump in the incident and suspension rates between elementary and middle grades.

The estimates presented so far have relied on the same-age comparisons between retained and promoted students. The primary concern in this approach is that the estimated differences in disciplinary incidents might be caused by the differences in incident rates across grades. Note, however, that the incident rates increase with grade, as reported in table 1. Further, table 5 presents the same-grade comparisons between retained and promoted students around the cutoff. That is, I compare promoted students with their retained peers around the cutoff when they reach the same grade level. Once again, I restrict the sample to the 2003–06 cohorts who are old enough to reach eighth grade by the end of my sample. The findings reinforce the conclusion that the retained students are significantly more likely to have disciplinary problems in the short run, yet these differences dissipate in middle school.

Other than the causal effect of retention, there are several alternative scenarios that might explain the observed discontinuities in disciplinary problems. One of these explanations is the differences in student attributes (e.g., prior disciplinary problems, achievement, demographics, family characteristics, and other observed and unobserved traits) between retained and promoted students around the cutoff. I investigate this possibility by replacing the disciplinary outcomes in equation 4b with student characteristics and check for possible discontinuities. The findings presented in table 6 reject this explanation and show that the students on the two sides of the retention cutoff are comparable along these observed traits. To further examine whether differences in student attributes explain the gaps in disciplinary outcomes at the cutoff, columns labeled as (II) in tables 3, 4, and 5 present the parametric estimates controlling for observed student attributes listed in table 6, along with cohort fixed-effects and within-school average peer outcomes. Tables A.1 and A.2 in the Appendix present the full first- and second-stage results for column II in table 2. The inclusion of these covariates does not seem to significantly change the estimated impact of retention, except for the discontinuities in the third year after retention.

Unlike test scores, disciplinary outcomes are not standardized measures across educational settings. That is, given two identical student behaviors, different disciplinary outcomes might emerge based on factors such as the principal attitude or the school environment. To see whether such differences explain the differences between retained and promoted students, I calculate the percentage of peers involved in disciplinary incidents at the school-year level for each student. If, for instance, retained students are attending schools with stricter principals, one would expect to observe higher peer incident rates for these students. The last five rows of table 5 present the discontinuity estimates along this dimension and show that students on the two sides of the cutoff are attending similar schools. This is not the case for the third year after retention, however, which presents evidence justifying the earlier explanation for the third-year discontinuity in disciplinary problems. In fact, when peer differences are accounted for in column II of table 4, the estimated differences at the cutoff in the third year are no longer statistically significant.

Another concern regarding identification in the RD design in this context, as noted in McCrary (2008), is the possibility of selection variable manipulation (i.e., the reading scores in this case) by teachers and/or principals. Under this scenario, one would expect to see an unusual discontinuity in the test score distribution around the promotion cutoff. It is important to note here that this is very unlikely, because FCAT scores are assessed without any teacher or principal involvement. Regardless, I present graphical evidence to dismiss this possibility, because the formal test developed by McCrary (2008) is not appropriate in this case as it relies on local linear regressions, which might lead to incorrect inferences when the running variable is discrete (Card and Lee 2008). Figure 3 provides the reading score distribution around the cutoff. The number of students in each bin seems to be increasing as the retention cutoff falls on the left tail of the normally distributed reading scores, but the results present no unusual discontinuity at the cutoff and hence no evidence of strategic sorting around the cutoff.

Finally, differential attrition might lead to biased estimates if retained students leave the sample at higher rates than their promoted peers or if retained stayers differ from the promoted stayers in the following years. I first check the attrition rates around the cutoff. The discontinuity estimates presented in table 7 suggest retained students are equally likely to leave the Florida public school system as the promoted students around the cutoff. Second, I compare the just-retained stayers with just-promoted stayers along observable student characteristics. Conditional on staying in the sample during the following two years, comparisons reported in table 8, combined with the results in table 6, suggest the retained leavers are quite similar to promoted students who left the sample in the years that follow.

To check the robustness of these findings, table 9 repeats the main analysis using various bandwidths and polynomial orders, ranging from a bandwidth of 1 and order zero, under which the RD design is equivalent to the traditional instrumental variable framework, to a bandwidth of 15 points and quartic polynomial. In all specifications, I include the aforementioned student covariates, average peer outcome at the school-level, and cohort fixed-effects to improve the precision of the estimates. The estimated discontinuities are positive and statistically different from zero in all but two specifications. The impact sizes are comparable to the ones in the original specifications, ranging from 3 to 7 percent in the first year and 4 to 10 percent in the second year.

I also conduct additional robustness checks using different covariates in the model. Table A.3 presents estimates from regression models where (1) I also control for special education and limited English proficiency indicators interacted with being below the cutoff to account for the exemption clauses in the policy (in column I); and (2) I use school fixed-effects to eliminate time-invariant across-school differences in disciplinary outcomes. The findings are almost identical to the estimates presented in table 3, reinforcing the previous conclusions.

Having provided evidence that grade retention, on average, leads to disruptive behavior, I now check to see whether the estimated effects of grade retention are heterogeneous by observed student characteristics. Table 10 presents the estimated discontinuities in whether the student was involved in a disciplinary incident in the following two years by socioeconomic status in the first panel, by race/ethnicity in the second panel, and by gender in the third panel. All regressions include student covariates, cohort fixed effects, and within-school average peer outcome. The most striking result is that the adverse effect of retention is primarily concentrated among economically disadvantaged students as measured by their FRPL eligibility. Grade retention leads to a 7 to 9 percentage point increase in disciplinary incidents for economically disadvantaged students, whereas it has no statistically significant effect on more affluent students. The estimated discontinuities are positive for all major racial/ethnic groups in Florida, but the largest effect seems to be on African American students. Similarly, grade retention affects students of both genders, but the point estimates are larger for boys.⁵

There are several mechanisms that might explain the adverse effect of grade retention on student behavior. For instance, the observed discontinuity at the retention cutoff might arise if the students who are old for their grade are more likely to misbehave. Controlling for relative age, however, would lead to misleading inferences in the framework outlined herein, because retention is highly correlated with relative age in the years following retention. Therefore, to see if relative age is associated with student misbehavior, I conduct an exploratory analysis where I restrict the sample to all fourth and fifth graders in the sample, the grades during which the negative effects of retention are observed. To account for the possibility that relative age is correlated with unobserved student characteristics, I also restrict the sample to students born in August and September and use the “September-born” indicator as an instrument for relative age (in months), taking advantage of Florida's school starting age policy.⁶ Hence, I exploit the variation in relative age created by the policy, which is presumably exogenous to unobserved student attributes.

The results, which are available upon request, suggest a strong first-stage (students born in September, on average, are six months older than their peers in the same grade), and a second stage estimate of 0.0008, which is statistically significant at the 1 percent level. This indicates that a twelve-month increase in relative age, such as the one created by grade retention, would increase the likelihood of disciplinary incident by 1 percentage point. Although not necessarily conclusive, these findings provide evidence that the increase in relative age caused by retention might be playing a role in the retention effect.

Another possible explanation is the emotional distress associated with loss of friends and stigma caused by being left behind. Although it is not possible to directly test for these hypotheses using administrative data, I present indirect evidence using subgroup analysis. First, I break down the regression discontinuity estimation by how much time the student has spent in the same school before the third grade. The idea here is that the longer the student has spent in the same school, the larger the emotional burden of loss of friends will be. The estimates provided in the second-to-last panel in table 10 somewhat support this hypothesis. For students who entered the school during the third grade, the effect of retention is not statistically different from zero, whereas for the “stable” students, the effects are negative and statistically significant.

I also check to see whether retention effects are larger for students in schools with fewer retained students. The idea in this exercise is that the stigma associated with being held back is presumably less severe for students if most of their peers are also retained. The last panel in table 10 presents the discontinuity estimates using schools with fewer than five retained students in a given year and schools with more than ten retained third graders. The estimated effects seem to support this hypothesis, with larger effect sizes for the former subgroup of schools. For instance, for schools with fewer than five retained students, the retained students are 10 to 16 percentage points more likely to be involved in a disciplinary incident, whereas that number is 2 to 4 percentage points for schools with high retention rates. Overall, all three explanations seem to be playing a role on the adverse effects of grade retention.

Test-based accountability has become the new norm in public education over the last decade, with demands for greater accountability intensifying in the wake of recent initiatives such as the Race to the Top. In many states and school districts nationwide, not only schools and teachers are held accountable for the performance of their students but low performance in standardized tests also carries significant implications for students. One of these implications is grade retention for low performers.

In this study, I examine the effects of grade retention on student misbehavior using the non-linearity created by the Just Read, Florida! program, a reading initiative that requires students with reading skills below grade level to be retained in the third grade. The regression discontinuity estimates suggest grade retention increases the likelihood of disciplinary incidents and suspensions among just-retained students who are otherwise comparable to their peers on the other side of the retention cutoff. The findings also suggest these adverse effects are concentrated among the economically disadvantaged, African Americans, and boys.

The overarching conclusion in the recent literature is that grade retention, especially in early grades, leads to significant achievement gains in the short run. The findings presented in this study reveal that these short-run benefits come with the burden of higher rates of student misbehavior. If, however, early grade retention policies gradually lead to improved learning in grades before the third grade, and hence lower retention rates (as retention policies typically intend to accomplish), then these adverse effects might become less significant in the long run. That is, despite the fact that the adverse effects of grade retention on misbehavior persist, these effects might become less concerning over time if reading achievement improves and fewer students are retained. In fact, this seems to be the trend in Florida, with significantly more students scoring above grade level in third grade and fewer students being retained (12 percent retention rate in 2003 compared with 7 percent in 2011).

Finally, it is important to note that the estimated effects in this study reflect the combined effects of the grade retention and the instructional support components of the Florida policy. Therefore, the findings presented here might not be generalizable to other grade retention policies. Nevertheless, this study might help better assess the costs and benefits associated with increasingly popular test-based retention policies that are commonly tied to support mechanisms for the retained students.

This research was supported by the National Center for the Analysis of Longitudinal Data in Education Research (CALDER) funded through grant R305A060018 to the American Institutes for Research from the Institute of Education Sciences, U.S. Department of Education. The opinions expressed are those of the author and do not represent views of the Institute or the U.S. Department of Education. I would like to thank Tiffany Chu and Kennan Cepa for excellent research assistance. All errors are mine.