Cyber versus Brick and Mortar: Achievement, Attainment, and Postsecondary Outcomes in Pennsylvania Charter High Schools

Over the course of almost thirty years, charter schools have become an important and growing sector of the national education market, with over seven thousand charter schools currently operating in forty-four states and the District of Columbia (Irwin et al. 2021). During this time, the sector has expanded beyond more traditional brick-and-mortar schools to include cyber charter schools, where enrollment grew almost tenfold over the past several years, from approximately 190,000 students in 2015 to over 1.8 million students in 2019–20 (author's calculations from the Common Core of Data). The number of states with cyber charter schools also grew from twenty-one to thirty-one states plus the District of Columbia during this same period. Yet, despite the rapid growth in cyber charter schools, little is known about their impacts, particularly on longer-term outcomes such as high school graduation and college enrollment. Rather, most work on charter schools focuses on the impacts of brick-and-mortar charter schools on test score outcomes (see for example, Bifulco and Ladd 2006; Sass 2006; Hoxby, Murarka, and Kang 2009; Angrist et al. 2010; Clark et al. 2015), with a smaller body of work examining the impacts of brick-and-mortar charter schools on attainment and postsecondary enrollment. In this paper, I contribute to the literature on charter school effects by using a combination of matching and regression to examine the impacts of both cyber and brick-and-mortar charter high schools in Pennsylvania on student attendance, achievement, graduation, and postsecondary outcomes. Pennsylvania offers an ideal context to study the effects of charter schools by modality given its diverse student population and robust cyber charter school sector, which is routinely one of the largest in the country.

Briefly, I find that brick-and-mortar charter schools have positive or no effects across all outcomes and that positive effects are concentrated among Black and economically disadvantaged students, and students living in urban districts. While students who attend cyber charter schools have higher attendance and lower chronic absenteeism, they have significantly worse test score, graduation, and postsecondary outcomes. Students who enroll in cyber charter schools at the beginning of ninth grade are 9.5 percentage points less likely to graduate, 16.8 percentage points less likely to enroll in college, and 15.2 percentage points less likely to persist beyond one semester compared with students who enrolled in traditional public schools (TPSs) at the beginning of high school. These negative relationships are consistent and of roughly equal magnitude across race/ethnicity, socioeconomic status, and locale, indicating that cyber charter schools are detrimental to a wide variety of students. Although caution is warranted in interpreting cyber charter school estimates as causal, they are robust to different specifications, which suggests the need for greater cyber charter school oversight/regulation and may call into question the efficacy of online education. The rest of the paper proceeds as follows: In section 2 I review the prior literature, and in section 3 I describe the charter school context in Pennsylvania. Section 4 discusses methods, I present results in section 5, and I conclude in section 6.

A large literature documents the effects of charter schools on test scores. Broadly, this work finds that, on average, charter schools have small negative or no effects on achievement but that location and organization may be important, with urban and Charter Management Organization (CMO) affiliated charters tending to have positive effects (see, for example, Tuttle, Gleason, and Clark 2012; Furgeson et al. 2012; Angrist et al. 2013; Center for Research on Education Outcomes [CREDO] 2013, 2017). A series of studies by CREDO finds similar results in Pennsylvania—namely, small negative or no effects on average and small positive effects of urban charter schools on reading (CREDO 2011, 2013, 2019). These studies all focus on single-year impacts, although an earlier study of Pennsylvania charter schools found positive effects for students always enrolled in charter schools, suggesting duration of exposure may be critical (Zimmer et al. 2009).

While important, this larger literature on achievement may tell us little about longer- term impacts of charter schools, as test scores may only be weakly linked to later life outcomes (Deming 2009; Jackson 2012). Indeed, a recent study on universal pre-K in Boston demonstrated no test score effects, but large increases in graduation and postsecondary enrollment (Gray-Lobe, Pathak, and Walters 2021). I am able to address this by examining not only student achievement, but longer-term outcomes including high school graduation and college enrollment.

A smaller but growing literature explores the effects of charter schools on high school graduation and postsecondary outcomes and tends to find no or positive effects. Focusing on students who were enrolled in charter schools in eighth grade, Booker et al. (2011) find that students who attend charter high schools in Florida and Chicago are 7 to 15 percentage points more likely to graduate from high school, while Furgeson et al. (2012) find no effects in a national sample of six CMOs serving high school students.

A number of studies find positive impacts of charter high schools on the probability of college enrollment (Zimmer et al. 2009; Booker et al. 2011; Dobbie and Fryer 2015; Sass et al. 2016; Angrist et al. 2016; Davis and Heller 2019; Coen, Nichols-Barrer, and Gleason 2019), with evidence from Boston that enrolling in charter high schools causes a shift in enrollment from two- to four-year institutions (Angrist et al. 2016). Many of these studies rely on charter school lotteries to estimate impacts and are city-specific, however, which may limit generalizability to a wider set of charter high schools and cannot explore heterogeneity by locale. In the only national study of charter school effects on postsecondary outcomes, Place and Gleason (2019) find no effect of being admitted to charter middle schools on the likelihood of college enrollment and completion.

In the study most similar to this, Dobbie and Fryer (2020) examine the impacts of Texas charter schools on achievement, postsecondary, and labor market outcomes. Using a combination of matching and regression analysis, they find that, on average, charter schools have small positive effects on test scores, graduation, and postsecondary enrollment, with negative effects on wages. There are substantial differences by the type of charter school, however, as No Excuses charter schools have large positive effects on test scores on the order of 0.1 standard deviation while other charter schools have small negative effects. While effects on graduation are positive for all charter schools, the effects of attending a No Excuses charter school are over three times as large as the effects of attending other charter schools. Finally, while attending a No Excuses charter school significantly increases the probability of enrolling in a four-year postsecondary institution, attending other charter schools appears to shift enrollment from four-year to two-year institutions. These results highlight the importance of examining heterogeneous impacts, which I am able to do here by exploring differences by modality and location.

Evidence on the effectiveness of cyber charter schools is much more consistent, finding that students who enroll in these schools tend to have significantly worse outcomes than students in either TPSs or brick-and-mortar charter schools (Ahn 2016; Ahn and McEachin 2017; Bueno 2020; Fitzpatrick et al. 2020). This is also true in Pennsylvania, where a recent CREDO study found that attending cyber charter schools is associated with lower performance on elementary and middle school exams (CREDO 2019). In the only study to date that explores the impacts of cyber charter schools on attainment, Bueno (2020) finds that ever attending a virtual charter in Georgia is associated with a 10 percentage-point reduction in the probability of graduating.

In this paper, I contribute to the literature on charter school effects by providing some of the first evidence linking cyber charter schools to attainment and postsecondary outcomes and exploring heterogeneity across a number of dimensions including student characteristics and locale. In addition, while much of the prior work exploring the impacts of brick-and-mortar charters on attainment and postsecondary outcomes focuses on cohorts of students who graduated over a decade ago when charter schools made up a significantly smaller share of the education market, I am able to provide evidence on a recent cohort of graduates. Finally, I am able to examine the extent to which estimates of brick-and-mortar effectiveness in Pennsylvania are similar to those from other states.

Pennsylvania's charter school legislation was signed into law in 1997 and as of academic year 2016–17 (the end of my study period), over 130,000 or approximately 7.7 percent of all public school students in Pennsylvania were enrolled in charter schools. Almost a third of these students were enrolled in grades 9–12. There is significant geographic variation in the location of charter schools and the students who enroll in them—slightly over half (52.2 percent) of charter high school students attend schools in urban districts, with 32.5 percent in suburban districts, and approximately 15.3 percent in rural districts, allowing me to explore differences by locale.

Importantly for this study, Pennsylvania is home to one of the largest cyber charter sectors in the country, which enrolled 33,000 students in 2017, more than half of whom were high school students (author's calculations). Like many other states with cyber charter schools, the sector has grown rapidly in recent years. In 2020, Pennsylvania cyber charters enrolled over 175,000 students or 2.1 percent of the total public school population, second only to California in terms of total cyber charter school enrollments, and to Oklahoma in terms of the percentage of public school students enrolled in cyber charter schools.

Cyber charter schools in Pennsylvania are fully online and enrollment is open to all students aged 5–21 years who are residents of Pennsylvania, although prior work indicates that rural students have the greatest likelihood of enrolling in these schools (Mann et al. 2016). While rural students may be more likely to enroll in cyber charter schools, their demographics largely mirror that of the public school population in the state (Cordes and Seifert 2021). Cyber charters vary in terms of size (from 97 to 9,723 students in 2017), attendance requirements, and programming, with some schools offering completely asynchronous instruction and others offering a blend of synchronous and asynchronous instruction. Cyber charter schools are authorized by the state and must undergo reauthorization every five years, unlike brick-and-mortar charter schools that are authorized by school districts. Students can enroll in cyber charter schools online or over the phone, although relatively little is known about what motivates students to enroll in these schools. During the study period, students/parents were required to submit separate applications for every cyber and/or brick-and-mortar charter school where they wished to enroll.¹

The large and growing cyber charter school sector in Pennsylvania allows me to explore the effects of cyber charter schools for students in a variety of contexts and with a wide set of characteristics, providing the opportunity not only to explore heterogeneity but also to generalize my findings to a wider set of states and districts.

Data for this analysis come from four different sources: student-level administrative records from the Pennsylvania Department of Education (PDE), the National Student Clearinghouse (NSC), the Education Names and Addresses (EdNA) database maintained by PDE, and the Common Core of Data (CCD). Administrative records from PDE include student demographic and program information; school attended; zoned school; and student outcomes including standardized test scores, attendance, and graduation. Importantly, they contain a unique student identifier that allows me to track students over time and link them with NSC data, which contain individual-level information on enrollment in postsecondary education, including intensity and institution type. EdNA data are used to identify charter schools and cyber charter schools as well as the operating dates of these schools. Data from the CCD are used to identify charter school locale (urban, suburban, and rural).

Students in the sample are all members of the ninth-grade cohorts of 2012 or 2013. There are five restrictions for students to be included in the final sample. First, students must be enrolled in Pennsylvania public schools—either TPS or charter—in eighth grade. Second, students must have baseline data on gender and race, which is used to create my comparison groups and at least one baseline test score, which is a key covariate. Third, students must be continuously enrolled in Pennsylvania public schools in the fall of every year from ninth grade until what would be their twelfth-grade year if they made standard academic progress.² I add this exclusion because I do not have data on graduation or postsecondary outcomes for students who exited the state prior to what would be their twelfth-grade year. This restriction results in the loss of 4,112 charter school and 17,081 TPS students. Fourth, following Dobbie and Fryer (2020), I exclude students who attend charter schools with fewer than ten students in a cohort, resulting in the loss of 42 students. Finally, students must be in a matched cell with at least one TPS student and one charter school student because my models include matched cell fixed effects. These restrictions are imposed due to my research design, which is discussed below in more detail. As shown in figure A1 (available in a separate online appendix that can be accessed on Education Finance and Policy’s website at https://doi.org/10.1162/edfp_a_00399), students included in the final sample attend schools throughout the state and do not appear to draw disproportionately from any single district.

A total of 246,184 students were enrolled in ninth grade in 2012 or 2013 who were also enrolled in eighth grade in the previous year (table 1). Of these, 81,036 (32.9 percent) are included in my sample. Sample students are more likely to be Black, less likely to receive special education services or be missing test scores, and are lower performing than the population of ninth graders. Sample students are also more likely to have ever been enrolled in either a brick-and-mortar or cyber charter school. Overall, however, most of these differences are modest and likely driven by the characteristics of districts where brick-and-mortar charter schools operate.

Across the two cohorts, 13,158 students enrolled in either a brick-and-mortar (9,418) or cyber charter (3,740) in ninth grade, of whom 8,385 (63.7 percent) are included in the sample. All cyber charter high schools and roughly 80 percent of brick-and-mortar charter schools serving a twelfth grade that were operating during the sample period are represented by this sample of students. The primary reason that charter students are dropped from the final sample is because they are not in a matched school-gender-race-cohort cell with at least one TPS student. White students are slightly overrepresented and students with disabilities are underrepresented in the final sample of cyber charter school students, although most other characteristics are similar. Compared with all students who enrolled in a brick-and-mortar or cyber charter school in ninth grade, a lower proportion of sample students repeated a grade, but average performance is quite similar. Overall, however, the differences between sample charter school students and all students who enrolled in a charter school at the beginning of ninth grade are modest and suggest that results from my sample may generalize to the larger population of Pennsylvania charter school students.

Key to this analysis is the measure of charter school enrollment, which can be captured contemporaneously (i.e., whether a student attended a charter high school in year t), as an average effect (i.e., whether or not a student enrolled in a charter high school at the beginning of ninth grade), or cumulatively (i.e., how many years a student has been enrolled in a charter high school as of year t). Because several of my outcomes are measured at one point in time or when students are no longer in high school (i.e., college enrollment), contemporaneous measures are not appropriate. Whereas Dobbie and Fryer (2020) use a cumulative measure of charter school enrollment, I use an average treatment. This is because estimates from a cumulative measure are identified not only by differences between students who attend charters and TPSs, but also by differences between those who spent more and less time in a charter school. Of particular concern is that students who spent more time in a charter school are those for whom the school is more beneficial, introducing an additional source of bias. Indeed, as shown in table 2, there is some evidence of this type of selection among brick-and-mortar charter school students. Therefore, my primary measure of charter school enrollment is an indicator equal to 1 if a student attended a brick-and-mortar or cyber charter school in the fall of ninth grade. This can be interpreted as an intent-to-treat (ITT) effect of charter school enrollment since treatment status is based solely on enrollment decisions in the fall of ninth grade. Students who enroll in a charter high school after ninth grade are included in the comparison group, although I explore the sensitivity of my results to this decision.

Importantly, I differentiate between enrollment in brick-and-mortar versus cyber charter schools, where a cyber charter school is defined based on the designation from the EdNA database. I also examine the effects by locale: urban, suburban, and rural, which is based on the locale designation of student's school from the CCD. For students attending cyber charter schools, locale is determined based on the location of a student's zoned school.

I explore four areas of student outcomes: achievement, attendance, attainment, and postsecondary enrollment. Achievement is measured using high school state test scores in Algebra, Biology, and Literature, standardized by subject and year to have mean 0 and standard deviation of 1. Regardless of the sequence of a student's coursework, these high school tests, called Keystone exams, are overwhelmingly administered in eleventh grade.³ Although students began taking Keystone exams in 2012, test score data are only available beginning in 2015, so test score results are limited to the cohort of students entering ninth grade in 2013. Attendance is measured annually as the percent of days present (0–100) and with an indicator for chronic absenteeism, which is equal to 1 if a student is absent for at least 10 percent of days enrolled in year t. This is consistent with how Pennsylvania measures chronic absenteeism. I explore chronic absenteeism in addition to attendance, both because chronic absenteeism is one of Pennsylvania's accountability metrics under the Every Student Succeeds Act. 20 U.S.C. § 6301 (2015) and because chronic absenteeism has been linked to a variety of other negative outcomes including high school dropout (Patnode, Gibbons, and Edmunds 2018). Attainment is measured as an indicator equal to 1 if the student ever graduated from high school.⁴ Finally, I explore a variety of postsecondary outcomes using the NSC including ever enrolled (equal to 1 if student appears in the NSC data) and type of institution (ever 2-year, ever 4-year institution). Although I am only able to track postsecondary outcomes for up to two years following graduation, I also examine short-term persistence as an indicator equal to 1 if a student is enrolled for more than one semester.

The primary challenge to identifying the effects of charter school attendance on student outcomes is selection—namely, that students who choose to attend charter schools are different from those who do not in ways that may be correlated with performance. For example, charter school students may come from more motivated families or place a higher value on education, and cyber charter schools in particular may require more parental involvement (Nowicki 2022), in which case charter students might have better outcomes regardless of charter school attendance. Conversely, because brick-and-mortar charter schools are more prevalent in urban and low-income neighborhoods serving disadvantaged families and cyber charter schools may attract students who have difficulty learning in traditional classrooms (Nowicki 2022), charter school students may tend to have worse outcomes than their TPS peers. Indeed, as shown in table 1, both brick-and-mortar and cyber charter school students differ from their TPS peers on a number of dimensions, indicating that selection is an issue. As a consequence, a simple comparison of brick-and-mortar or cyber charter school student outcomes to those of TPS students is likely to yield biased estimates of charter school impacts.

In order to overcome this issue, much prior work on charter school impacts uses a lottery design, which compares outcomes of students who apply and are offered admission to charter schools to those who apply and are not offered admission. The benefit of this design is that because charter offers are random, it mimics a randomized control trial and produces unbiased impact estimates of charter school enrollment for charter school applicants. The drawback, however, is that this design can only be used to study oversubscribed charter schools, which are likely different from other charter schools. Indeed, in Pennsylvania, no cyber charter schools have lotteries and therefore such an approach would preclude an analysis of cyber charter schools. Further, the findings of lottery studies also only apply to charter school applicants and cannot reasonably be generalized to a broader student population. In addition, this type of design requires access to lottery records that can then be matched to other administrative data. Since the purpose of this analysis is to examine the impact of all Pennsylvania charter high schools, particularly cyber charter schools where lottery records are not available, I use an alternative approach, which combines matching with regression analysis.

Specifically, I match students enrolled in charter schools at the beginning of ninth grade to a group of students enrolled in TPSs at the beginning of ninth grade using baseline “cells” of eighth-grade school, gender, race, and cohort. Charter school students are included in the analysis only if they are in a baseline cell with at least one TPS student and TPS students are included only if they are in a baseline cell with at least one charter school student. All other students are dropped from the sample. A similar approach has been by used by Dobbie and Fryer (2020) to examine the impact of charter schools on postsecondary outcomes in Texas, as well as by Angrist, Pathak, and Walters (2013) and Dobbie and Fryer (2013).

Using this approach, I compare the outcomes of observationally similar students who attended the same school at baseline, and presumably face similar choice sets, but enrolled in different high schools. Any differences in student outcomes are then attributed to attending a brick-and-mortar or cyber charter high school.⁵

The key assumption necessary for this approach to yield unbiased estimates is that baseline school-gender-race-cohort effects, test scores, and characteristics account for all observed and unobserved differences between charter and TPS students. Previous research finds that similar designs yield estimates comparable to lottery-based designs when examining test score outcomes in brick-and-mortar charter schools (Deming 2009; Abdulkadiroğlu et al. 2011; Angrist, Pathak, and Walters 2013; Dobbie and Fryer 2013), suggesting that this may be a reasonable assumption for these particular analyses.

Because similar matching and regression analyses have not been thoroughly validated for graduation or postsecondary enrollment outcomes or for cyber charter schools, I also explore the potential for selection bias by regressing baseline characteristics on whether a student enrolled in a brick-and-mortar or cyber charter school at the beginning of ninth grade and all other baseline characteristics excluding the dependent variable. I perform a similar analysis with baseline outcomes that are not included in the matching process (eighth-grade science and reading scores) and for seventh-grade test scores for the 2013 cohort of ninth graders.⁶ I also test the robustness to a variety of specifications. However, caution is still warranted in interpreting these estimates as causal.

There are notable differences between brick-and-mortar charter school students and the TPS students in Pennsylvania—brick-and-mortar charter school students are much less likely to be White (17.8 versus 75.2 percent) and are more likely to be Black (56.6 versus 12.5 percent), Hispanic (21.7 versus 7.6 percent), economically disadvantaged (61.4 versus 31.5 percent), and eligible for special education compared with their TPS peers (table 1, columns 6 and 8). Brick-and-mortar charter school students also tend to have worse outcomes—on average, they have lower test scores and are almost twice as likely to be chronically absent (30.8 versus 16.3 percent).

Cyber charter school students, while demographically similar to TPS students, are less likely to be English Language learners (0.4 versus 2.0 percent) and more likely to have repeated a grade (2.0 versus 0.4 percent). Like brick-and-mortar charter students, cyber charter students tend to have worse test scores and higher rates of chronic absenteeism than TPS students, although the differences between cyber charter and TPS student outcomes are somewhat smaller.

There are also clear patterns in the relationship between number of years enrolled in a brick-and-mortar charter school and student outcomes (table 2). In particular, even after controlling for baseline characteristics and matched cell fixed effects, it is evident that students who enroll in brick-and-mortar charter schools for a longer period of time have better outcomes, particularly students who have been enrolled for four years. This suggests that students who remain enrolled in brick-and-mortar charter schools may be those for whom the schools are beneficial and raises concerns that measuring charter school enrollment based on number of years enrolled could yield upwardly biased estimates of brick-and-mortar charter school impacts. This appears to be less of an issue for cyber charter school students where point estimates, although not always significant, are generally in the same direction across the number of years enrolled and, for postsecondary enrollment and persistence, are of roughly the same magnitude.⁷ However, given concerns about selection into number of years enrolled among brick-and-mortar charter school students, I focus on ITT estimates of charter school enrollment.

There are large differences in charter school outcomes by modality—while brick-and-mortar charter schools have modest positive effects on high school test scores, cyber charter school enrollment has modest to large negative associations with these outcomes. Attending a brick-and-mortar charter high school increases Algebra 1 scores by 0.042 standard deviation (SD), Biology scores by 0.037 SD, and Literature scores by 0.109 SD (table 3, panel A). Conversely, attending a cyber charter high school is related to significantly lower scores in Algebra 1 (0.142 SD) and Biology (0.102 SD). These are both sizeable differences.

The effects of charter schools on attendance and chronic absenteeism are quite consistent—charter school enrollment is beneficial for attendance, regardless of modality (table 3, panel A). Attending a cyber charter school is related to a 1.841 percentage point increase in attendance and 4.3 percentage point reduction in chronic absenteeism, while attending a brick-and-mortar charter increases attendance by 1.168 percentage points and decreases chronic absenteeism by 4.5 percentage points. To put these estimates in context, baseline attendance of TPS students is 94.9 percent and baseline chronic absenteeism is 11 percent. Therefore, while increases in attendance are relatively small relative to baseline, reductions in chronic absenteeism are quite large.

There are two reasons why cyber charter school students may have significantly better attendance. First, it may be easier for certain types of students, such as those who work, have children, or with chronic health conditions, to attend asynchronously or virtually. Second, measures of attendance at cyber charter schools are not directly comparable to those at brick-and-mortar schools. There is a wide definition of what counts as attendance in cyber charter schools—while some simply require students to log on to the system to be counted as present, others require a certain amount of participation and contact with teachers. Therefore, attendance and chronic absenteeism may not be good measures of seat time or engagement in the cyber charter context, which may explain why, even with significantly higher attendance and lower chronic absenteeism, students who enroll in cyber charter schools have lower test scores.

I find no effect of brick-and-mortar charter schools on graduation or postsecondary enrollment (table 4, panel A). However, I do find evidence that brick-and-mortar charter schools may shift enrollment from two-year to four-year institutions, as attending a brick-and-mortar charter school increases the probability of enrolling at a four-year institution by 5.6 percentage points and decreases the probability of enrolling in a two-year institution by 2.0 percentage points. This finding is similar to Angrist et al (2016) in Massachusetts.

Conversely, cyber charter school students have notably worse attainment and postsecondary outcomes. Students who enroll in a cyber charter school are 9.5 percentage points less likely to graduate, 16.8 percentage points less likely to enroll in a postsecondary institution, 15.8 percentage points less likely to enroll in a four-year institution, and 2.3 percentage points less likely to enroll in a two-year institution. These differences are large and, in the case of graduation, similar in magnitude to what was found in Georgia (Bueno 2020).

Mirroring other patterns, I find that brick-and-mortar charter school enrollment is related to significant increases in short-term persistence, while cyber charter school enrollment is associated with large decreases (table 4, panel A). Enrolling in a brick-and-mortar charter school increases the probability of enrolling in college for at least two semesters by 2.6 percentage points relative to TPS students (significant at the 10 percent level). This represents roughly a 5 percent increase in the TPS persistence rate of 52.9 percent. Conversely, cyber charter school enrollment is related to sizable declines in persistence—cyber charter students are 15.2 percentage points or nearly 29 percent less likely to enroll for at least two semesters compared with TPS students.

My key identification assumption is that baseline covariates and matched cell fixed effects account for student selection into charter schools. In order to more formally test this assumption, I regressed an indicator for enrollment in a brick-and-mortar or cyber charter school in the fall of ninth grade on each baseline outcome controlling for matched cell fixed effects and all other baseline characteristics, as well as on baseline scores in science and writing that were not used in the analysis. To the extent that selection is still a problem in my models, I would expect large and significant coefficient estimates for charter enrollment. The results from these analyses are reported in online appendix table A1. Columns 2 and 3 show results from a single regression with separate measures for brick-and-mortar charter and cyber charter enrollment in ninth grade. Column 1 provides the non-charter school mean of each variable to provide context about magnitude.

While there are some statistically significant differences in baseline characteristics by brick-and-mortar charter enrollment, including free lunch eligibility, special education status, and gifted status, none of these are particularly large in magnitude relative to the non-charter mean. There are, however, differences in baseline math test scores at the 5 percent level and in reading at the 10 percent level. However, these are in opposite directions—enrolling in a brick-and-mortar charter school in ninth grade is related to higher reading scores but lower math scores. There are no other significant differences in test scores, including science and writing, which are not used in the analysis. For students who enrolled in ninth grade in 2013, I am also able to examine whether charter school enrollment is related to seventh-grade test scores. Here I also find that enrolling in a brick-and-mortar charter school in ninth grade is negatively related to seventh-grade math scores, which is in the opposite direction of high school test score estimates. This strengthens the argument that estimates for brick-and-mortar charter schools are causal. While there may be concern that Algebra 1 estimates reflect a regression to the mean or that Literature results are picking up pre-existing differences in performance, this is less of a concern for Biology, where I also find significant positive results.

I also find significant differences in characteristics of cyber charter school students—enrolling in a cyber charter school is negatively associated with the probability of baseline special education and English language learner designation and positively associated with the probability of baseline free lunch eligibility and gifted classification. There are also significant differences in baseline test scores—students who enroll in a cyber charter school in ninth grade have lower baseline math and writing scores and higher baseline science scores. In addition, enrolling in a cyber charter is related to higher seventh-grade reading and math scores. These differences in seventh-grade performance are in the opposite direction and roughly half the magnitude of high school performance estimates. However, given these differences, some caution is warranted in interpreting cyber charter school results as causal. The magnitude of high school test score estimates is often larger, and, in most cases, in the opposite direction of baseline differences, so my estimates are unlikely to be fully explained by selection, but I also conduct a number of robustness checks.

Although I use a similar matching and regression technique as prior studies exploring charter school effectiveness, these studies largely focus on test score outcomes and brick-and-mortar charters. Further, my falsification test for cyber charter school reveals some significant differences between cyber charter and TPS students at baseline, even accounting for matched cell and baseline performance. Therefore, I conduct a variety of robustness checks to explore the sensitivity of my results. For most robustness checks, I present results for a subset outcomes: Biology Keystones, chronic absenteeism, graduation, postsecondary enrollment, and persistence. I present results for chronic absenteeism rather than attendance, as chronic absenteeism is one of Pennsylvania's accountability measures under ESSA and is therefore a more policy-relevant metric. All other outcomes show similar patterns and are available upon request.

As my first robustness test, I reestimate models excluding controls for baseline student characteristics. When I do so, results are similar, although point estimates are slightly more positive for test score, attendance, graduation, and postsecondary outcomes and slightly more negative for chronic absenteeism (table 3, panel B; table 4, panel B). Second, because my brick-and-mortar and cyber charter school measures are defined based on where students enrolled at the beginning of ninth grade, there are some students in the comparison group who enrolled in a charter school later in high school. I therefore test the sensitivity of my results to including separate charter school measures for these students. Again, results are nearly identical, although I do find that students who switch into charter schools after ninth grade, particularly those who switched into cyber charters, tend to have worse outcomes than charter school students who enrolled in ninth grade and TPS students (online appendix table A2, panel A). Third, some students who initially enrolled in a brick-and-mortar charter school in ninth grade later enroll in a cyber charter school, and vice versa. Students who initially enrolled in a brick-and-mortar (cyber) charter school only contribute to the brick-and-mortar (cyber) charter school estimates. I therefore test the robustness of my models to including controls for whether students switch between the brick-and-mortar and cyber charter sectors. Again, my results are robust to this choice (online appendix table A2, panel B).

Finally, my main sample includes only those students who were continuously enrolled from ninth grade to what would be their twelfth grade year if they made standard academic progress. This may lead me to exclude students who dropped out but had not been recorded as such, which could be problematic if charter schools affect dropout. Further, if the majority of dropout occurs before students are enrolled for four years and this dropout is not recorded, then my estimates of graduation conditional on four years of high school enrollment may be different than the effects on high school graduation overall. Therefore, I reran my analyses without this sample restriction. Test score and attendance results for this larger sample of students are nearly identical, although effects on Algebra 1 and Biology are no longer statistically significant (online appendix table A3).

To examine graduation and postsecondary outcomes for this larger sample of students, I performed a bounding exercise. To construct a lower bound estimate, I assumed that all charter school students did not graduate/enroll while all TPS students did. To construct the upper bound, I assumed the opposite.⁸ For this larger sample of students, I find that lower bound estimates for both brick-and-mortar and cyber charter effects on graduation and postsecondary enrollment are negative, although the magnitude of the cyber charter estimates are nearly three times as large as those for brick-and-mortar charter schools. However, while I find that the upper bound estimates for brick-and-mortar charters is positive and significant, and estimates for cyber charter school enrollment are not statistically significant. This suggests that even among this larger population of students, cyber charters have a negative or, in the best case scenario, no effect on graduation or postsecondary enrollment.

Next, I examine whether charter school impacts vary for key subgroups of students, which has important equity implications. Of particular concern is whether charter schools exacerbate existing educational disparities. In the results below, I focus on subgroup results by race/ethnicity and economic disadvantage.⁹

Brick-and-mortar charter schools appear to be particularly beneficial for Black and Hispanic student performance and attendance, increasing test scores across subjects, improving attendance, and decreasing chronic absenteeism, with no or negative effects for White students except for Literature test scores, which increase (table 5). Effects of brick-and-mortar charter schools on postsecondary outcomes are larger among Black students. Notably, Black students who enroll in brick-and-mortar charter schools are 6.8 percentage points more likely to enroll in a four-year institution and 4.3 percentage points more likely to persist. While both Hispanic and White students who enroll in brick-and-mortar charter schools are roughly 3–4 percentage points more likely to enroll in a four-year institution, there are no significant effects for either group on short-term persistence.

While there is heterogeneity in the effects of brick-and-mortar charter schools, attending a cyber charter school is almost uniformly related to worse outcomes across all racial/ethnic groups, with the exception of attendance and chronic absenteeism for White students. Notably, differences in postsecondary outcomes are large and roughly equal in magnitude across all racial/ethnic groups. For example, attending a cyber charter school is related to a 15.0 percentage point reduction in the probability that Black students enroll in postsecondary education, compared to a 10.8 percentage point reduction among Hispanic students and a 17.8 percentage point reduction among White students.

Taken together, these results suggest that enrolling in brick-and-mortar charter schools is more beneficial for Black and Hispanic students. This could indicate either that brick-and-mortar charter schools are more effective at educating students of color, or that White students tend to have better access to high-quality TPSs, making charter school enrollment less valuable.

The effects of brick-and-mortar charter schools on test scores, attendance, chronic absenteeism, and short-term persistence are concentrated among economically disadvantaged students, with no effects for non–economically disadvantaged students other than positive effects on literature scores (table 6). However, the effects of brick-and-mortar charters on the probability of enrolling in a four-year institution are positive and roughly similar for both groups.

Attending a cyber charter school is negatively associated with test scores, graduation, postsecondary enrollment, and persistence for both groups. In addition, the magnitude of these differences is roughly the same for both groups with the exception of graduation where the magnitude for economically disadvantaged students (12.2 percentage points) is almost twice that for non-economically disadvantaged students (6.9 percentage points). This suggests that cyber charter schools may be detrimental to student performance regardless of socioeconomic status.

Given prior literature documenting large positive effects of urban charter schools on student achievement, I examine whether effects vary by charter school or student locale. Because cyber charter schools are open to students across the state and do not require in-person attendance, I assign cyber charter school students’ locale based on their zoned school. This is of particular interest given that one of the arguments offered in favor of cyber charter schools is that they offer an important form of choice for rural students.

As shown in table 7 and consistent with prior evidence, the positive effects of brick-and-mortar charter schools are almost entirely driven by urban charter schools, while suburban and rural charter schools have small negative or no effects across outcomes. Similar to the racial/ethnic differences, the larger positive effects of urban brick-and-mortar charter schools may be because students in suburban or rural districts have better access to high-quality TPSs or the types of charter schools that tend to locate in urban areas.¹⁰

Enrollment in cyber charter schools is related to worse test score, attainment, and postsecondary outcomes regardless of locale. Further, the large magnitude of these differences for graduation, postsecondary enrollment, and persistence is nearly identical across locales. Therefore, while cyber charter schools may increase access to choice for students living in rural areas, this does not appear to translate into better outcomes.

My findings about the effects of brick-and-mortar charter high schools are largely consistent with prior literature. Specifically, I find that enrolling in these schools improves attendance and high school test scores, increases the probability of postsecondary enrollment, shifts enrollment from two- to four-year institutions, and increases short-term persistence. These effects are concentrated among urban charter schools. Importantly, I also find evidence that brick-and-mortar charter high schools in Pennsylvania appear to be particularly beneficial for Black and economically disadvantaged students without negatively impacting other groups. Thus, these schools may be an important vehicle for closing achievement and opportunity gaps, particularly because Black students are more likely to enroll in brick-and-mortar charter schools in Pennsylvania (Cordes and Seifert 2021).

By contrast, enrollment in cyber charters is associated with substantially worse outcomes except for attendance and chronic absenteeism. Of particular concern is that enrollment in cyber charter schools is associated with a much lower probability of high school graduation, college enrollment, and short-term persistence. This finding is consistent across subgroups, locales, and length of cyber charter school enrollment. One potential explanation for substantially worse attainment and postsecondary outcomes is that the flexibility of cyber charter school models may attract students in need of credit recovery, who then take longer to graduate. Therefore, an important area of future research would be to explore whether some of this disparity disappears when exploring five-year or six-year graduation rates and longer-term postsecondary enrollment. It is also possible that cyber charter schools tend to attract students facing other challenges or obligations at home, such as work, parenting, or chronic health issues, which I am unable to fully account for in my analysis. Regardless, at a minimum these findings suggest a need for additional oversight and regulation of cyber charter schools, a better understanding of who chooses to attend cyber charter schools, and research into best practices. Combined with other evidence that remote instruction during the pandemic was a primary driver of increasing achievement gaps (Goldhaber et al. 2022) and that districts with full in-person instruction had significantly smaller declines in pass rates (Jack et al. 2022), my findings may also bring into question the general efficacy of online learning.

It should be noted that there are some limitations to this study. Most notably, the analytic approach I use here has not been validated for non–test score outcomes or cyber charter schools. Although my findings about brick-and-mortar charter schools are largely consistent with prior literature from other states, and my results for both brick-and-mortar and cyber charter schools are robust to various specifications and samples, estimates may still suffer from bias. Another potential limitation is that my sample includes only about one third of all students enrolled in ninth grade in 2012 or 2013, and only about two thirds of students who enrolled in a charter school in ninth grade. However, differences between my sample and the overall population of students in Pennsylvania are relatively small, so that it may be reasonable to generalize my findings to the larger population of charter school students in the state.

These results have several implications for future research. The first is the importance of not treating charter schools as a monolith, given that charter modality appears to matter for impacts. A second related point is the role of the counterfactual in interpreting effects, as my results suggest that charter schools may be particularly beneficial to students who may otherwise have limited access to high-quality TPSs. The third point is the need for future work that explores the selection of students into cyber charter schools specifically and the role/efficacy of online learning more generally. Finally, additional work should explore the impacts of both brick-and-mortar and cyber charter schools on postsecondary persistence, quality, and later life outcomes.

I would like to thank the Office of Data Quality at the Pennsylvania Department of Education, and particularly Candy Miller, for data assistance. I also thank participants at the Association for Education Finance and Policy annual meetings for helpful advice and feedback. This work was supported by a Research Scholar grant from the Pennsylvania Department of Education. All opinions are those of the author and do not represent views of the Pennsylvania Department of Education.