Abstract

This article presents findings from the first independent, third-party appraisal of the impact of the Teacher Advancement Program (TAP) on student test score gains in mathematics. TAP is a comprehensive school reform model designed to attract highly effective teachers, improve instructional effectiveness, and elevate student achievement. We use a panel data set to estimate a TAP treatment effect by comparing student test score gains in mathematics in schools that participated in TAP with student test score gains in non-TAP schools. Ordinary least squares estimation reveals a positive TAP treatment effect on student test score gains in the elementary grades, with weaker but still positive point estimates in the secondary grades. When estimation methods control for selection bias, the positive effect remains at the elementary level, but most estimates for grades 6 through 10 turn negative. Our findings are qualified by the lack of information on the fidelity of implementation across TAP schools and on variation in features of TAP programs at the school level.

1.  Introduction

A number of school districts and states are instituting performance incentive policies as a potential lever to enhance teacher effectiveness and school productivity. Performance incentive policies also are being used to recruit and retain more effective teachers. These policy innovations are driven, in part, by the fact that existing teacher remuneration practices are not closely related to student performance and schooling outcomes (Hanushek 2003; Goldhaber 2002).1

The Teacher Advancement Program (TAP), a comprehensive school reform model providing teachers with opportunity to earn performance pay, has gained considerable attention in recent years. Developed in 1999 by Lowell Milken and other individuals at the Milken Family Foundation (MFF) to attract highly effective teachers, improve instructional effectiveness, and elevate student achievement, TAP operates in more than 220 schools in fifteen states and the District of Columbia (NIET 2007; Rotherham 2010). In the aggregate, there are approximately 5,000 teachers and 72,000 students in TAP schools across the United States (Rotherham 2010).

TAP also figured prominently in the 2006 and 2010 announcements of Teacher Incentive Fund (TIF) grantees. TIF, a federally enacted direct discretionary grant program, funds development and implementation of principal and teacher performance incentive pay programs. Of the approximate $240 million awarded during fall 2006, $88.3 million (36.80 percent) went to districts and states that proposed to implement TAP. Similarly, in 2010, approximately $135.8 (30.74 percent) of the $443 million awarded went to the TAP and/or its partner school systems.

Evaluations of TAP report generally positive findings. Studies have found positive effects on teachers’ and schools’ value added (Solmon et al. 2007) and student achievement gains (Schacter et al. 2002, 2004). Furthermore, Solmon et al. (2007) found that an equal or higher percentage of TAP schools make Adequate Yearly Progress under No Child Left Behind than other schools in their respective states, despite larger concentrations in TAP schools of students qualifying for free and reduced price lunch. All of these studies, however, have been conducted in partnership with MFF or the National Institute for Excellence in Teaching (NIET),2 causing some to raise concerns about the independence of evaluators from stakeholders.

To our knowledge, the research reported here represents the first independent, third-party assessment of TAP. We use a panel data set to estimate a TAP treatment effect by comparing student test score gains in schools that participated in TAP with student test score gains in non-TAP schools. Our data set includes thirty-two TAP schools and roughly 1,200 non-TAP schools from two states over a five-year period from 2002–03 to 2006–07.3 Student test scores are available in mathematics two times per year in second through tenth grades, allowing for a fall-to-spring gain score as the outcome of interest.

In our models TAP is usually represented by a simple binary indicator. Although the coefficient on this variable is meant to measure the effect of TAP on student test score gains, as always in this kind of research, other factors might be confounded with TAP. This is of particular concern as TAP schools are self-selected. Thus, one might expect distinctive outcomes in TAP schools even in the program's absence. We address this concern in four ways. First, we include a variety of school and student characteristics in the model. Second, we use a school fixed effects estimator to control for unobserved characteristics of schools that may explain selection into TAP as well as achievement. Third, we re-estimate the model using a matched sample of treatment (TAP) and control (non-TAP) schools. Finally, we use a two-step selection-correction estimator (as in Heckman 1979) to remove selection bias.

The results we obtain when controlling for selection on unobservables stand in contrast to prior studies. Ordinary least squares (OLS) estimation reveals a positive TAP treatment effect on student test score gains in mathematics in the elementary grades, with weaker but still positive point estimates in the secondary grades. When we use an estimator that controls for self-selection, the positive effect remains at the elementary level but most estimates for grades 6 through 10 turn negative, some significantly so.

Although our study is the first to estimate a TAP treatment effect that controls for selection, it is important to acknowledge limitations of our study. The sample of TAP schools is small. We also lack information both on the fidelity of implementation and on variation in features of TAP programs at the school or grade level (e.g., minimum and maximum bonus sizes, percent of teachers voting in favor of TAP adoption, and so forth).4 Furthermore, it is unknown whether our sample is representative of other schools and locations that will be or are actually implementing TAP across the nation. The MFF has continued to revise elements of the TAP, including an interactive Web-based training portal that provides individualized training and support, a catalog of successful instructional and implementation strategies for schools in the program, and expanded development offerings at the annual national TAP conference and TAP summer institutes (NIET 2010). Finally, TAP is designed to attract highly effective teachers, improve instructional effectiveness, and elevate student achievement. This study directly tests only the latter.

In the next section, we provide more detail on TAP. We follow with a review of relevant literature in section 3. In sections 4 and 5 we describe our analytic strategy and our data and sample, respectively. Findings are presented in section 6. Section 7 discusses results and explores some alternative explanations to our findings.

2.  Description of the Teacher Advancement Program

TAP's design has four components: (1) multiple career paths, (2) ongoing applied professional growth, (3) instructionally focused accountability, and (4) performance-based compensation.5 Multiple career paths create opportunities for teachers and specialists to advance professionally without leaving the classroom by becoming a career teacher, master teacher, or mentor teacher. Ongoing applied professional growth is encouraged by providing teachers collaboration time to develop and implement new instructional practices and curricula focused on increasing student learning. Professional growth occurs both individually and in groups of teachers (grade-level or subject-level). TAP-identified mentor and master teachers are engaged to facilitate discussion and planning and conduct classroom observations.

Table 1 provides a summary of TAP's assessment and compensation system for career, master, and mentor teachers. Teacher knowledge, skills, and responsibilities is the first indicator in TAP's assessment and compensation system. Fifty percent of a teacher's performance award is contingent on classroom observations. Four to six observations are done by certified evaluators who are master teachers, mentor teachers, or administrators. All evaluators conduct observations separately, not as a group. Teachers are hired as career, mentor, and master teachers on a competitive basis that includes formal interviews, classroom observations, evidence of instructional expertise, demonstrated leadership, and expertise in adult learning, to name a few.

Table 1.
Select Characteristics of TAP's Assessment and Compensation System
Career Track
AssessmentCareer/SpecialistMentorMaster
ComponentsTeacherTeacherTeacher
Knowledge, Skills, and Responsibilities 
  Evaluators Mentor review; Self review; Mentor review; Self review; Teachers’ review; 
   Master teacher review;   Master teacher review;   Self review; 
   Administrator review   Administrator review   Administrator review 
  Measurement Portfolio documentation; Portfolio documentation; Portfolio documentation; 
Instruments   Observation; Interview   Observation; Interview   Observation; Interview 
   process   process   process 
  Unit of Analysis Teacher Teacher Teacher 
  Percentage of award 50% 50% 50% 
pool designated for    
this component    
Teacher Value Added    
  Measurement Standardized Standardized Standardized 
Instrument   assessment   assessment   assessment 
  Unit of Analysis Classroom Classroom Classroom 
  Percentage of award 30% 30% 30% 
pool designated for    
this componenta    
School Value Added    
  Measurement Standardized Standardized Standardized 
Instrument   assessment   assessment   assessment 
  Unit of Analysis School School School 
  Percentage of award 20% 20% 20% 
pool designated for    
this componenta    
Career Track Bonus    
  Amount $0 $2,500–$4,500 $6,000–$12,000 
Career Track
AssessmentCareer/SpecialistMentorMaster
ComponentsTeacherTeacherTeacher
Knowledge, Skills, and Responsibilities 
  Evaluators Mentor review; Self review; Mentor review; Self review; Teachers’ review; 
   Master teacher review;   Master teacher review;   Self review; 
   Administrator review   Administrator review   Administrator review 
  Measurement Portfolio documentation; Portfolio documentation; Portfolio documentation; 
Instruments   Observation; Interview   Observation; Interview   Observation; Interview 
   process   process   process 
  Unit of Analysis Teacher Teacher Teacher 
  Percentage of award 50% 50% 50% 
pool designated for    
this component    
Teacher Value Added    
  Measurement Standardized Standardized Standardized 
Instrument   assessment   assessment   assessment 
  Unit of Analysis Classroom Classroom Classroom 
  Percentage of award 30% 30% 30% 
pool designated for    
this componenta    
School Value Added    
  Measurement Standardized Standardized Standardized 
Instrument   assessment   assessment   assessment 
  Unit of Analysis School School School 
  Percentage of award 20% 20% 20% 
pool designated for    
this componenta    
Career Track Bonus    
  Amount $0 $2,500–$4,500 $6,000–$12,000 

Notes:aIf a teacher does not have direct instructional responsibilities for a group of students, or a teacher works in a non-tested subject or grade, this assessment component is shifted to school achievement gains. In these instances, the percentage of award pool designated for the school value-added component is 50 percent for that teacher.

Adapted from Milken Family Foundation (2004, p. 7).

Teacher value added is the second indicator in TAP's assessment and compensation structure. Thirty percent of a teacher's performance award is based on value-added measurement of gains the teacher produces in his/her classroom's achievement. Based on the number of standard errors a teacher's estimated value added falls above or below the average estimate in the state,6 teachers are rated as level 1 through level 5.7 If a teacher does not have direct instructional responsibilities for a group of students, or a teacher works in a non-tested subject or grade, this component of TAP's assessment and compensation structure is shifted to school-wide achievement gains.

School-wide achievement is the final indicator in TAP's assessment and compensation structure. Twenty percent of a teacher's individual performance award is dependent upon school-wide achievement. Like the classroom achievement score, school-wide performance is determined by how many standard errors above (or below) a school's value-added estimate falls from the average school-wide effect estimate in the state.8 The school-wide achievement award is equally distributed to all teachers.

NIET recommends allocating a minimum of $2,500 per teacher to a school's performance award fund. The performance award fund is then apportioned based on the ratio of the number of teachers in each career level (i.e., career, mentor, or master teacher) to the total number of teachers in the school. NIET's recommended compensation structure enables teachers to earn anywhere from zero to $12,000 per year, though there can be variation across sites. Performance is judged against an absolute standard so as not to create competition among teachers for a fixed amount of awards. NIET estimates that the program costs approximately $400 per student, or about 6 percent of a school's operating budget.9

3.  Review of Relevant Literature

We know of only four studies that have analyzed the impact of TAP on student outcomes. The first evaluation of TAP reported by MFF analyzed student achievement growth in four Arizona TAP schools relative to a similar set of non-TAP schools using data from the 2000–01 and 2001–02 school years (Schacter et al. 2002).10 Schools were compared on the basis of a “gap reduction model.” This model quantifies school performance using the relative decrease in the distance between a school's baseline percentile rank score on the Stanford 9 and a fixed achievement target defined by MFF. The formula for school j in year t appears in equation 1
formula
1
T was established as the 85th percentile for all schools in the state.11 Clearly, two schools that made the same absolute progress (Ajt – Aj, t-1), but started with different initial values of Aj, t-1, will fare differently by this metric. Unfortunately, in selecting the sample of comparison schools, schools were not perfectly matched on Aj, t-1. In one case, the difference in Aj, t-1 between a TAP school and its matched comparison school was 17.5 percentile points. Moreover, the metric is perverse: A school that started just below the 85th percentile will achieve a much larger proportional reduction than a school with the same value of Ajt – Aj, t-1 but a value of Aj, t-1 farther from T. Thus, gains count most in schools that were initially doing best.

The same metric was used in a second study that examined twelve TAP schools in two states—six in Arizona and six in South Carolina (Schacter et al. 2004). The gap reduction technique was used to evaluate TAP in the Arizona sample.12 Differences in initial percentile ranks (Aj, t-1) between TAP schools and control schools ranged from –13 to 12 percentile points for the Arizona sample. In 57 percent of the individually matched cases, the non-TAP school had the larger denominator. In these instances, the average difference in Aj, t-1 was 7.44 percentile points.

The third study, the most recent one reported by MFF (Solmon et al. 2007), compared TAP to non-TAP teachers and schools using the SAS EVAAS methodology developed by Wright et al. (2010).13 TAP teachers outperformed comparison group teachers 63 percent of the time, and TAP schools outperformed comparison group schools 57 percent of the time in mathematics and 67 percent of the time in reading.14

The Solmon et al. (2007) report contains no discussion of how the comparison sample was chosen, however. It appears the sample of control group schools was one of convenience; that is, all non-TAP schools with a valid EVAAS score located in the same state as a TAP school were included in the control group.

Finding an appropriate comparison group for TAP schools is a critical issue. The process schools go through to select into TAP raises a strong likelihood that these are distinctive schools. For example, Glazerman et al. (2006, p. 9) remarked that:

Selection as a TAP school occurs via a competitive process. Typically, a state department of education or district superintendent invites schools to learn about TAP and apply for the program. Candidate TAP schools also need to show an ability to provide financial support for the program. Ultimately, selection as a TAP school depends on the ability of the schools to implement, fund, and sustain the program, as well as on demonstrated faculty support.

In 2009, Glazerman and colleagues reported findings from an impact evaluation of the TAP in the Chicago Public Schools system. At the beginning of the 2007–08 school year, sixteen schools were randomly assigned to either the TAP intervention or the control condition. Another sixteen schools were then recruited and randomly assigned to the TAP intervention or control conditions at the beginning of the 2009–10 and 2010–11 school years. After two years of implementation, Glazerman et al. (2009) report an indeterminate effect on both student outcomes as well as teacher behavior.15

Our review of relevant studies on the impact of TAP on student outcomes highlights several limitations that warrant further research on the topic. First, except for the Glazerman et al. (2009) study, all evaluations to date have been conducted by MFF. Second, two of the studies use an idiosyncratic metric to compare TAP and non-TAP schools without ensuring the two groups are treated even-handedly. Finally, only the Glazerman et al. (2009) study appears to identify an appropriate comparison group of schools.

4.  Analytical Strategy

This research seeks to estimate a TAP treatment effect by comparing student test score gains in schools that participated in TAP with student test score gains in non-TAP schools. Our general model of student achievement gains is:
formula
2
where is the fall-to-spring test score gain in reading, mathematics, or language arts for student i attending school j in year t; Xit is a vector of student characteristics; Wjt is a vector of school characteristics; Tjt is a dummy variable indicating whether school j had become a TAP school by year t; uj is the influence of unobserved time-invariant characteristics of school j; uit is a student by year effect; ujt is a school by year effect; and uijt is an independent error.16 Denote . We assume Xit and Wjt are uncorrelated with . This is an unrestrictive assumption, given that we place no causal interpretation on the coefficients of these variables.
To develop a model of TAP participation, we begin by examining participation patterns over time. These are summarized in table 2. Participation in TAP has increased from ten schools in 2002–03 school year to fifty schools in the 2007–08 school year. This is consistent with a model in which resistance to participation is high but declining. Only two schools in these states have abandoned TAP, suggesting that schools do not typically reconsider this decision, or that the factors that might lead them to reverse the decision have a negligible variance over time: Once in, schools stay in TAP. These facts shape our specification of the participation model, as follows:
formula
3
formula
3a
formula
3b
Table 2.
TAP Participation (States A and B)
YearJoinedLeftTotal
2002–03 10 — 10 
2003–04  5 14 
2004–05  9 23 
2005–06  4 26 
2006–07  7 33 
2007–08 18 50 
YearJoinedLeftTotal
2002–03 10 — 10 
2003–04  5 14 
2004–05  9 23 
2005–06  4 26 
2006–07  7 33 
2007–08 18 50 
where Zj0 are observed characteristics of school j in baseline year 0 (2001–02); cs is the perceived “cost” of participating in TAP as of year s, monotonically decreasing in s; and ej is an unobserved school effect.

T* is a latent variable representing a school's disposition toward the TAP program. By assumption, schools do not change in this regard, at least over our sample period. Thus, the schools that are most favorably disposed toward TAP at the outset remain most favorably disposed. However, actual participation rates start out quite low, possibly due to teachers’ uncertainty and suspicions about the nature and effect of the program. Over time, these perceived “costs” of participation have declined, leading more of the schools with high values of to join. We estimate cs as the coefficient on a dummy variable for year s in an equation predicting the year in which a school adopts TAP. Note that once a school adopts TAP it continues to be identified as a TAP school for our purposes, even if it subsequently drops out of the program. Although this does some violence to reality, allowing for separate treatment of dropouts would considerably complicate the model with little benefit, given that so few schools have reversed this decision.

The assumption that participation depends only on characteristics in a baseline year may be unduly restrictive. A more conventional specification would allow for T* to vary over time; thus, = Zjtγ + ejt + ej. However, most of the candidate variables for Z change slowly over time, and as just noted, the variance of ejt must be quite small relative to ej or there would be more reversals of the participation decision. Nonetheless, it is possible that teachers decide to join TAP following an upturn in test scores that encourages them to think they will qualify for future bonuses. Given the noisiness of test results, this expectation is not likely to be met. As scores revert back to more normal levels, it will appear that TAP has been ineffective—a negative bias in the estimated treatment effect caused by mean reversion. We look for evidence of such a bias in the subsequent sensitivity tests.17

There are some natural restrictions on the relationships among the error terms that appear in the model. First, as the ujt are deviations about the average uj, cov(uj,ujt) = cov(ej,ujt) = 0. In addition, as we restrict the sample to students who remain in the same school for an entire year, the mean of uit over students in a given school in a given year is absorbed in ujt. Thus, cov(uit,ujt) = cov(uj,uit) = cov(ej,uit) = 0 as well.18 We leave open the possibility that uj is correlated with ej; indeed, it is just this possibility that gives rise to the selection effects discussed earlier. Specifically, if early-adopters of TAP are schools where achievement differs systematically from later adopters (for example, they might be underachieving schools), then the year of adoption contains information on uj. Our selection correction estimator exploits that information.

Thus, how we estimate this model depends on additional assumptions about the relationships among these unobservables.

Assumption 1: Ordinary Least Squares Regression

Assume that uj is uncorrelated with ej; that is, once we have conditioned on X and W, there are no selection effects. This assumption is plausible if W is a large set of school characteristics that includes Z as well as any other pre-existing differences between TAP and non-TAP schools that affect Y. Under Assumption 1, OLS estimation of equation 2 yields an unbiased estimate of α. In our OLS estimates, standard errors are corrected for clustering of students within schools.

Assumption 2: School-Fixed Effects Estimator

Alternatively, assume cov(ej,uj) ≠ 0. Differencing out uj through the inclusion of school fixed effects yields an unbiased estimate of α, which is identified through variation in outcomes over time within schools that enter the TAP program.

As an alternative to school fixed effects, we construct a matched sample of TAP and non-TAP schools. This gives us a comparison group that in important respects “looks like” schools that adopted TAP. We match on four variables from the 2001–02 school year immediately prior to our sample period: the percentage of minority students at the school, the percentage of students eligible for the free and reduced-price lunch program, and the percentages of a school's students scoring below basic on state achievement tests in mathematics and reading.19 The use of a matched sample guards against specification error in equation 2, for example, the assumption of linearity. However, the matched-sample estimator controls for self-selection of TAP schools only with respect to observable variables. We might therefore anticipate that results using this estimator will fall somewhere between those obtained using OLS on the full sample and those obtained using the fixed-effects estimator. By and large, this expectation is confirmed.

Assumption 3: Ordered-Probit Selection Correction Model

Finally, assume (, ej) have a bivariate normal distribution with covariance .

We estimate equation 3 as an ordered probit model, obtaining an estimate of the expectation of ej conditional on the decision to adopt TAP in year s. For schools that adopt TAP in year 1 of our sample (there were no TAP schools prior to that year in these states), this is ; for schools joining in the second year, this is , and so forth. For schools that remain outside TAP throughout the period, the term is .20 We introduce this expectation into the structural achievement equation, which becomes:
formula
4
where is our estimate of and is an estimate of . The new error term, , is asymptotically uncorrelated with by construction. Estimation of equation 4 by OLS yields a consistent estimate of α.

is weakly identified through the fact that is a nonlinear function of the variables in the participation model. Stronger identification requires that there be at least one variable in the participation model that is not in the structural achievement equation. In our case, we obtain this identification through the year-of-adoption effects, cs. Note that we can still include year effects in the achievement equation without affecting identification, as is a function of the year in which a school joined TAP, not the current year.

This identification strategy is not robust to all alternative specifications of the achievement model. If there are cohort TAP effects, that is, how a school responds to TAP depends on the year in which it adopts TAP (and we seek to estimate them by including a TAP-cohort interaction in the model), then our identification strategy does not identify selection effects, as all schools in the same cohort with the same value of Z also have the same value of . We are aware of no reason to suspect a priori that there are cohort effects, however, and their existence would appear to be a distinctly second-order concern compared with estimation of a TAP main effect.

More generally, both of our strategies for dealing with selection into TAP—the fixed effects estimator and the selection correct model—will fail to identify TAP effects when unobserved events with an independent effect on achievement occur at the same time as TAP adoption. For example, suppose TAP adoption coincides with the arrival of a dynamic new principal who carries out a variety of other measures, in addition to TAP, that help turn a school around. Because the fixed effects estimator controls only for time invariant unobservables, such consequences would be confounded with the effect of TAP in the fixed effects estimator. Likewise, year of adoption may no longer be a valid excluded instrument in the selection correction model.21 Neither of our identification strategies is robust to these sorts of confounding influences.

Our model of student achievement gains includes student gender and race/ethnicity. School level controls include the percentages of students by race/ethnicity, the percentage of students eligible for the free price lunch program, average teacher salary, the student teacher ratio, and percentages of students scoring basic and below basic under the state's accountability program in the 2001–02 school year, the last year prior to the introduction of TAP. We include controls for the percentage of students tested in case mean scores were affected by the exclusion of some students from testing, but these variables were never significant.22

Mean gains vary substantially by grade. When pooled across all years, the average fall-to-spring gain score ranges from 2.66 to 13.79 points. The magnitude of the fall-to-spring gain decreases monotonically from low to high grades, with the average eighth-grade gain less than half the size of the average third-grade gain. Effects of covariates may also differ by grade. We therefore estimate separate equations for each grade.

All models also include state by year effects to control for changes in the test, changes in how well aligned the test is with curricula, and student cohort effects. These variables also control for changes in the composition of the sample, as described in the next section under the subsection Sample.

Participation in TAP is a function of school characteristics in the baseline school year (2001–02): student–teacher ratio, average teacher salary, percentages of minority students and students eligible for the free or reduced-price lunch program, and percentages of students scoring basic or below basic in English and in mathematics.

5.  Data and Sample

Data

The primary data for this study are drawn from the Northwest Evaluation Association's (NWEA) Growth Research Database (GRD). The GRD contains longitudinal student test score results from approximately 2,200 school districts in forty-five states. The NWEA test is administered twice per year, allowing for construction of a fall-to-spring gain score for each student. All scores reference a single cross-grade, equal-interval scale developed using a one parameter Raasch model (Kingsbury 2003). The GRD also contains a limited number of student characteristics, notably race/ethnicity, gender, grade, and date of birth.

We supplement GRD with publicly available school report card data from state department of education Web sites and information from the National Center for Education Statistics's Common Core of Data (CCD). State school report cards include information on average teacher salary, student attendance, student–teacher ratio, and aggregate school performance on a state's high-stakes assessments. CCD data contain fiscal and non-fiscal information on schools and school districts, including students and school personnel.

Sample

Our sample includes roughly 1,200 schools from two states over a five-year period comprising the 2002–03 to 2006–07 school years. By contrast, the matched sample contains only 199 non-TAP schools. The number of TAP schools in our sample increases from six in the 2002–03 school year to thirty-two in the 2006–07 school year, and the number of TAP student observations with valid fall and spring test scores in mathematics rises from 663 in the 2002–03 school year to 8,496 in the 2006–07 school year.23 Although all of the TAP schools in these states had contracts with NWEA, the NWEA tests are not the high-stakes exams used to determine which teachers earn bonuses. Thus, there is no particular reason for teachers to teach to the NWEA exams or otherwise manipulate scores on these tests. Indeed, because these exams are used mainly for diagnostic and formative assessments, teachers have every reason to see that an accurate reading is obtained from these exams on all of their students.

The number of districts contracting for testing services with NWEA increased at the same time as the number of TAP schools. As a result, the set of comparison schools varied across years. To illustrate, in table 3 we present the number of TAP and non-TAP schools by state and year for grades 3 and 8. We also present average mathematics scores (spring) and fall-to-spring gains. Gain scores in second grade trended upward in the non-TAP schools. The opposite trend prevailed in eighth grade. The same contrast is evident between other elementary and secondary grades. Though trends are less clear in the much smaller number of TAP schools, there are some sizeable differences across years that could be related to the entry of new TAP schools. These year-to-year differences are controlled for in our model's state-by-year effects.

Table 3.
Sample Make-Up
Grade 3Grade 8
StateYearNo. SchoolsNo. StudentsMean Spring ScoreMean GainNo. SchoolsNo. StudentsMean Spring ScoreMean Gain
Non-TAP Schools          
2002–03 57 3,395 204.4 9.2 19 3,274 236 4.6 
 2003–04 129 9,835 203.8 9.8 59 10,177 235.2 4.7 
 2004–05 299 21,934 203.7 9.9 133 21,958 234.2 3.9 
 2005–06 395 31,213 203.4 9.9 176 32,595 233.4 3.8 
 2006–07 483 39,214 202.1 9.9 214 38,856 234.2 3.5 
2002–03 176 5,875 196.6 9.5 76 5,667 230.3 5.2 
 2003–04 246 8,429 199.1 10.6 92 7,723 231.8 5.9 
 2004–05 161 5,938 199.4 11.6 71 4,307 231.6 5.4 
 2005–06 220 8,457 201.1 11.6 108 6,049 231.6 4.8 
 2006–07 310 13,154 201.8 11.5 150 10,922 233.9 4.4 
TAP Schools          
2002–03 
 2003–04 188 205.6 13.5 
 2004–05 212 203 10.3 372 231.7 2.5 
 2005–06 352 200.9 10.3 816 229.9 5.1 
 2006–07 572 196.2 9.5 968 229.1 2.7 
2002–03 121 201.1 13.1 52 228.3 6.2 
 2003–04 232 199.3 13.9 209 240 8.5 
 2004–05 340 203 13.9 292 237.1 8.8 
 2005–06 340 202.8 13.2 344 239 6.7 
 2006–07 350 200.7 13.4 326 233.1 8.8 
Grade 3Grade 8
StateYearNo. SchoolsNo. StudentsMean Spring ScoreMean GainNo. SchoolsNo. StudentsMean Spring ScoreMean Gain
Non-TAP Schools          
2002–03 57 3,395 204.4 9.2 19 3,274 236 4.6 
 2003–04 129 9,835 203.8 9.8 59 10,177 235.2 4.7 
 2004–05 299 21,934 203.7 9.9 133 21,958 234.2 3.9 
 2005–06 395 31,213 203.4 9.9 176 32,595 233.4 3.8 
 2006–07 483 39,214 202.1 9.9 214 38,856 234.2 3.5 
2002–03 176 5,875 196.6 9.5 76 5,667 230.3 5.2 
 2003–04 246 8,429 199.1 10.6 92 7,723 231.8 5.9 
 2004–05 161 5,938 199.4 11.6 71 4,307 231.6 5.4 
 2005–06 220 8,457 201.1 11.6 108 6,049 231.6 4.8 
 2006–07 310 13,154 201.8 11.5 150 10,922 233.9 4.4 
TAP Schools          
2002–03 
 2003–04 188 205.6 13.5 
 2004–05 212 203 10.3 372 231.7 2.5 
 2005–06 352 200.9 10.3 816 229.9 5.1 
 2006–07 572 196.2 9.5 968 229.1 2.7 
2002–03 121 201.1 13.1 52 228.3 6.2 
 2003–04 232 199.3 13.9 209 240 8.5 
 2004–05 340 203 13.9 292 237.1 8.8 
 2005–06 340 202.8 13.2 344 239 6.7 
 2006–07 350 200.7 13.4 326 233.1 8.8 

Testing dates vary considerably by school. Average time elapsed between fall and spring testing is about 212 calendar days in TAP schools, with a standard deviation of slightly more than three weeks. Average time elapsed for non-TAP schools is about 194 calendar days, with a standard deviation of 27.78 days. Recognizing that gains are positively correlated with time elapsed since the previous test administration, we include this variable in all model specifications.

Because familiarity with a test is generally associated with rising scores, we expect that the more frequently a school has used NWEA tests in the past, the higher scores will be. We define NWEA cohort based on the year that a school first begins using NWEA tests. Dummy variables for the 2003, 2004, 2005, and 2006 cohorts are included in the model.

Table 4 summarizes the means and standard deviations of key school and student variables. TAP schools in State A have a greater percentage of black students (60.59 vs. 37.34) compared with other schools in the state. The same holds true for free lunch status students (60.99 vs. 43.08) and students scoring below basic on the high-stakes assessment (43.15 vs. 28.79). Other school and student covariates tend to be very similar between TAP and non-TAP schools in State A.

Table 4.
Select Sample Statistics (Adding Non-TAP Matched Sample)
State AState BState A and State B
Non-TAPNon-TAPNon-TAP
Non-TAPMatchedNon-TAPMatchedNon-TAPMatched
TAPAllSampleTAPAllSampleTAPAllSample
School Variables          
  Average Teacher Salary/100 41.4661 42.2509 41.1253 44.0304 42.2640 44.2800 42.6290 42.2546 41.6371 
 (2.0402) (2.5576) (2.5984) (2.8856) (5.9942) (6.0818) (2.7713) (3.8609) (3.6070) 
  Student–Teacher Ratio 14.2311 15.1572 14.4791 14.3514 16.5263 16.8066 14.2857 15.5475 14.8568 
 (1.9967) (1.8309) (2.0154) (1.4694) (7.9787) (3.1732) (1.7781) (4.5748) (2.4026) 
  Percent Asian 0.5810 1.3449 1.0022 0.7598 2.2480 3.0105 0.6621 1.6023 1.3281 
 (0.5971) (1.4289) (1.2739) (0.5599) (2.7150) (2.1038) (0.5873) (1.9306) (1.6204) 
  Percent Hispanic 2.8726 4.1328 4.6775 48.3527 31.6667 32.5370 23.4972 11.9814 9.1980 
 (3.5905) (4.9403) (6.0059) (16.0123) (23.2051) (21.2367) (25.2184) (18.0401) (14.4533) 
  Percent Black 60.5920 37.3409 56.2715 0.4713 3.5373 10.4207 33.3280 27.7051 48.8318 
 (19.1620) (23.5781) (22.6908) (0.3869) (6.9244) (11.8699) (33.1145) (25.3772) (27.2022) 
  Percent Free/Reduced Lunch 60.9929 43.0831 59.1779 22.7651 32.2977 25.0810 43.6571 40.0087 53.6454 
 (15.8674) (19.8725) (19.7163) (12.0029) (21.1330) (21.3525) (23.7722) (20.8172) (23.6150) 
  Percent Tested 99.7181 99.3529 99.3169 98.1580 98.8468 98.5987 99.0181 99.2174 99.2035 
 (0.7331) (3.0407) (2.7322) (4.2613) (4.5189) (5.0517) (3.0075) (3.5052) (3.2222) 
  Percent Below Basic (Math) 43.1514 28.7932 39.5153 16.0086 21.2033 21.0765 30.3854 26.4862 36.5212 
 (12.3013) (13.7996) (12.7680) (9.0382) (14.1963) (12.3347) (17.3813) (14.3524) (14.4048) 
  Percent Basic (Math) 36.9646 39.2737 40.2417 32.8451 33.5400 32.6728 35.0271 37.5309 39.0127 
 (5.0693) (5.5177) (5.7811) (7.0237) (9.3029) (9.2508) (6.4063) (7.3792) (7.0485) 
Student Variables          
  Male 0.5057 0.5087 0.5035 0.5132 0.5086 0.5045 0.5091 0.5087 0.5036 
 (0.5000) (0.4999) (0.5000) (0.4998) (0.4999) (0.5000) (0.4999) (0.4999) (0.5000) 
  Black 0.5902 0.3527 0.5405 0.0040 0.0356 0.1055 0.3243 0.2631 0.4698 
 (0.4918) (0.4778) (0.4984) (0.0629) (0.1852) (0.3072) (0.4681) (0.4403) (0.4991) 
  Hispanic 0.0280 0.0397 0.0454 0.4816 0.2924 0.2972 0.2337 0.1110 0.0863 
 (0.1651) (0.1952) (0.2082) (0.4997) (0.4549) (0.4570) (0.4232) (0.3142) (0.2808) 
  Asian 0.0051 0.0113 0.0090 0.0079 0.0299 0.0360 0.0063 0.0165 0.0134 
 (0.0709) (0.1056) (0.0944) (0.0883) (0.1702) (0.1862) (0.0793) (0.1275) (0.1149) 
  American Indian/Native Alaskan 0.0026 0.0023 0.0045 0.0045 0.0173 0.0069 0.0035 0.0065 0.0049 
 (0.0513) (0.0476) (0.0666) (0.0669) (0.1302) (0.0829) (0.0589) (0.0804) (0.0695) 
  Other 0.0007 0.0041 0.0042 0.0018 0.0425 0.0366 0.0012 0.0150 0.0095 
 (0.0257) (0.0640) (0.0647) (0.0420) (0.2018) (0.1879) (0.0340) (0.1214) (0.0968) 
  Time Elapsed Between Fall and Spring Assessment 195.9135 185.5276 185.1508 231.7004 217.8205 223.5608 212.1424 194.6451 191.3878 
 (19.7490) (22.3877) (22.0279) (10.3309) (26.6779) (22.6916) (24.0616) (27.7839) (26.2814) 
Dependent Variable          
Fall-to-Spring Test Score Gain          
  Grade 2 13.4558 11.7771 11.2715 18.1887 13.6778 13.7082 16.0003 12.0861 11.4397 
 (7.6417) (7.5815) (7.5823) (8.0098) (8.4851) (9.0865) (8.1878) (7.7673) (7.7200) 
  Grade 3 10.4077 9.8799 9.3850 13.5360 11.0645 11.4859 12.0059 10.2161 9.6503 
 (7.6431) (7.4116) (7.4325) (7.3501) (7.7462) (7.9416) (7.6549) (7.5270) (7.5309) 
  Grade 4 9.2324 7.4809 7.0451 11.3943 9.0749 8.6332 10.1238 7.9269 7.2244 
 (7.8399) (7.4646) (7.5448) (7.3740) (7.5767) (7.6475) (7.7238) (7.5302) (7.5729) 
  Grade 5 8.7361 6.9709 6.5197 11.0993 8.8039 8.1421 9.7172 7.4929 6.7373 
 (7.8454) (7.5764) (7.7353) (7.3505) (7.6136) (7.7759) (7.7309) (7.6319) (7.7603) 
  Grade 6 5.6734 4.9464 4.8148 9.1642 6.6641 5.9974 6.9052 5.3768 5.0049 
 (8.1351) (7.8125) (7.9814) (7.6204) (7.6506) (7.4899) (8.1292) (7.8078) (7.9162) 
  Grade 7 3.8654 4.3785 4.1167 7.9489 5.9036 5.8066 5.3470 4.7557 4.4069 
 (8.2977) (7.9728) (8.0860) (7.3002) (7.8694) (7.4756) (8.1881) (7.9745) (8.0097) 
  Grade 8 3.5834 3.8140 3.8721 8.0592 5.0560 4.6258 5.2034 4.1182 3.9998 
 (8.5710) (8.0361) (8.1528) (7.7116) (8.0566) (7.5559) (8.5443) (8.0588) (8.0595) 
  Grade 9 1.1235 1.5039 3.6158 3.3779 2.5171 3.4214 2.6697 1.7887 3.4964 
 (7.9740) (9.4487) (8.9580) (8.1998) (9.0442) (9.5372) (8.1939) (9.3479) (9.3171) 
  Grade 10 0.2601 1.6383 4.8204 4.4981 1.7711 3.1489 3.7524 1.7009 3.2936 
 (8.4438) (9.4792) (7.4675) (10.2352) (9.5198) (9.4939) (10.0702) (9.4985) (9.3454) 
State AState BState A and State B
Non-TAPNon-TAPNon-TAP
Non-TAPMatchedNon-TAPMatchedNon-TAPMatched
TAPAllSampleTAPAllSampleTAPAllSample
School Variables          
  Average Teacher Salary/100 41.4661 42.2509 41.1253 44.0304 42.2640 44.2800 42.6290 42.2546 41.6371 
 (2.0402) (2.5576) (2.5984) (2.8856) (5.9942) (6.0818) (2.7713) (3.8609) (3.6070) 
  Student–Teacher Ratio 14.2311 15.1572 14.4791 14.3514 16.5263 16.8066 14.2857 15.5475 14.8568 
 (1.9967) (1.8309) (2.0154) (1.4694) (7.9787) (3.1732) (1.7781) (4.5748) (2.4026) 
  Percent Asian 0.5810 1.3449 1.0022 0.7598 2.2480 3.0105 0.6621 1.6023 1.3281 
 (0.5971) (1.4289) (1.2739) (0.5599) (2.7150) (2.1038) (0.5873) (1.9306) (1.6204) 
  Percent Hispanic 2.8726 4.1328 4.6775 48.3527 31.6667 32.5370 23.4972 11.9814 9.1980 
 (3.5905) (4.9403) (6.0059) (16.0123) (23.2051) (21.2367) (25.2184) (18.0401) (14.4533) 
  Percent Black 60.5920 37.3409 56.2715 0.4713 3.5373 10.4207 33.3280 27.7051 48.8318 
 (19.1620) (23.5781) (22.6908) (0.3869) (6.9244) (11.8699) (33.1145) (25.3772) (27.2022) 
  Percent Free/Reduced Lunch 60.9929 43.0831 59.1779 22.7651 32.2977 25.0810 43.6571 40.0087 53.6454 
 (15.8674) (19.8725) (19.7163) (12.0029) (21.1330) (21.3525) (23.7722) (20.8172) (23.6150) 
  Percent Tested 99.7181 99.3529 99.3169 98.1580 98.8468 98.5987 99.0181 99.2174 99.2035 
 (0.7331) (3.0407) (2.7322) (4.2613) (4.5189) (5.0517) (3.0075) (3.5052) (3.2222) 
  Percent Below Basic (Math) 43.1514 28.7932 39.5153 16.0086 21.2033 21.0765 30.3854 26.4862 36.5212 
 (12.3013) (13.7996) (12.7680) (9.0382) (14.1963) (12.3347) (17.3813) (14.3524) (14.4048) 
  Percent Basic (Math) 36.9646 39.2737 40.2417 32.8451 33.5400 32.6728 35.0271 37.5309 39.0127 
 (5.0693) (5.5177) (5.7811) (7.0237) (9.3029) (9.2508) (6.4063) (7.3792) (7.0485) 
Student Variables          
  Male 0.5057 0.5087 0.5035 0.5132 0.5086 0.5045 0.5091 0.5087 0.5036 
 (0.5000) (0.4999) (0.5000) (0.4998) (0.4999) (0.5000) (0.4999) (0.4999) (0.5000) 
  Black 0.5902 0.3527 0.5405 0.0040 0.0356 0.1055 0.3243 0.2631 0.4698 
 (0.4918) (0.4778) (0.4984) (0.0629) (0.1852) (0.3072) (0.4681) (0.4403) (0.4991) 
  Hispanic 0.0280 0.0397 0.0454 0.4816 0.2924 0.2972 0.2337 0.1110 0.0863 
 (0.1651) (0.1952) (0.2082) (0.4997) (0.4549) (0.4570) (0.4232) (0.3142) (0.2808) 
  Asian 0.0051 0.0113 0.0090 0.0079 0.0299 0.0360 0.0063 0.0165 0.0134 
 (0.0709) (0.1056) (0.0944) (0.0883) (0.1702) (0.1862) (0.0793) (0.1275) (0.1149) 
  American Indian/Native Alaskan 0.0026 0.0023 0.0045 0.0045 0.0173 0.0069 0.0035 0.0065 0.0049 
 (0.0513) (0.0476) (0.0666) (0.0669) (0.1302) (0.0829) (0.0589) (0.0804) (0.0695) 
  Other 0.0007 0.0041 0.0042 0.0018 0.0425 0.0366 0.0012 0.0150 0.0095 
 (0.0257) (0.0640) (0.0647) (0.0420) (0.2018) (0.1879) (0.0340) (0.1214) (0.0968) 
  Time Elapsed Between Fall and Spring Assessment 195.9135 185.5276 185.1508 231.7004 217.8205 223.5608 212.1424 194.6451 191.3878 
 (19.7490) (22.3877) (22.0279) (10.3309) (26.6779) (22.6916) (24.0616) (27.7839) (26.2814) 
Dependent Variable          
Fall-to-Spring Test Score Gain          
  Grade 2 13.4558 11.7771 11.2715 18.1887 13.6778 13.7082 16.0003 12.0861 11.4397 
 (7.6417) (7.5815) (7.5823) (8.0098) (8.4851) (9.0865) (8.1878) (7.7673) (7.7200) 
  Grade 3 10.4077 9.8799 9.3850 13.5360 11.0645 11.4859 12.0059 10.2161 9.6503 
 (7.6431) (7.4116) (7.4325) (7.3501) (7.7462) (7.9416) (7.6549) (7.5270) (7.5309) 
  Grade 4 9.2324 7.4809 7.0451 11.3943 9.0749 8.6332 10.1238 7.9269 7.2244 
 (7.8399) (7.4646) (7.5448) (7.3740) (7.5767) (7.6475) (7.7238) (7.5302) (7.5729) 
  Grade 5 8.7361 6.9709 6.5197 11.0993 8.8039 8.1421 9.7172 7.4929 6.7373 
 (7.8454) (7.5764) (7.7353) (7.3505) (7.6136) (7.7759) (7.7309) (7.6319) (7.7603) 
  Grade 6 5.6734 4.9464 4.8148 9.1642 6.6641 5.9974 6.9052 5.3768 5.0049 
 (8.1351) (7.8125) (7.9814) (7.6204) (7.6506) (7.4899) (8.1292) (7.8078) (7.9162) 
  Grade 7 3.8654 4.3785 4.1167 7.9489 5.9036 5.8066 5.3470 4.7557 4.4069 
 (8.2977) (7.9728) (8.0860) (7.3002) (7.8694) (7.4756) (8.1881) (7.9745) (8.0097) 
  Grade 8 3.5834 3.8140 3.8721 8.0592 5.0560 4.6258 5.2034 4.1182 3.9998 
 (8.5710) (8.0361) (8.1528) (7.7116) (8.0566) (7.5559) (8.5443) (8.0588) (8.0595) 
  Grade 9 1.1235 1.5039 3.6158 3.3779 2.5171 3.4214 2.6697 1.7887 3.4964 
 (7.9740) (9.4487) (8.9580) (8.1998) (9.0442) (9.5372) (8.1939) (9.3479) (9.3171) 
  Grade 10 0.2601 1.6383 4.8204 4.4981 1.7711 3.1489 3.7524 1.7009 3.2936 
 (8.4438) (9.4792) (7.4675) (10.2352) (9.5198) (9.4939) (10.0702) (9.4985) (9.3454) 

Note: Standard errors in parentheses.

TAP schools in State B tend to have a greater percentage of Hispanic students (48.36 vs. 31.67) when compared with the state-wide average. Non-TAP schools in State B have higher percentages of black (3.53 vs. 0.47) and free lunch status (32.30 vs. 22.77) students and more students scoring below basic on the state high-stakes assessment (21.20 vs. 16.01). Other school and student covariates tend to be very similar between TAP and non-TAP schools in State B. In both states, non-TAP schools in the matched sample are much more like TAP schools, as one would expect. They are very similar with respect to percentage of students eligible for the free and reduced-price lunch program and with respect to overall percentage minority, though in State B non-TAP schools have a somewhat lower percentage of Hispanic students and a somewhat higher percentage of black students.

Table 4 also summarizes student test score information. TAP schools have modestly larger test score gains when compared with the average test score gain in their respective state. TAP schools in both State A and State B have over three weeks more time between the fall and spring administration of the NWEA test. Nearly all students were tested in both fall and spring, irrespective of whether they attended a TAP school (99.02 vs. 99.22).

6.  Results

Table 5 reports OLS regression estimates with robust standard errors to correct for clustering of students within schools. At all grade levels, there is a positive association between TAP and a student's fall-to-spring test score gain. Although the positive coefficients for the seventh through ninth grade-level models fail to attain conventional levels of statistical significance, the dominant impression is positive. The largest effect is in second grade (2.72 points), though differences of between 1 and 2 points are more common. Given that the standard deviation of fall-to-spring test score gains is approximately 7 to 8 points in the elementary and middle school grades, statistically significant effect sizes range from 12 (grade 3) to 34 percent (grade 2).

Table 5.
Fall-to-Spring Mathematics Gains, OLS on Full Sample
Grade
 2 3 4 5 6 7 8 9 10 
Independent Variables          
  TAP 2.7156*** 0.9503** 1.8694*** 2.0434*** 0.9737** 0.2651 0.9218 0.8770 2.6224** 
 (0.4493) (0.3362) (0.2854) (0.2943) (0.3526) (0.3795) (0.4857) (0.8172) (0.9202) 
  Time Elapsed Between Fall and Spring Assessment 0.0522*** 0.0396*** 0.0380*** 0.0318*** 0.0227*** 0.0205*** 0.0106* 0.0084 –0.0048 
 (0.0037) (0.0032) (0.0032) (0.0031) (0.0032) (0.0032) (0.0042) (0.0066) (0.0076) 
  Average Teacher Salary/1000 –0.0427 0.0413* 0.0070 0.0213 0.0268 0.0396* 0.0371 0.0192 0.0600 
 (0.0263) (0.0193) (0.0178) (0.0172) (0.0198) (0.0198) (0.0243) (0.0355) (0.0455) 
  Student–Teacher Ratio –0.0290 –0.0446 –0.0692 –0.0588 –0.1412*** –0.0223 0.0338 –0.1141 –0.1225 
 (0.0577) (0.0353) (0.0419) (0.0347) (0.0385) (0.0374) (0.0433) (0.0696) (0.0829) 
  Percent Below Basic (Math) –0.0356** –0.0289** –0.0349*** –0.0467*** –0.0359*** –0.0329*** –0.0300** –0.0161 –0.0214 
 (0.0123) (0.0101) (0.0085) (0.0091) (0.0090) (0.0077) (0.0099) (0.0150) (0.0206) 
  Percent Basic (Math) –0.0372* –0.0331** –0.0147 –0.0282** –0.0152 –0.0075 –0.0027 –0.0014 –0.0045 
 (0.0155) (0.0117) (0.0103) (0.0095) (0.0124) (0.0137) (0.0159) (0.0226) (0.0238) 
  Intercept 6.7262** 3.2634 1.4495 2.5633 5.6942** 2.6739 –0.7027 4.5411 3.7969 
 (2.2387) (1.6636) (1.5491) (1.5260) (1.9491) (1.7837) (2.1677) (2.6983) (3.5404) 
  R2 0.0551 0.0345 0.0342 0.0324 0.0231 0.0185 0.0135 0.0140 0.0123 
  N 90,555 127,474 128,567 135,480 135,781 132,893 129,642 25,468 16,849 
Grade
 2 3 4 5 6 7 8 9 10 
Independent Variables          
  TAP 2.7156*** 0.9503** 1.8694*** 2.0434*** 0.9737** 0.2651 0.9218 0.8770 2.6224** 
 (0.4493) (0.3362) (0.2854) (0.2943) (0.3526) (0.3795) (0.4857) (0.8172) (0.9202) 
  Time Elapsed Between Fall and Spring Assessment 0.0522*** 0.0396*** 0.0380*** 0.0318*** 0.0227*** 0.0205*** 0.0106* 0.0084 –0.0048 
 (0.0037) (0.0032) (0.0032) (0.0031) (0.0032) (0.0032) (0.0042) (0.0066) (0.0076) 
  Average Teacher Salary/1000 –0.0427 0.0413* 0.0070 0.0213 0.0268 0.0396* 0.0371 0.0192 0.0600 
 (0.0263) (0.0193) (0.0178) (0.0172) (0.0198) (0.0198) (0.0243) (0.0355) (0.0455) 
  Student–Teacher Ratio –0.0290 –0.0446 –0.0692 –0.0588 –0.1412*** –0.0223 0.0338 –0.1141 –0.1225 
 (0.0577) (0.0353) (0.0419) (0.0347) (0.0385) (0.0374) (0.0433) (0.0696) (0.0829) 
  Percent Below Basic (Math) –0.0356** –0.0289** –0.0349*** –0.0467*** –0.0359*** –0.0329*** –0.0300** –0.0161 –0.0214 
 (0.0123) (0.0101) (0.0085) (0.0091) (0.0090) (0.0077) (0.0099) (0.0150) (0.0206) 
  Percent Basic (Math) –0.0372* –0.0331** –0.0147 –0.0282** –0.0152 –0.0075 –0.0027 –0.0014 –0.0045 
 (0.0155) (0.0117) (0.0103) (0.0095) (0.0124) (0.0137) (0.0159) (0.0226) (0.0238) 
  Intercept 6.7262** 3.2634 1.4495 2.5633 5.6942** 2.6739 –0.7027 4.5411 3.7969 
 (2.2387) (1.6636) (1.5491) (1.5260) (1.9491) (1.7837) (2.1677) (2.6983) (3.5404) 
  R2 0.0551 0.0345 0.0342 0.0324 0.0231 0.0185 0.0135 0.0140 0.0123 
  N 90,555 127,474 128,567 135,480 135,781 132,893 129,642 25,468 16,849 

Notes: Models include year fixed effects, NWEA cohort fixed effects, school-level controls for race/ethnicity, special education, free/reduced lunch status, and student-level controls for race/ethnicity and gender. Standard errors reported in parentheses. Standard errors corrected for clustering of students within schools.

*Statistically significant at the 10% level; **statistically significant at the 5% level; ***statistically significant at the 1% level.

As explained previously, we first use a school fixed effects estimator to control for unobserved characteristics of schools that may explain selection into TAP as well as achievement. We then implement a two-step selection-correction estimator as a second way of controlling for selection bias.

Table 6 reports estimates from the school fixed effects models. After differencing out time invariant school characteristics that may explain selection into TAP, a clear division arises between elementary and secondary grades. At the elementary level (grades 2 through 5), the TAP effect continues to be positive, although coefficients are generally somewhat smaller than previously reported using OLS regression and significant in some instances, not all. However, coefficients in the sixth, seventh, ninth, and tenth grade-level models are now negative and statistically significant in seventh and ninth grades. The positive coefficient for eighth grade remains statistically insignificant.

Table 6.
Fall-to-Spring Mathematics Gains, School Fixed Effects Models
Grade
2345678910
Independent Variables          
  TAP 1.5729** 0.5391 1.0816** 1.1060** –0.4072 –1.3518*** 0.4340 –2.0587*** –0.9878 
 (0.5220) (0.4330) (0.3956) (0.3942) (0.3465) (0.3963) (0.4338) (0.6113) (0.7005) 
  Time Elapsed Between Fall and Spring Assessment 0.0530*** 0.0481*** 0.0357*** 0.0329*** 0.0238*** 0.0204*** 0.0115*** 0.0133*** –0.0129* 
 (0.0017) (0.0015) (0.0015) (0.0015) (0.0016) (0.0016) (0.0017) (0.0038) (0.0053) 
  Average Teacher Salary/1000 –0.0313 0.0594* –0.0022 –0.0222 –0.0132 0.0697* –0.0088 0.1776* 0.0608 
 (0.0381) (0.0263) (0.0269) (0.0259) (0.0337) (0.0345) (0.0359) (0.0886) (0.1152) 
  Student–Teacher Ratio –0.0286 –0.0822* –0.0867* –0.1124** –0.0922* –0.1078** –0.1486*** –0.2261** –0.1175 
 (0.0511) (0.0364) (0.0359) (0.0348) (0.0372) (0.0415) (0.0429) (0.0849) (0.1011) 
  Intercept 2.6213 –6.7418*** 0.5808 0.8578 3.8061 2.2590 –2.7491 –7.1900 2.8348 
 (3.1228) (1.7779) (1.8251) (1.7633) (2.1308) (2.1562) (2.2488) (5.1054) (6.9069) 
  R2 0.0241 0.0164 0.0102 0.0082 0.0046 0.0040 0.0028 0.0087 0.0055 
  N 90,555 127,474 128,567 135,480 135,781 132,893 129,642 25,468 16,849 
Grade
2345678910
Independent Variables          
  TAP 1.5729** 0.5391 1.0816** 1.1060** –0.4072 –1.3518*** 0.4340 –2.0587*** –0.9878 
 (0.5220) (0.4330) (0.3956) (0.3942) (0.3465) (0.3963) (0.4338) (0.6113) (0.7005) 
  Time Elapsed Between Fall and Spring Assessment 0.0530*** 0.0481*** 0.0357*** 0.0329*** 0.0238*** 0.0204*** 0.0115*** 0.0133*** –0.0129* 
 (0.0017) (0.0015) (0.0015) (0.0015) (0.0016) (0.0016) (0.0017) (0.0038) (0.0053) 
  Average Teacher Salary/1000 –0.0313 0.0594* –0.0022 –0.0222 –0.0132 0.0697* –0.0088 0.1776* 0.0608 
 (0.0381) (0.0263) (0.0269) (0.0259) (0.0337) (0.0345) (0.0359) (0.0886) (0.1152) 
  Student–Teacher Ratio –0.0286 –0.0822* –0.0867* –0.1124** –0.0922* –0.1078** –0.1486*** –0.2261** –0.1175 
 (0.0511) (0.0364) (0.0359) (0.0348) (0.0372) (0.0415) (0.0429) (0.0849) (0.1011) 
  Intercept 2.6213 –6.7418*** 0.5808 0.8578 3.8061 2.2590 –2.7491 –7.1900 2.8348 
 (3.1228) (1.7779) (1.8251) (1.7633) (2.1308) (2.1562) (2.2488) (5.1054) (6.9069) 
  R2 0.0241 0.0164 0.0102 0.0082 0.0046 0.0040 0.0028 0.0087 0.0055 
  N 90,555 127,474 128,567 135,480 135,781 132,893 129,642 25,468 16,849 

Notes: Models include year fixed effects, NWEA cohort fixed effects, school fixed effects, school-level controls for race/ethnicity, special education, free lunch status, and student-level controls for race/ethnicity and gender. Standard errors reported in parentheses.

*Statistically significant at the 10% level; **statistically significant at the 5% level; ***statistically significant at the 1% level.

It may be the case that our school fixed effects results are sensitive to the fact that eleven TAP schools, equivalent to 40 percent of our TAP school sample, do not change TAP status during the sample period. These schools make no contribution to the estimated TAP effect in the school fixed effects model. We checked the possibility that this accounts for the difference between the OLS and fixed effects estimates by dropping these TAP schools from the sample and re-estimating the OLS regressions. Results were very similar to those reported in table 5, indicating that the difference between OLS and fixed effects results was due to the inclusion of school effects, not to the loss of TAP school observations.

Results of the matched-sample estimator (table 7) are qualitatively similar to the school fixed-effects results. The main difference is that the estimate for sixth grade remains positive and statistically significant. However, point estimates in grades 7, 9, and 10 are negative (significantly so, in ninth grade), and the only positive estimate, in eighth grade, is quite small and insignificant.

Table 7.
Fall-to-Spring Mathematics Gains, OLS on Matched Sample
Grade
2345678910
Independent Variables          
  TAP 1.6710*** 0.4839 2.0159*** 1.8683*** 1.0059** –0.1784 0.1714 –2.5698* –1.9424 
 (0.4283) (0.4237) (0.3300) (0.3280) (0.3816) (0.4075) (0.4953) (0.9345) (1.5781) 
  Time Elapsed Between Fall and Spring Assessment 0.0529*** 0.0594*** 0.0390*** 0.0389*** 0.0216*** 0.0321*** 0.0215** 0.0052 –0.0033 
 (0.0074) (0.0065) (0.0052) (0.0052) (0.0063) (0.0063) (0.0075) (0.0155) (0.0242) 
  Average Teacher Salary/1000 –0.0724 0.0052 –0.0351 –0.0474 –0.0467 –0.0456 –0.0602 0.4411 0.3221 
 (0.0419) (0.0449) (0.0380) (0.0352) (0.0451) (0.0406) (0.0573) (0.3228) (0.9502) 
  Student–Teacher Ratio –0.0975 –0.0547 –0.0363 –0.0560 –0.2572*** –0.1147 –0.0857 –0.2513 0.6573** 
 (0.0707) (0.0563) (0.0557) (0.0611) (0.0557) (0.0643) (0.0730) (0.2760) (0.1683) 
  Percent Below Basic (Math) –0.0376 –0.0224 –0.0526** –0.0575*** –0.0479*** –0.0322* –0.0001 0.1160 –0.1409 
 (0.0220) (0.0182) (0.0162) (0.0164) (0.0137) (0.0133) (0.0168) (0.0843) (0.4223) 
  Percent Basic (Math) –0.0309 0.0130 –0.0184 –0.0517* –0.0213 –0.0037 –0.0204 –0.1758 0.0218 
 (0.0298) (0.0189) (0.0215) (0.0234) (0.0284) (0.0277) (0.0258) (0.1155) (0.1851) 
  Intercept 17.2437* 2.8397 9.2267* 8.2706** 14.1528** 7.8609 7.7407 –5.7948 –10.1036 
 (8.2219) (4.8563) (4.0021) (2.9791) (5.2625) (4.3879) (4.1517) (11.2772) (32.5989) 
  R2 0.0992 0.0672 0.0455 0.0447 0.0309 0.0273 0.0214 0.0262 0.0351 
  N 17,396 21,683 22,889 23,248 26,094 25,710 24,863 3,965 2,282 
Grade
2345678910
Independent Variables          
  TAP 1.6710*** 0.4839 2.0159*** 1.8683*** 1.0059** –0.1784 0.1714 –2.5698* –1.9424 
 (0.4283) (0.4237) (0.3300) (0.3280) (0.3816) (0.4075) (0.4953) (0.9345) (1.5781) 
  Time Elapsed Between Fall and Spring Assessment 0.0529*** 0.0594*** 0.0390*** 0.0389*** 0.0216*** 0.0321*** 0.0215** 0.0052 –0.0033 
 (0.0074) (0.0065) (0.0052) (0.0052) (0.0063) (0.0063) (0.0075) (0.0155) (0.0242) 
  Average Teacher Salary/1000 –0.0724 0.0052 –0.0351 –0.0474 –0.0467 –0.0456 –0.0602 0.4411 0.3221 
 (0.0419) (0.0449) (0.0380) (0.0352) (0.0451) (0.0406) (0.0573) (0.3228) (0.9502) 
  Student–Teacher Ratio –0.0975 –0.0547 –0.0363 –0.0560 –0.2572*** –0.1147 –0.0857 –0.2513 0.6573** 
 (0.0707) (0.0563) (0.0557) (0.0611) (0.0557) (0.0643) (0.0730) (0.2760) (0.1683) 
  Percent Below Basic (Math) –0.0376 –0.0224 –0.0526** –0.0575*** –0.0479*** –0.0322* –0.0001 0.1160 –0.1409 
 (0.0220) (0.0182) (0.0162) (0.0164) (0.0137) (0.0133) (0.0168) (0.0843) (0.4223) 
  Percent Basic (Math) –0.0309 0.0130 –0.0184 –0.0517* –0.0213 –0.0037 –0.0204 –0.1758 0.0218 
 (0.0298) (0.0189) (0.0215) (0.0234) (0.0284) (0.0277) (0.0258) (0.1155) (0.1851) 
  Intercept 17.2437* 2.8397 9.2267* 8.2706** 14.1528** 7.8609 7.7407 –5.7948 –10.1036 
 (8.2219) (4.8563) (4.0021) (2.9791) (5.2625) (4.3879) (4.1517) (11.2772) (32.5989) 
  R2 0.0992 0.0672 0.0455 0.0447 0.0309 0.0273 0.0214 0.0262 0.0351 
  N 17,396 21,683 22,889 23,248 26,094 25,710 24,863 3,965 2,282 

Notes: Models include year fixed effects, NWEA cohort fixed effects, school-level controls for race/ethnicity, special education, free/reduced lunch status, and student-level controls for race/ethnicity and gender. Standard errors reported in parentheses. Standard errors corrected for clustering of students within schools.

*Statistically significant at the 10% level; **statistically significant at the 5% level; ***statistically significant at the 1% level.

Table 8 reports results from the two-step selection correction model. In the first-stage ordered probit model (Appendix  A), the only significant school-level variables were the percentage of minority students, the percentage of students scoring below basic in mathematics (both coefficients positive), and the percentage of students scoring at the basic level in English (negative). The two positive coefficients suggest that schools serving high percentages of at-risk and low-performing students have been more likely to join TAP. The negative coefficient is harder to interpret, given that an increase in the percentage of students at the basic level implies a decrease at both extremes (below basic and proficient). The year of adoption effects (the cs in equation 3) were all significant and monotonically declining over time.

Table 8.
Fall-to-Spring Mathematics Gains, Two-Step Selection Correction Models
Grade
2345678910
Independent Variables          
  TAP 2.4271*** 0.6672 1.7683*** 1.6250*** –0.2196 –0.7441 0.3427 –1.2366 –0.8531 
 (0.7166) (0.7993) (0.4748) (0.4326) (0.5225) (0.5220) (0.5507) (0.6926) (0.5379) 
  Time Elapsed Between Fall and Spring Assessment 0.0522*** 0.0396*** 0.0380*** 0.0319*** 0.0227*** 0.0204*** 0.0105* 0.0089 –0.0032 
 (0.0037) (0.0032) (0.0032) (0.0031) (0.0032) (0.0032) (0.0042) (0.0065) (0.0070) 
  Average Teacher Salary/1000 –0.0424 0.0412* 0.0069 0.0211 0.0257 0.0390* 0.0369 0.0166 0.0500 
 (0.0264) (0.0194) (0.0178) (0.0172) (0.0197) (0.0198) (0.0242) (0.0352) (0.0436) 
  Student–Teacher Ratio –0.0286 –0.0447 –0.0691 –0.0583 –0.1356*** –0.0172 0.0367 –0.0891 –0.0880 
 (0.0577) (0.0353) (0.0418) (0.0347) (0.0383) (0.0372) (0.0433) (0.0683) (0.0780) 
  Percent Below Basic (Math) –0.0357** –0.0290** –0.0349*** –0.0467*** –0.0344*** –0.0312*** –0.0290** –0.0150 –0.0175 
 (0.0123) (0.0102) (0.0086) (0.0091) (0.0090) (0.0078) (0.0100) (0.0148) (0.0206) 
  Percent Basic (Math) –0.0374* –0.0332** –0.0147 –0.0285** –0.0163 –0.0088 –0.0033 –0.0004 0.0030 
 (0.0155) (0.0118) (0.0103) (0.0095) (0.0121) (0.0135) (0.0159) (0.0216) (0.0226) 
  Mill's Ratio 0.1143 0.1130 0.0434 0.1794 0.5446** 0.4295* 0.2468 1.1342*** 2.1418*** 
 (0.2488) (0.2855) (0.1605) (0.1378) (0.1756) (0.1782) (0.2318) (0.2519) (0.4782) 
  Intercept 6.6976** 3.2739* 1.4518 2.5697 5.4148** 2.3335 –0.8394 2.0590 0.5803 
 (2.2316) (1.6638) (1.5491) (1.5255) (1.8907) (1.7141) (2.1396) (2.3265) (2.7970) 
  R2 0.0552 0.0345 0.0342 0.0325 0.0234 0.0187 0.0135 0.0154 0.0153 
  N 90,555 127,474 128,567 135,480 135,781 132,893 129,642 25,468 16,849 
Grade
2345678910
Independent Variables          
  TAP 2.4271*** 0.6672 1.7683*** 1.6250*** –0.2196 –0.7441 0.3427 –1.2366 –0.8531 
 (0.7166) (0.7993) (0.4748) (0.4326) (0.5225) (0.5220) (0.5507) (0.6926) (0.5379) 
  Time Elapsed Between Fall and Spring Assessment 0.0522*** 0.0396*** 0.0380*** 0.0319*** 0.0227*** 0.0204*** 0.0105* 0.0089 –0.0032 
 (0.0037) (0.0032) (0.0032) (0.0031) (0.0032) (0.0032) (0.0042) (0.0065) (0.0070) 
  Average Teacher Salary/1000 –0.0424 0.0412* 0.0069 0.0211 0.0257 0.0390* 0.0369 0.0166 0.0500 
 (0.0264) (0.0194) (0.0178) (0.0172) (0.0197) (0.0198) (0.0242) (0.0352) (0.0436) 
  Student–Teacher Ratio –0.0286 –0.0447 –0.0691 –0.0583 –0.1356*** –0.0172 0.0367 –0.0891 –0.0880 
 (0.0577) (0.0353) (0.0418) (0.0347) (0.0383) (0.0372) (0.0433) (0.0683) (0.0780) 
  Percent Below Basic (Math) –0.0357** –0.0290** –0.0349*** –0.0467*** –0.0344*** –0.0312*** –0.0290** –0.0150 –0.0175 
 (0.0123) (0.0102) (0.0086) (0.0091) (0.0090) (0.0078) (0.0100) (0.0148) (0.0206) 
  Percent Basic (Math) –0.0374* –0.0332** –0.0147 –0.0285** –0.0163 –0.0088 –0.0033 –0.0004 0.0030 
 (0.0155) (0.0118) (0.0103) (0.0095) (0.0121) (0.0135) (0.0159) (0.0216) (0.0226) 
  Mill's Ratio 0.1143 0.1130 0.0434 0.1794 0.5446** 0.4295* 0.2468 1.1342*** 2.1418*** 
 (0.2488) (0.2855) (0.1605) (0.1378) (0.1756) (0.1782) (0.2318) (0.2519) (0.4782) 
  Intercept 6.6976** 3.2739* 1.4518 2.5697 5.4148** 2.3335 –0.8394 2.0590 0.5803 
 (2.2316) (1.6638) (1.5491) (1.5255) (1.8907) (1.7141) (2.1396) (2.3265) (2.7970) 
  R2 0.0552 0.0345 0.0342 0.0325 0.0234 0.0187 0.0135 0.0154 0.0153 
  N 90,555 127,474 128,567 135,480 135,781 132,893 129,642 25,468 16,849 

Notes: Models include year fixed effects, NWEA cohort fixed effects, school-level controls for race/ethnicity, special education, free/reduced lunch status, and student-level controls for race/ethnicity and gender. Standard errors reported in parentheses. Standard errors corrected for clustering of students within schools.

*Statistically significant at the 10% level; **statistically significant at the 5% level; ***statistically significant at the 1% level.

Results in the achievement equation are qualitatively similar to school fixed effect estimates reported in table 6. Estimated TAP effects in the elementary grades are significantly positive in grades 2, 4, and 5. This is not the case in the sixth grade-level model and higher, where, except in eighth grade, coefficients are negative though not statistically significant.24

The coefficients on , the inverse Mill's ratio, confirm the presence of selection bias in the upper grades. In the sixth, seventh, ninth, and tenth grade-level models, these coefficients are positive and statistically significant, indicating a tendency for schools with above average outcomes to adopt TAP. Additionally, there are no grades with a negative TAP selection effect.

Both the fixed effects estimates and the selection-correction estimates rest on the assumption that selection into TAP is a function of time-invariant characteristics of the school. If this is not the case and participation is influenced by transitory changes in test scores, estimated TAP effects could be biased by regression to the mean. We test for this by including an indicator for new TAP schools (schools in their first year in the program). The new-TAP indicator could also pick up implementation problems and first-year bugs in starting up. Results are displayed in table 9.

Table 9.
Fall-to-Spring Mathematics Gain, Including New-TAP Indicator
Grade
ModelIndependent Variables2345678910
OLS Full Sample   TAP 2.8924*** 1.0394** 1.8393*** 2.1740*** 0.7650* 0.1307 0.9830 1.2634 2.4667*** 
  (0.5192) (0.3431) (0.2913) (0.3302) (0.3128) (0.3597) (0.5413) (0.8323) (0.7099) 
   New-TAP –1.0108 –0.4396 0.1236 –0.5326 0.7055 0.7501 –0.3322 –1.3035*** 0.4867 
  (0.8106) (0.8167) (0.5225) (0.4984) (0.5088) (0.5112) (0.7688) (0.2915) (1.5485) 
Fixed Effects   TAP 2.8497*** 0.1783 0.9309* 1.0743* –1.3957*** –2.2073*** 0.6018 –1.8242** –1.1479 
  (0.5994) (0.4999) (0.4426) (0.4446) (0.4107) (0.4556) (0.4926) (0.6515) (0.7456) 
   New-TAP –2.1503*** 0.6183 0.2675 0.0553 1.6129*** 1.6316*** –0.3107 –0.7136 0.4584 
  (0.4962) (0.4281) (0.3526) (0.3578) (0.3597) (0.4287) (0.4320) (0.6852) (0.7309) 
OLS Matched Sample   TAP 1.8440*** 0.5479 2.0092*** 2.0440*** 0.7507* –0.3287 0.1530 –1.6739 –2.6707 
  (0.5030) (0.4843) (0.3194) (0.3465) (0.3488) (0.4190) (0.5367) (0.8716) (1.4988) 
   New-TAP –0.8771 –0.2796 0.0248 –0.6513 0.7281 0.7728 0.0876 –2.1426*** 3.6719** 
  (0.8147) (0.9349) (0.5573) (0.5337) (0.4985) (0.5238) (0.6886) (0.3458) (0.8654) 
2-Step Selection Correction   TAP 2.6822*** 0.7728 1.7183*** 1.7709*** –0.6600 –1.0403 0.3918 –0.8581 –1.0421 
  (0.7610) (0.8630) (0.4663) (0.4681) (0.5321) (0.5492) (0.6816) (0.7093) (0.9189) 
   New-TAP –0.9579 –0.3825 0.1504 –0.4445 1.0821* 1.0706* –0.1730 –1.2416*** 0.5699 
  (0.8110) (0.8422) (0.5112) (0.4876) (0.4982) (0.4902) (0.8242) (0.2870) (1.6171) 
Grade
ModelIndependent Variables2345678910
OLS Full Sample   TAP 2.8924*** 1.0394** 1.8393*** 2.1740*** 0.7650* 0.1307 0.9830 1.2634 2.4667*** 
  (0.5192) (0.3431) (0.2913) (0.3302) (0.3128) (0.3597) (0.5413) (0.8323) (0.7099) 
   New-TAP –1.0108 –0.4396 0.1236 –0.5326 0.7055 0.7501 –0.3322 –1.3035*** 0.4867 
  (0.8106) (0.8167) (0.5225) (0.4984) (0.5088) (0.5112) (0.7688) (0.2915) (1.5485) 
Fixed Effects   TAP 2.8497*** 0.1783 0.9309* 1.0743* –1.3957*** –2.2073*** 0.6018 –1.8242** –1.1479 
  (0.5994) (0.4999) (0.4426) (0.4446) (0.4107) (0.4556) (0.4926) (0.6515) (0.7456) 
   New-TAP –2.1503*** 0.6183 0.2675 0.0553 1.6129*** 1.6316*** –0.3107 –0.7136 0.4584 
  (0.4962) (0.4281) (0.3526) (0.3578) (0.3597) (0.4287) (0.4320) (0.6852) (0.7309) 
OLS Matched Sample   TAP 1.8440*** 0.5479 2.0092*** 2.0440*** 0.7507* –0.3287 0.1530 –1.6739 –2.6707 
  (0.5030) (0.4843) (0.3194) (0.3465) (0.3488) (0.4190) (0.5367) (0.8716) (1.4988) 
   New-TAP –0.8771 –0.2796 0.0248 –0.6513 0.7281 0.7728 0.0876 –2.1426*** 3.6719** 
  (0.8147) (0.9349) (0.5573) (0.5337) (0.4985) (0.5238) (0.6886) (0.3458) (0.8654) 
2-Step Selection Correction   TAP 2.6822*** 0.7728 1.7183*** 1.7709*** –0.6600 –1.0403 0.3918 –0.8581 –1.0421 
  (0.7610) (0.8630) (0.4663) (0.4681) (0.5321) (0.5492) (0.6816) (0.7093) (0.9189) 
   New-TAP –0.9579 –0.3825 0.1504 –0.4445 1.0821* 1.0706* –0.1730 –1.2416*** 0.5699 
  (0.8110) (0.8422) (0.5112) (0.4876) (0.4982) (0.4902) (0.8242) (0.2870) (1.6171) 

Notes: Models include year fixed effects, NWEA cohort fixed effects, school fixed effects, school-level controls for race/ethnicity, special education, free lunch status, and student-level controls for race/ethnicity and gender. Standard errors reported in parentheses.

*Statistically significant at the 10% level; **statistically significant at the 5% level; ***statistically significant at the 1% level.

The new-TAP indicator is statistically significant and negative in the OLS estimates in ninth grade, and insignificant in the other grades. About half the point estimates are positive and half negative. In general, inclusion of new-TAP has very little effect on the coefficients on TAP. The new-TAP coefficients are positive in sixth and seventh grades in the fixed effects model, pushing the coefficients on TAP in those grades still further in the negative direction. In the selection-correction model new-TAP is marginally significant and positive in the sixth and seventh grades, significant and negative in ninth grade.

With the exceptions just noted, including new-TAP generally leaves the earlier results qualitatively unchanged. All sets of estimates continue to indicate TAP has a significant positive effect in the elementary grades. OLS estimates for grades 6–9 are positive but insignificant. The matched-sample and two-step selection correction estimates are almost always insignificant for grades 6–10, with a mix of signs. The fixed effects estimates are, as noted, more strongly negative in grades 6–10 when new-TAP is included in the model. There is no evidence in these results that our conclusions about TAP are caused by a tendency for schools to enter TAP after a particularly good year and experience reversion to the mean afterwards.25

In our fixed effect and selection correction models there are two break points—one between fifth and sixth grades, where point estimates first turn negative, and a second between eighth and ninth grades, where there is a much larger drop. The fact that these shifts occur precisely when students move from elementary to middle school and then from middle to high school raises the possibility that our estimates are confounding two distinct effects: the continuing effect of TAP on students who have been in TAP schools before, and its initial impact on students who are encountering it for the first time as they enter a new school. Thus, TAP might succeed when students have grown used to it from the beginning of their schooling, but not with students at more advanced stages of their education who are accustomed to the (conceivably) more lax environments of non-TAP schools.

To explore this hypothesis, we have re-estimated all of the earlier models after replacing the binary TAP indicator with three new variables: cumulative TAP (equal to the number of years a student has spent in a TAP school); first-year TAP (a binary indicator equal to one during the first year a student is enrolled in a TAP school); and after-TAP (the number of years since a student left his most recent TAP school).26 Results are presented in Appendices B through E. In the fixed effects model, cumulative TAP has no systematic effect in the elementary grades. Point estimates are sometimes negative, sometimes positive, but never significant. In grades 6 and higher, the effect is typically negative and statistically significant. In the selection correction model coefficients on cumulative TAP tend to be positive in elementary grades, negative in higher grades, though rarely significant. By contrast, OLS estimates of cumulative TAP are positive and significant in the elementary grades. Thus, once we control for selection into TAP, it appears that the positive effects evident in the elementary grades are produced almost entirely during the first year a student attends a TAP school.

First-year TAP effects are not significantly negative except for grade 9 in the selection correction model. With this one exception, there is no evidence that students unused to TAP are bringing down the estimated effect.

7.  Conclusion

This study has presented findings from a study of the impact of TAP on student test score gains in mathematics. We have used a panel data set to estimate a TAP treatment effect by comparing student test score gains in schools that participated in TAP with student test score gains in non-TAP schools. OLS estimation revealed a significant, positive TAP treatment effect on student test score gains in grades 2–6 and 10. Point estimates were still positive in grades 7–9, though no estimates were statistically different from zero.

We use three different approaches to control for selection into TAP: a school fixed-effects estimator, a matched-sample estimator, and selection correction model. Regardless of the approach used, the estimates on TAP effects remain positive in the elementary grades, but are generally insignificant after grade 5. Point estimates in the upper grades are more often negative than positive. In the fixed effects estimator, these negative estimates are significant in seventh and ninth grades. The tests that furnished the data for this study are not the high-stakes exams on which teacher bonuses are based, but this does not account for the difference between our findings and those of earlier investigators, inasmuch as we reproduce their results when estimating an achievement equation using OLS regression techniques. It is only when we control for selection into TAP that we obtain markedly different findings in the higher grades.27

Given the small number of schools in this study, the failure of TAP to produce positive outcomes at the middle and high school levels may have been due to idiosyncratic failures in program implementation in these schools. The explanation, however, is not simply that the secondary schools in this study were not as effective as the elementary schools (as a result, say, of poor leadership). We have re-estimated the model by OLS including an indicator for pre-TAP status (taking the value 1 in the years before a school joined the program and zero in all other instances). The sign on the coefficient on pre-TAP is generally positive but insignificant in the elementary grades (Appendices F and G). It is positive in all of the higher grades, significantly so in all but one, and larger in magnitude than in the lower grades. (This is so using both the full sample and the matched sample.) These estimates suggest TAP middle and high schools were outperforming their non-TAP comparison schools before they implemented the TAP program.

It is possible that the way teachers and schools respond to TAP does not work as well in the middle and high school grades. Exhorting students to try harder on tests, for example, may succeed with younger children who are eager to please their teachers, but not with adolescents. It is also possible that TAP incentives work best in schools where most teachers are doing essentially the same job, but that differences in the way instructors of core subjects are treated from other teachers produces acrimony and a breakdown in teamwork at the secondary school level. We do not know that either of these explanations is correct, and offer them only as illustrative of differences between elementary and secondary schools that could account for our findings.

Although the focus of this paper has been on mathematics achievement, TAP is a school-wide incentive program that includes all teachers. It is therefore of some interest whether our findings for mathematics characterize other subjects. We have repeated all the analyses described here for test results in reading and language arts. We now briefly describe these findings.

As with mathematics, OLS results are positive at every grade level. Unlike mathematics, these results are frequently statistically significant even in the higher grades (7 and 8 in reading, 6–8 and 10 in language arts). These positive findings virtually disappear when school fixed effects are included in the model, however. Coefficients drop by 30–50 percent in most grades in language arts. In reading they frequently fall to zero or become negative. Only one positive significant result remains (in eighth grade). Results for the selection correction estimates are not quite this dramatic. Point estimates using the selection correction estimator remain positive, but are significant only in fourth- and fifth-grade reading and fourth- and tenth-grade language arts. As with mathematics, these results do not appear to be explained by model misspecification. Including new-TAP does not fundamentally alter the pattern of findings. Nor do we obtain negative effects on students in their first year of exposure to TAP.

It is important to acknowledge several limitations of this study. The sample of TAP schools is small. The numbers of student test score observations in second, ninth, and tenth grades are far fewer than other grades in our data panel. We also lack information on the fidelity of implementation and on variation in features of TAP programs at the school level (e.g., minimum and maximum bonus sizes, percent of teachers voting in favor of TAP adoption, and so forth). Furthermore, it is unclear whether our sample is representative of other schools and locations across the United States that will be or are actually implementing TAP.

Finally, we have investigated only one aspect of the TAP reform model—the impact of TAP on student achievement. TAP is a comprehensive school reform model designed to attract highly effective teachers, improve instructional effectiveness, and elevate student achievement. Although student achievement is ultimately the outcome of interest, we have not determined whether TAP has altered teacher recruitment and retention or instructional practices. Clearly, these are important components of TAP and important areas for future research.

Notes

1. 

See Podgursky and Springer (2007, 2010) for a comprehensive review of teacher performance pay.

2. 

In May 2006, it was announced that the Teacher Advancement Program Foundation developed into the National Institute for Excellence in Teaching (NIET) to further its mission of improving teacher quality.

3. 

Thirty-five schools adopted TAP in these two states over this period. We do not have student-level test score information on one TAP school and exclude two schools that abandoned the program shortly after adoption.

4. 

TAP typically requires teachers to vote on whether to adopt the program at their school. In some instances, however, TAP is implemented without a vote.

5. 

Both the MFF and the NIET have produced a large number of resources about the TAP reform model. More details can be found at www.talented.teachers.org.

6. 

In select instances, the reference group is not the average teacher value-added effect estimate in the state.

7. 

Levels are defined as follows: level 1: two standard errors or more below the state average; level 2: one standard error below the mean; level 3: between one standard error below and one standard error above; level 4: one standard error above the mean; level 5: two standard errors above the mean.

8. 

In select instances, the reference group is not the average school value-added effect estimate in the state.

9. 

Eckert (2010) presents an analysis of six sites that are implementing teacher and principal compensation reforms under the Teacher Incentive Fund, four of which are implementing the TAP.

10. 

The TAP school sample included a total of 1,114 student observations during the 2000–01 school year and 1,277 observations during the 2001–02 school year. The comparison group sample included a total of 2,009 students during the 2000–01 school year and 1,372 students during the 2001–02 school year. The authors do not explain the 31.7 percent reduction in comparison group students, even though the number of comparison group schools remained constant across years. Furthermore, the matching strategy was contingent upon districts supplying MFF with student level data from matched comparison schools. If a school district was unwilling to provide data to MFF, MFF moved to the next best matched school until necessary data for comparison group schools were obtained.

11. 

All TAP schools and schools in the matched comparison group were below the 85th percentile.

12. 

A different approach was used in South Carolina because individual student achievement results were reported as performance levels (i.e., Below Basic, Basic, Proficient, and Advanced).

13. 

This study was based on a larger number of observations than previous studies reported by the MFF. Their sample included a total of 2,947 teachers and 346 schools from six states. TAP teachers and schools constituted roughly 21 percent and 18 percent of the sample, respectively.

14. 

Solmon et al. (2007) also analyzed adequate yearly progress results for the 2004–05 and 2005–06 school years in TAP schools as compared to the statewide average. In terms of adequate yearly progress comparison, the authors found that, “In most cases an equal or higher percentage of TAP schools in the six states make Adequate Yearly Progress than all schools in their states, despite TAP schools having more students receiving free or reduced-price lunch” (p. 7).

15. 

Quasi-experimental design estimates (nearest-neighbor propensity score methods) indicated that the program had a statistically significant positive effect on increasing teacher retention at TAP schools during the first year.

16. 

The model omits a time-invariant student effect. Because we estimate separate equations for each grade level, only a few students (those retained in a grade) appear more than once in any estimation sample.

17. 

It is also possible that the reverse occurs, and that schools are more likely to join TAP after a bad year, when under pressure from district or state officials to improve. As already noted, a model that allows for transitory changes in test results (or anything else) to influence TAP participation must explain why so few participation decisions are reversed. However, there may be some inertia among participating schools that discourages teachers from revisiting a decision: reluctance to admit a previous error, aversion to reopening a contentious debate, pressure from higher up, and so forth.

18. 

Despite cov(ujt, ej) = 0, it is still possible for ujt to be correlated with Tjt, if students and their parents use the fact that a school participates in TAP as a signal of quality. We consider this quite unlikely, given how little the general public knows about TAP. Indeed, in the school of education where we work, there are many persons who do not know what TAP is.

19. 

To obtain the matched sample, we estimated the probability that a school adopted TAP as a function of the four 2002 school-level variables indicated, then used the resulting propensity scores to match TAP schools to their 10 non-TAP nearest neighbors. Because this adoption model differs from the behavioral model in equation 3, this procedure should be regarded merely as a device to obtain a non-TAP sample that resembled TAP schools in 2001–2002.

20. 

The first year for which we have achievement data is the 2002–03 school year, the final school year in our sample is 2006–07. We know which schools have adopted TAP through the 2007–08 school year, however. Under the assumptions above, all of these adoption decisions contain information helping to identify . Thus the year-of-adoption index, s, ranges from 1 to 6.

21. 

The argument is more complicated than for the fixed effects model in that the quality of the new principals would have to vary by year of adoption. This is not altogether implausible. If effectiveness is an increasing function of how long they have been on the job, new principals hired in 2002–03 will, over the sample period, be more effective on average than those hired the following year, and so forth. Clearly other stories could also be told wherein year of adoption coincides with unobserved developments that affect achievement.

22. 

The National Center for Education Statistics had not released the Common Core of Data for the 2006–07 school year at the time analyses were completed. We therefore imputed values at the school-level for the following variables for the 2006–07 school year in both State A and State B: average teacher salary, student–teacher ratio, percent Asian, percent Hispanic, percent black, and percent free or reduced-price lunch eligible.

23. 

As shown in table 2, there were ten TAP schools in these states in 2002–03, but for four of these schools our data start in a later year.

24. 

Introducing an estimate of the inverse Mill's ratio into the model makes the error term heteroskedastic (if it were not so already). Rather than assume the errors were initially homoskedastic and that the two-step selection correction is the only source of heteroskedasticity, we use estimates of standard errors that are robust to heteroskedasticity of an unknown form and clustering of students within schools.

25. 

The fact that coefficients on new-TAP are either insignificant or positive also suggests that there has been little learning-by-doing associated with TAP. Results tend to be as good or better in the first year of TAP implementation as later. This conclusion is further buttressed by our analysis of students’ cumulative exposure to TAP (see below). The first year in TAP seems to be the important one for a student. Cumulative exposure, given the first-year effect, is not generally found to be beneficial in models that control for self-selection of TAP participants.

26. 

Because cumulative TAP is frozen at previous levels for students who have left TAP schools, we include after-TAP to pick up any decay in the TAP effect that occurs when students are no longer exposed to the program. Although the coefficients on after-TAP suggest there is some fall-off, it should be borne in mind that these estimates are based on very small numbers of observations.

27. 

Our estimates remained qualitatively similar when using fall test score as an independent variable. We also estimated all models with one TAP school dropped from the regression data set to check whether an outlier school or grade was influencing our estimates. This was done thirty-two times, once for each TAP school. All estimates remained qualitatively similar to those reported.

Acknowledgments

This working paper was supported by the National Center on Performance Incentives, which is funded by the United States Department of Education's Institute of Education Sciences (R305A06034). We appreciate helpful comments and suggestions from Torberg Falch, David Figlio, Steve Glazerman, Carolyn Hoxby, F. Howard Nelson, Mike Podgursky, Tamara Schiff, Gary Stark, and participants at the Twenty-Eighth Annual Meeting of the Association for Public Policy Analysis and Management, a Federal Reserve Bank of New York research seminar, the National Center on Performance Incentives’ 2008 Research to Policy forum, and the CESifo/PEPG forum on economic incentives. The authors also wish to acknowledge Kelly Fork, Rebekah Hutton, and Kurt Scheib for their research assistance and the Northwest Evaluation Association for providing data and technical support to conduct our analyses. Any errors remain the sole responsibility of the authors. The views expressed in this paper do not necessarily reflect those of sponsoring agencies or individuals acknowledged.

REFERENCES

Eckert
,
Jonathan
.
2010
.
Performance-based compensation: Design and implementation at six Teacher Incentive Fund sites
.
Seattle, WA
:
Bill & Melinda Gates Foundation
.
Glazerman
,
Steven
,
Tim
Silva
,
Nii
Addy
,
Sarah
Avellar
,
Jeffrey
Max
,
Allison
McKie
,
Brenda
Natzke
,
Michael
Puma
,
Patrick
Wolf
, and
Rachel Ungerer
Greszler
.
2006
.
Options for studying teacher pay reform using natural experiments
.
Washington, DC
:
Mathematica Policy Research Report No. 6212–200
.
Glazerman
,
Steven
,
Allison
McKie
,
Nancy
Carey
, and
Dominic
Harris
.
2009
.
Evaluation of the Teacher Advancement Program (TAP) in the Chicago Public Schools: Study design report
.
Washington, DC
:
Mathematica Policy Research Report No. 6319–520
.
Goldhaber
,
Dan
.
2002
.
The mystery of good teachers
.
Education Next
(
Spring
):
1
7
.
Hanushek
,
Eric A.
2003
.
The failure of input-based resource policies
.
Economic Journal
113
(
485
):
F64
F68
. doi: 10.1111/1468-0297.00099
Heckman
,
James
.
1979
.
Sample selection bias as a specification error
.
Econometrica
47
(
1
):
153
161
. doi: 10.2307/1912352
Kingsbury
,
Gage G.
2003
.
A long-term study of the stability of item parameter estimates. Paper presented at the annual meeting of the American Educational Research Association
,
Chicago, IL, April
.
Milken Family Foundation (MFF)
.
2004
.
Teacher Advancement Program: Teacher evaluation and performance award guide. Unpublished paper, MFF
,
Santa Monica, CA
.
National Institute for Excellence in Teaching (NIET)
.
2007
.
More than 60 schools join Teacher Advancement Program: TAP enjoys greatest growth yet, gaining ground in urban centers like Philly, Chicago
.
Available
www.tapsystem.org/newsroom/newsroom.taf?page=features&_function=detail&id=30.
Accessed 2 April 2013
.
National Institute for Excellence in Teaching (NIET)
.
2010
.
Resource for current TAP schools
.
Available
www.talented.teachers.org.
Accessed 31 January 2013
.
Podgursky
,
Michael
, and
Matthew G.
Springer
.
2007
.
Teacher performance pay: A review
.
Journal of Policy Analysis and Management
26
(
4
):
909
949
. doi: 10.1002/pam.20292
Podgursky
,
Michael
, and
Matthew G.
Springer
.
2010
.
Teacher compensation systems in the United States K-12 public school system
.
National Tax Journal
64
(
1
):
165
192
.
Rotherham
,
Andrew J.
2010
.
Achieving teacher and principal excellence: A guidebook for donors
.
Washington, DC
:
The Philanthropy Roundtable
.
Schacter
,
John
,
Tammara
Schiff
,
Yeow Meng
Thum
,
Cheryl
Fagnano
,
Micheline
Bendotti
,
Lewis
Solmon
,
Kimberly
Firetag
, and
Lowell
Milken
.
2002
.
The impact of the Teacher Advancement Program on student achievement, teacher attitudes, and job satisfaction. Unpublished paper, Milken Family Foundation, Santa Monica, CA
.
Schacter
,
John
,
Yeow Meng
Thum
,
Daren
Reifsneider
, and
Tamara
Schiff
.
2004
.
The Teacher Advancement Program report two: Year three results from Arizona and year one results from South Carolina TAP schools. Unpublished paper, Milken Family Foundation
,
Santa Monica, CA
.
Solmon
,
Lewis C.
,
Todd
White
,
Donna
Cohen
, and
Deborah
Woo
.
2007
.
The effectiveness of the Teacher Advancement Program
.
Santa Monica, CA
:
NIET
.
Wright
,
S. Paul
,
John T.
White
,
William L.
Sanders
, and
June C.
Rivers
.
2010
.
SAS EVAAS statistical models
.
Cary, NC
:
SAS Institute, Inc
.

APPENDIX  A

Table A.1.
Ordered Probit Estimates of TAP Adoption
Independent Variables
  Percent Below Basic (Math) 0.0163* 
 (0.0082) 
  Percent Basic (Math) 0.0069 
 (0.0108) 
  Percent Below Basic (Reading) –0.0057 
 (0.0124) 
  Percent Basic (Reading) –0.0389* 
 (0.0152) 
  Student–Teacher Ratio –0.0141 
 (0.0249) 
  Average Teacher Salary/1000 –0.0030 
 (0.0189) 
  Percent Minority 0.0211*** 
 (0.0054) 
  Percent Free/Reduced Lunch –0.0084 
 (0.0066) 
  State Dummy 0.6499* 
 (0.3165) 
Threshold Variables  
  Cutoff1 1.6267* 
 (0.8142) 
  Cutoff2 1.8485* 
 (0.8152) 
  Cutoff3 1.9131* 
 (0.8153) 
  Cutoff4 1.9834* 
 (0.8153) 
  Cutoff5 2.1542** 
 (0.8152) 
  Cutoff6 2.3025** 
 (0.8155) 
  N 1,082 
Independent Variables
  Percent Below Basic (Math) 0.0163* 
 (0.0082) 
  Percent Basic (Math) 0.0069 
 (0.0108) 
  Percent Below Basic (Reading) –0.0057 
 (0.0124) 
  Percent Basic (Reading) –0.0389* 
 (0.0152) 
  Student–Teacher Ratio –0.0141 
 (0.0249) 
  Average Teacher Salary/1000 –0.0030 
 (0.0189) 
  Percent Minority 0.0211*** 
 (0.0054) 
  Percent Free/Reduced Lunch –0.0084 
 (0.0066) 
  State Dummy 0.6499* 
 (0.3165) 
Threshold Variables  
  Cutoff1 1.6267* 
 (0.8142) 
  Cutoff2 1.8485* 
 (0.8152) 
  Cutoff3 1.9131* 
 (0.8153) 
  Cutoff4 1.9834* 
 (0.8153) 
  Cutoff5 2.1542** 
 (0.8152) 
  Cutoff6 2.3025** 
 (0.8155) 
  N 1,082 

Note: Standard errors reported in parentheses.

*Statistically significant at the 10% level; **statistically significant at the 5% level; ***statistically significant at the 1% level.

APPENDIX  B

Table B.1.
OLS Including First-, Cumulative-, and After-TAP Indicators, Full Sample
Grade
Dependent Variables2345678910
Independent Variables          
  First-TAP 3.1857*** 0.3804 1.5743*** 1.7209*** 0.6934* 0.2659 0.6749 0.2156 2.7982 
 (0.7283) (0.4923) (0.4337) (0.3594) (0.2931) (0.3203) (0.6359) (0.3769) (1.5337) 
  Cumulative-TAP –0.4420 0.5043** 0.6012*** 0.5928*** 0.2472 0.1145 0.3311 0.3928 0.6695** 
 (0.5862) (0.1724) (0.1406) (0.1769) (0.1539) (0.1538) (0.2144) (0.2768) (0.2214) 
  After-TAP –11.7457* –3.0896* –1.8565** –1.1339* –0.9952* –0.3909 –0.5079 0.8201 –11.0000 
 (4.6857) (1.4269) (0.5803) (0.4601) (0.3890) (0.3226) (0.3330) (0.6706) (5.7231) 
  Time Elapsed Between Fall and Spring Assessment 0.0522*** 0.0396*** 0.0379*** 0.0318*** 0.0228*** 0.0205*** 0.0106* 0.0083 –0.0048 
 (0.0037) (0.0032) (0.0032) (0.0031) (0.0032) (0.0032) (0.0042) (0.0066) (0.0076) 
  Average Teacher Salary/100 –0.0427 0.0412* 0.0073 0.0213 0.0272 0.0394* 0.0371 0.0189 0.0597 
 (0.0263) (0.0193) (0.0178) (0.0172) (0.0198) (0.0198) (0.0243) (0.0353) (0.0455) 
  Student–Teacher Ratio –0.0284 –0.0446 –0.0700 –0.0593 –0.1425*** –0.0218 0.0335 –0.1114 –0.1227 
 (0.0577) (0.0353) (0.0419) (0.0348) (0.0385) (0.0373) (0.0432) (0.0688) (0.0828) 
  Percent Below Basic (Math) –0.0356** –0.0289** –0.0352*** –0.0465*** –0.0359*** –0.0330*** –0.0301** –0.0168 –0.0219 
 (0.0123) (0.0101) (0.0085) (0.0091) (0.0090) (0.0078) (0.0099) (0.0150) (0.0203) 
  Percent Basic (Math) –0.0373* –0.0332** –0.0144 –0.0283** –0.0154 –0.0076 –0.0029 0.0010 –0.0051 
 (0.0155) (0.0118) (0.0103) (0.0096) (0.0124) (0.0137) (0.0159) (0.0229) (0.0237) 
  Intercept 6.7125** 3.2667 1.4196 2.5270 5.7084** 2.6709 –0.7109 4.4464 3.6466 
 (2.2382) (1.6643) (1.5474) (1.5238) (1.9589) (1.7821) (2.1682) (2.7145) (3.4817) 
  R2 0.0552 0.0345 0.0343 0.0323 0.0231 0.0185 0.0135 0.0142 0.0126 
  N 90,555 127,474 128,567 135,480 135,781 132,893 129,642 25,468 16,849 
Grade
Dependent Variables2345678910
Independent Variables          
  First-TAP 3.1857*** 0.3804 1.5743*** 1.7209*** 0.6934* 0.2659 0.6749 0.2156 2.7982 
 (0.7283) (0.4923) (0.4337) (0.3594) (0.2931) (0.3203) (0.6359) (0.3769) (1.5337) 
  Cumulative-TAP –0.4420 0.5043** 0.6012*** 0.5928*** 0.2472 0.1145 0.3311 0.3928 0.6695** 
 (0.5862) (0.1724) (0.1406) (0.1769) (0.1539) (0.1538) (0.2144) (0.2768) (0.2214) 
  After-TAP –11.7457* –3.0896* –1.8565** –1.1339* –0.9952* –0.3909 –0.5079 0.8201 –11.0000 
 (4.6857) (1.4269) (0.5803) (0.4601) (0.3890) (0.3226) (0.3330) (0.6706) (5.7231) 
  Time Elapsed Between Fall and Spring Assessment 0.0522*** 0.0396*** 0.0379*** 0.0318*** 0.0228*** 0.0205*** 0.0106* 0.0083 –0.0048 
 (0.0037) (0.0032) (0.0032) (0.0031) (0.0032) (0.0032) (0.0042) (0.0066) (0.0076) 
  Average Teacher Salary/100 –0.0427 0.0412* 0.0073 0.0213 0.0272 0.0394* 0.0371 0.0189 0.0597 
 (0.0263) (0.0193) (0.0178) (0.0172) (0.0198) (0.0198) (0.0243) (0.0353) (0.0455) 
  Student–Teacher Ratio –0.0284 –0.0446 –0.0700 –0.0593 –0.1425*** –0.0218 0.0335 –0.1114 –0.1227 
 (0.0577) (0.0353) (0.0419) (0.0348) (0.0385) (0.0373) (0.0432) (0.0688) (0.0828) 
  Percent Below Basic (Math) –0.0356** –0.0289** –0.0352*** –0.0465*** –0.0359*** –0.0330*** –0.0301** –0.0168 –0.0219 
 (0.0123) (0.0101) (0.0085) (0.0091) (0.0090) (0.0078) (0.0099) (0.0150) (0.0203) 
  Percent Basic (Math) –0.0373* –0.0332** –0.0144 –0.0283** –0.0154 –0.0076 –0.0029 0.0010 –0.0051 
 (0.0155) (0.0118) (0.0103) (0.0096) (0.0124) (0.0137) (0.0159) (0.0229) (0.0237) 
  Intercept 6.7125** 3.2667 1.4196 2.5270 5.7084** 2.6709 –0.7109 4.4464 3.6466 
 (2.2382) (1.6643) (1.5474) (1.5238) (1.9589) (1.7821) (2.1682) (2.7145) (3.4817) 
  R2 0.0552 0.0345 0.0343 0.0323 0.0231 0.0185 0.0135 0.0142 0.0126 
  N 90,555 127,474 128,567 135,480 135,781 132,893 129,642 25,468 16,849 

Notes: Models include year fixed effects, NWEA cohort fixed effects, school-level controls for race/ethnicity, special education, free/reduced lunch status, and student-level controls for race/ethnicity and gender. Standard errors reported in parentheses. Standard errors corrected for clustering of students within schools.

*Statistically significant at the 10% level; **statistically significant at the 5% level; ***statistically significant at the 1% level.

APPENDIX  C

Table C.1.
Fixed Effects Including First-, Cumulative-, and After-TAP Indicators, Full Sample
Grade
Dependent Variables2345678910
Independent Variables          
  First-TAP 2.3083* 0.8807** 0.9972*** 0.9287*** 0.4028 –0.3135 0.7033* –0.5534 0.2333 
 (0.9212) (0.3058) (0.2967) (0.2804) (0.2724) (0.2711) (0.2960) (0.6849) (0.6902) 
  Cumulative-TAP –0.7230 0.0376 0.3105 0.0591 –0.6179*** –0.5530*** –0.1040 –0.3963* –0.4523* 
 (0.9143) (0.2344) (0.1618) (0.1339) (0.1350) (0.1461) (0.1501) (0.1965) (0.2230) 
  After-TAP –10.0480 –3.4183** –2.0470** –0.5086 0.3857 0.1444 –0.3250 2.5559* –8.6326 
 (5.2736) (1.1260) (0.6282) (0.4812) (0.3640) (0.2803) (0.2495) (1.1382) (6.5347) 
  Time Elapsed Between Fall and Spring Assessment 0.0530*** 0.0480*** 0.0357*** 0.0328*** 0.0236*** 0.0202*** 0.0114*** 0.0127*** –0.0127* 
 (0.0017) (0.0015) (0.0015) (0.0015) (0.0016) (0.0016) (0.0017) (0.0038) (0.0053) 
  Average Teacher Salary/100 –0.0307 0.0664* 0.0022 –0.0117 0.0073 0.0760* –0.0011 0.1469 0.0417 
 (0.0381) (0.0265) (0.0272) (0.0261) (0.0339) (0.0347) (0.0359) (0.0886) (0.1148) 
  Student–Teacher Ratio –0.0288 –0.0823* –0.0880* –0.1197*** –0.1040** –0.1082** –0.1500*** –0.2296** –0.1211 
 (0.0511) (0.0364) (0.0359) (0.0348) (0.0373) (0.0416) (0.0429) (0.0850) (0.1009) 
  Intercept 2.5931 –7.0811*** 0.3287 0.4107 2.6542 1.8083 –3.4376 –5.7446 3.9864 
 (3.1228) (1.7831) (1.8356) (1.7729) (2.1429) (2.1733) (2.2497) (5.1244) (6.9058) 
  R2 0.0241 0.0165 0.0103 0.0082 0.0048 0.0041 0.0029 0.0085 0.0058 
  N 90,555 127,474 128,567 135,480 135,781 132,893 129,642 25,468 16,849 
Grade
Dependent Variables2345678910
Independent Variables          
  First-TAP 2.3083* 0.8807** 0.9972*** 0.9287*** 0.4028 –0.3135 0.7033* –0.5534 0.2333 
 (0.9212) (0.3058) (0.2967) (0.2804) (0.2724) (0.2711) (0.2960) (0.6849) (0.6902) 
  Cumulative-TAP –0.7230 0.0376 0.3105 0.0591 –0.6179*** –0.5530*** –0.1040 –0.3963* –0.4523* 
 (0.9143) (0.2344) (0.1618) (0.1339) (0.1350) (0.1461) (0.1501) (0.1965) (0.2230) 
  After-TAP –10.0480 –3.4183** –2.0470** –0.5086 0.3857 0.1444 –0.3250 2.5559* –8.6326 
 (5.2736) (1.1260) (0.6282) (0.4812) (0.3640) (0.2803) (0.2495) (1.1382) (6.5347) 
  Time Elapsed Between Fall and Spring Assessment 0.0530*** 0.0480*** 0.0357*** 0.0328*** 0.0236*** 0.0202*** 0.0114*** 0.0127*** –0.0127* 
 (0.0017) (0.0015) (0.0015) (0.0015) (0.0016) (0.0016) (0.0017) (0.0038) (0.0053) 
  Average Teacher Salary/100 –0.0307 0.0664* 0.0022 –0.0117 0.0073 0.0760* –0.0011 0.1469 0.0417 
 (0.0381) (0.0265) (0.0272) (0.0261) (0.0339) (0.0347) (0.0359) (0.0886) (0.1148) 
  Student–Teacher Ratio –0.0288 –0.0823* –0.0880* –0.1197*** –0.1040** –0.1082** –0.1500*** –0.2296** –0.1211 
 (0.0511) (0.0364) (0.0359) (0.0348) (0.0373) (0.0416) (0.0429) (0.0850) (0.1009) 
  Intercept 2.5931 –7.0811*** 0.3287 0.4107 2.6542 1.8083 –3.4376 –5.7446 3.9864 
 (3.1228) (1.7831) (1.8356) (1.7729) (2.1429) (2.1733) (2.2497) (5.1244) (6.9058) 
  R2 0.0241 0.0165 0.0103 0.0082 0.0048 0.0041 0.0029 0.0085 0.0058 
  N 90,555 127,474 128,567 135,480 135,781 132,893 129,642 25,468 16,849 

Notes: Models include year fixed effects, NWEA cohort fixed effects, school-level controls for race/ethnicity, special education, free/reduced lunch status, and student-level controls for race/ethnicity and gender. Standard errors reported in parentheses. Standard errors corrected for clustering of students within schools.

*Statistically significant at the 10% level; **statistically significant at the 5% level; ***statistically significant at the 1% level.

APPENDIX  D

Table D.1.
OLS Including First-, Cumulative-, and After-TAP Indicators, Matched Sample
Grade
Dependent Variables2345678910
Independent Variables          
  First-TAP 2.4662*** 0.5735 1.7140*** 1.5807*** 0.6557* 0.0760 0.5220 –1.0273* 0.5674 
 (0.6365) (0.4139) (0.4750) (0.4109) (0.3079) (0.3157) (0.5919) (0.3578) (0.6207) 
  Cumulative-TAP –0.7667 0.1179 0.6184*** 0.5614** 0.2874 –0.0521 –0.0471 –0.3734 –1.0507* 
 (0.6114) (0.2688) (0.1535) (0.1834) (0.2217) (0.1672) (0.2223) (0.3052) (0.4062) 
  After-TAP –16.9505*** –3.8862 –1.8761** –1.5280* –0.7151 0.1373 –0.1226 0.1322 
 (0.6681) (2.1340) (0.6777) (0.7456) (0.6509) (0.4242) (0.6161) (0.4393) 
  Time Elapsed Between Fall and Spring Assessment 0.0529*** 0.0592*** 0.0387*** 0.0389*** 0.0219*** 0.0316*** 0.0216** 0.0054 –0.0019 
 (0.0074) (0.0065) (0.0052) (0.0052) (0.0062) (0.0063) (0.0074) (0.0159) (0.0269) 
  Average Teacher Salary/100 –0.0726 0.0050 –0.0341 –0.0482 –0.0447 –0.0478 –0.0578 0.1932 0.4273 
 (0.0419) (0.0448) (0.0378) (0.0352) (0.0449) (0.0410) (0.0573) (0.3949) (0.4550) 
  Student–Teacher Ratio –0.0944 –0.0530 –0.0373 –0.0546 –0.2607*** –0.1119 –0.0882 –0.2157 0.5530*** 
 (0.0707) (0.0565) (0.0558) (0.0609) (0.0561) (0.0646) (0.0735) (0.1969) (0.0874) 
  Percent Below Basic (Math) –0.0375 –0.0224 –0.0541*** –0.0572*** –0.0478*** –0.0323* –0.0003 0.1036 –0.1144 
 (0.0220) (0.0182) (0.0161) (0.0164) (0.0138) (0.0134) (0.0170) (0.0828) (0.2945) 
  Percent Basic (Math) –0.0312 0.0135 –0.0178 –0.0524* –0.0217 –0.0024 –0.0199 –0.1044 0.0158 
 (0.0298) (0.0190) (0.0215) (0.0234) (0.0289) (0.0277) (0.0259) (0.1011) (0.1743) 
  Intercept 17.2773* 2.8329 9.0290* 8.1328** 14.1839** 7.9240 7.5843 1.5073 –13.1688 
 (8.2298) (4.8595) (3.9739) (2.9706) (5.3056) (4.4001) (4.1480) (13.3828) (17.4356) 
  R2 0.0996 0.0673 0.0459 0.0447 0.0307 0.0273 0.0216 0.0251 0.0394 
  N 17,396 21,683 22,889 23,248 26,094 25,710 24,863 3,965 2,282 
Grade
Dependent Variables2345678910
Independent Variables          
  First-TAP 2.4662*** 0.5735 1.7140*** 1.5807*** 0.6557* 0.0760 0.5220 –1.0273* 0.5674 
 (0.6365) (0.4139) (0.4750) (0.4109) (0.3079) (0.3157) (0.5919) (0.3578) (0.6207) 
  Cumulative-TAP –0.7667 0.1179 0.6184*** 0.5614** 0.2874 –0.0521 –0.0471 –0.3734 –1.0507* 
 (0.6114) (0.2688) (0.1535) (0.1834) (0.2217) (0.1672) (0.2223) (0.3052) (0.4062) 
  After-TAP –16.9505*** –3.8862 –1.8761** –1.5280* –0.7151 0.1373 –0.1226 0.1322 
 (0.6681) (2.1340) (0.6777) (0.7456) (0.6509) (0.4242) (0.6161) (0.4393) 
  Time Elapsed Between Fall and Spring Assessment 0.0529*** 0.0592*** 0.0387*** 0.0389*** 0.0219*** 0.0316*** 0.0216** 0.0054 –0.0019 
 (0.0074) (0.0065) (0.0052) (0.0052) (0.0062) (0.0063) (0.0074) (0.0159) (0.0269) 
  Average Teacher Salary/100 –0.0726 0.0050 –0.0341 –0.0482 –0.0447 –0.0478 –0.0578 0.1932 0.4273 
 (0.0419) (0.0448) (0.0378) (0.0352) (0.0449) (0.0410) (0.0573) (0.3949) (0.4550) 
  Student–Teacher Ratio –0.0944 –0.0530 –0.0373 –0.0546 –0.2607*** –0.1119 –0.0882 –0.2157 0.5530*** 
 (0.0707) (0.0565) (0.0558) (0.0609) (0.0561) (0.0646) (0.0735) (0.1969) (0.0874) 
  Percent Below Basic (Math) –0.0375 –0.0224 –0.0541*** –0.0572*** –0.0478*** –0.0323* –0.0003 0.1036 –0.1144 
 (0.0220) (0.0182) (0.0161) (0.0164) (0.0138) (0.0134) (0.0170) (0.0828) (0.2945) 
  Percent Basic (Math) –0.0312 0.0135 –0.0178 –0.0524* –0.0217 –0.0024 –0.0199 –0.1044 0.0158 
 (0.0298) (0.0190) (0.0215) (0.0234) (0.0289) (0.0277) (0.0259) (0.1011) (0.1743) 
  Intercept 17.2773* 2.8329 9.0290* 8.1328** 14.1839** 7.9240 7.5843 1.5073 –13.1688 
 (8.2298) (4.8595) (3.9739) (2.9706) (5.3056) (4.4001) (4.1480) (13.3828) (17.4356) 
  R2 0.0996 0.0673 0.0459 0.0447 0.0307 0.0273 0.0216 0.0251 0.0394 
  N 17,396 21,683 22,889 23,248 26,094 25,710 24,863 3,965 2,282 

Notes: Models include year fixed effects, NWEA cohort fixed effects, school-level controls for race/ethnicity, special education, free lunch status, and student-level controls for race/ethnicity and gender. Standard errors reported in parentheses. Standard errors corrected for clustering of students within schools.

*Statistically significant at the 10% level; **statistically significant at the 5% level; ***statistically significant at the 1% level.

APPENDIX  E

Table E.1.
Ordered Probit Selection Correction Models Including First-, Cumulative-, and After-TAP Indicators, Full Sample
Grade
Dependent Variables2345678910
Independent Variables          
  First-TAP 3.0403*** 0.2743 1.4908** 1.3651*** –0.1465 –0.1512 0.3685 –0.9874* 0.6340 
 (0.7589) (0.5603) (0.4819) (0.3896) (0.3857) (0.3381) (0.6558) (0.3786) (0.8770) 
  Cumulative-TAP –0.5836 0.3679 0.5243** 0.3601 –0.1826 –0.1773 0.1305 –0.1871 –0.3438 
 (0.6742) (0.4209) (0.1901) (0.2223) (0.1941) (0.2031) (0.2214) (0.2253) (0.2639) 
  After-TAP –11.4494* –2.8640 –1.7533** –0.7755 –0.0087 0.0009 –0.3223 2.5925* –8.4022 
 (4.7269) (1.5610) (0.6078) (0.4995) (0.4636) (0.3637) (0.3351) (1.0875) (5.9572) 
  Time Elapsed Between Fall and Spring Assessment 0.0522*** 0.0396*** 0.0379*** 0.0318*** 0.0227*** 0.0203*** 0.0105* 0.0087 –0.0030 
 (0.0037) (0.0032) (0.0032) (0.0031) (0.0032) (0.0032) (0.0042) (0.0065) (0.0072) 
  Average Teacher Salary/100 –0.0425 0.0411* 0.0073 0.0212 0.0258 0.0388 0.0369 0.0159 0.0496 
 (0.0264) (0.0194) (0.0178) (0.0172) (0.0197) (0.0198) (0.0243) (0.0351) (0.0437) 
  Student–Teacher Ratio –0.0280 –0.0447 –0.0699 –0.0586 –0.1356*** –0.0165 0.0365 –0.0876 –0.0877 
 (0.0577) (0.0353) (0.0418) (0.0347) (0.0383) (0.0373) (0.0433) (0.0681) (0.0779) 
  Percent Below Basic (Math) –0.0357** –0.0290** –0.0353*** –0.0465*** –0.0344*** –0.0318*** –0.0292** –0.0159 –0.0179 
 (0.0123) (0.0102) (0.0085) (0.0091) (0.0090) (0.0078) (0.0099) (0.0148) (0.0204) 
  Percent Basic (Math) –0.0375* –0.0333** –0.0145 –0.0287** –0.0165 –0.0084 –0.0034 0.0026 0.0039 
 (0.0155) (0.0118) (0.0103) (0.0095) (0.0122) (0.0136) (0.0159) (0.0218) (0.0224) 
  Mill's Ratio 0.1137 0.1041 0.0779 0.2743* 0.5921** 0.3253* 0.2282 0.9961*** 2.0456*** 
 (0.2488) (0.2870) (0.1543) (0.1340) (0.1787) (0.1617) (0.2098) (0.2146) (0.4904) 
  Intercept 6.6841** 3.2756* 1.4241 2.5392 5.3839** 2.4016 –0.8358 2.1558 0.5548 
 (2.2311) (1.6643) (1.5471) (1.5237) (1.8854) (1.7264) (2.1411) (2.3640) (2.7727) 
  R2 0.0552 0.0345 0.0343 0.0324 0.0235 0.0186 0.0135 0.0154 0.0156 
  N 90,555 127,474 128,567 135,480 135,781 132,893 129,642 25,468 16,849 
Grade
Dependent Variables2345678910
Independent Variables          
  First-TAP 3.0403*** 0.2743 1.4908** 1.3651*** –0.1465 –0.1512 0.3685 –0.9874* 0.6340 
 (0.7589) (0.5603) (0.4819) (0.3896) (0.3857) (0.3381) (0.6558) (0.3786) (0.8770) 
  Cumulative-TAP –0.5836 0.3679 0.5243** 0.3601 –0.1826 –0.1773 0.1305 –0.1871 –0.3438 
 (0.6742) (0.4209) (0.1901) (0.2223) (0.1941) (0.2031) (0.2214) (0.2253) (0.2639) 
  After-TAP –11.4494* –2.8640 –1.7533** –0.7755 –0.0087 0.0009 –0.3223 2.5925* –8.4022 
 (4.7269) (1.5610) (0.6078) (0.4995) (0.4636) (0.3637) (0.3351) (1.0875) (5.9572) 
  Time Elapsed Between Fall and Spring Assessment 0.0522*** 0.0396*** 0.0379*** 0.0318*** 0.0227*** 0.0203*** 0.0105* 0.0087 –0.0030 
 (0.0037) (0.0032) (0.0032) (0.0031) (0.0032) (0.0032) (0.0042) (0.0065) (0.0072) 
  Average Teacher Salary/100 –0.0425 0.0411* 0.0073 0.0212 0.0258 0.0388 0.0369 0.0159 0.0496 
 (0.0264) (0.0194) (0.0178) (0.0172) (0.0197) (0.0198) (0.0243) (0.0351) (0.0437) 
  Student–Teacher Ratio –0.0280 –0.0447 –0.0699 –0.0586 –0.1356*** –0.0165 0.0365 –0.0876 –0.0877 
 (0.0577) (0.0353) (0.0418) (0.0347) (0.0383) (0.0373) (0.0433) (0.0681) (0.0779) 
  Percent Below Basic (Math) –0.0357** –0.0290** –0.0353*** –0.0465*** –0.0344*** –0.0318*** –0.0292** –0.0159 –0.0179 
 (0.0123) (0.0102) (0.0085) (0.0091) (0.0090) (0.0078) (0.0099) (0.0148) (0.0204) 
  Percent Basic (Math) –0.0375* –0.0333** –0.0145 –0.0287** –0.0165 –0.0084 –0.0034 0.0026 0.0039 
 (0.0155) (0.0118) (0.0103) (0.0095) (0.0122) (0.0136) (0.0159) (0.0218) (0.0224) 
  Mill's Ratio 0.1137 0.1041 0.0779 0.2743* 0.5921** 0.3253* 0.2282 0.9961*** 2.0456*** 
 (0.2488) (0.2870) (0.1543) (0.1340) (0.1787) (0.1617) (0.2098) (0.2146) (0.4904) 
  Intercept 6.6841** 3.2756* 1.4241 2.5392 5.3839** 2.4016 –0.8358 2.1558 0.5548 
 (2.2311) (1.6643) (1.5471) (1.5237) (1.8854) (1.7264) (2.1411) (2.3640) (2.7727) 
  R2 0.0552 0.0345 0.0343 0.0324 0.0235 0.0186 0.0135 0.0154 0.0156 
  N 90,555 127,474 128,567 135,480 135,781 132,893 129,642 25,468 16,849 

Notes: Models include year fixed effects, NWEA cohort fixed effects, school-level controls for race/ethnicity, special education, free/reduced lunch status, and student-level controls for race/ethnicity and gender. Standard errors reported in parentheses. Standard errors corrected for clustering of students within schools.

*Statistically significant at the 10% level; **statistically significant at the 5% level; ***statistically significant at the 1% level.

APPENDIX  F

Table F.1.
OLS With Pre-TAP Indicator, Full Sample
Grade
Dependent Variables2345678910
Independent Variables          
  TAP 2.7146*** 0.9516** 1.8708*** 2.0534*** 1.0319** 0.3077 0.9576* 0.9012 3.2917*** 
 (0.4505) (0.3361) (0.2872) (0.2953) (0.3441) (0.3701) (0.4845) (0.7964) (0.8666) 
  Pre-TAP –0.0647 0.0861 0.0356 0.2521 1.0325** 0.8944* 0.7483 1.6717*** 3.8733*** 
 (0.4498) (0.6238) (0.2845) (0.2518) (0.3626) (0.3782) (0.4569) (0.4555) (1.0174) 
  Time Elapsed Between Fall and Spring Assessment 0.0522*** 0.0396*** 0.0380*** 0.0319*** 0.0228*** 0.0203*** 0.0104* 0.0088 –0.0032 
 (0.0037) (0.0032) (0.0033) (0.0031) (0.0032) (0.0032) (0.0042) (0.0065) (0.0071) 
  Average Teacher Salary/100 –0.0427 0.0412* 0.0069 0.0210 0.0255 0.0386 0.0365 0.0181 0.0522 
 (0.0263) (0.0194) (0.0178) (0.0172) (0.0198) (0.0198) (0.0242) (0.0354) (0.0433) 
  Student–Teacher Ratio –0.0293 –0.0444 –0.0690 –0.0579 –0.1327*** –0.0127 0.0416 –0.0911 –0.0920 
 (0.0577) (0.0356) (0.0417) (0.0347) (0.0381) (0.0374) (0.0436) (0.0693) (0.0797) 
  Percent Below Basic (Math) –0.0355** –0.0291** –0.0350*** –0.0472*** –0.0362*** –0.0323*** –0.0295** –0.0154 –0.0167 
 (0.0123) (0.0102) (0.0087) (0.0092) (0.0089) (0.0078) (0.0099) (0.0147) (0.0212) 
  Percent Basic (Math) –0.0373* –0.0329** –0.0146 –0.0279** –0.0133 –0.0065 –0.0017 –0.0010 0.0022 
 (0.0155) (0.0117) (0.0104) (0.0095) (0.0120) (0.0134) (0.0156) (0.0220) (0.0235) 
  Intercept 6.7426** 3.2574 1.4469 2.5418 5.2581** 2.1752 –1.0108 2.4924 0.5454 
 (2.2349) (1.6646) (1.5493) (1.5250) (1.8869) (1.7081) (2.1205) (2.3891) (2.7771) 
  R2 0.0551 0.0345 0.0342 0.0325 0.0234 0.0187 0.0136 0.0151 0.0148 
  N 90,555 127,474 128,567 135,480 135,781 132,893 129,642 25,468 16,849 
Grade
Dependent Variables2345678910
Independent Variables          
  TAP 2.7146*** 0.9516** 1.8708*** 2.0534*** 1.0319** 0.3077 0.9576* 0.9012 3.2917*** 
 (0.4505) (0.3361) (0.2872) (0.2953) (0.3441) (0.3701) (0.4845) (0.7964) (0.8666) 
  Pre-TAP –0.0647 0.0861 0.0356 0.2521 1.0325** 0.8944* 0.7483 1.6717*** 3.8733*** 
 (0.4498) (0.6238) (0.2845) (0.2518) (0.3626) (0.3782) (0.4569) (0.4555) (1.0174) 
  Time Elapsed Between Fall and Spring Assessment 0.0522*** 0.0396*** 0.0380*** 0.0319*** 0.0228*** 0.0203*** 0.0104* 0.0088 –0.0032 
 (0.0037) (0.0032) (0.0033) (0.0031) (0.0032) (0.0032) (0.0042) (0.0065) (0.0071) 
  Average Teacher Salary/100 –0.0427 0.0412* 0.0069 0.0210 0.0255 0.0386 0.0365 0.0181 0.0522 
 (0.0263) (0.0194) (0.0178) (0.0172) (0.0198) (0.0198) (0.0242) (0.0354) (0.0433) 
  Student–Teacher Ratio –0.0293 –0.0444 –0.0690 –0.0579 –0.1327*** –0.0127 0.0416 –0.0911 –0.0920 
 (0.0577) (0.0356) (0.0417) (0.0347) (0.0381) (0.0374) (0.0436) (0.0693) (0.0797) 
  Percent Below Basic (Math) –0.0355** –0.0291** –0.0350*** –0.0472*** –0.0362*** –0.0323*** –0.0295** –0.0154 –0.0167 
 (0.0123) (0.0102) (0.0087) (0.0092) (0.0089) (0.0078) (0.0099) (0.0147) (0.0212) 
  Percent Basic (Math) –0.0373* –0.0329** –0.0146 –0.0279** –0.0133 –0.0065 –0.0017 –0.0010 0.0022 
 (0.0155) (0.0117) (0.0104) (0.0095) (0.0120) (0.0134) (0.0156) (0.0220) (0.0235) 
  Intercept 6.7426** 3.2574 1.4469 2.5418 5.2581** 2.1752 –1.0108 2.4924 0.5454 
 (2.2349) (1.6646) (1.5493) (1.5250) (1.8869) (1.7081) (2.1205) (2.3891) (2.7771) 
  R2 0.0551 0.0345 0.0342 0.0325 0.0234 0.0187 0.0136 0.0151 0.0148 
  N 90,555 127,474 128,567 135,480 135,781 132,893 129,642 25,468 16,849 

Notes: Models include year fixed effects, NWEA cohort fixed effects, school-level controls for race/ethnicity, special education, free/reduced lunch status, and student-level controls for race/ethnicity and gender. Standard errors reported in parentheses. Standard errors corrected for clustering of students within schools.

*Statistically significant at the 10% level; **statistically significant at the 5% level; ***statistically significant at the 1% level.

APPENDIX  G

Table G.1.
OLS with Pre-TAP Indicator, Matched Sample
Grade
Dependent Variables2345678910
Independent Variables          
  TAP 1.6665*** 0.4790 2.0369*** 1.8761*** 1.1691** –0.0409 0.2853 –0.2440 5.1874 
 (0.4347) (0.4237) (0.3328) (0.3300) (0.3640) (0.3877) (0.5094) (1.0872) (4.7119) 
  Pre-TAP –0.0644 –0.2224 0.3329 0.1115 1.5719*** 1.3741** 1.0797* 4.1393* 8.8730 
 (0.3816) (0.4715) (0.4102) (0.2728) (0.3684) (0.4168) (0.5098) (1.3880) (5.8275) 
  Time Elapsed Between Fall and Spring Assessment 0.0528*** 0.0594*** 0.0391*** 0.0390*** 0.0239*** 0.0336*** 0.0226** 0.0066 –0.0028 
 (0.0075) (0.0065) (0.0052) (0.0052) (0.0059) (0.0058) (0.0074) (0.0157) (0.0248) 
  Average Teacher Salary/100 –0.0722 0.0063 –0.0365 –0.0478 –0.0482 –0.0479 –0.0622 1.3367* 1.6151 
 (0.0417) (0.0452) (0.0380) (0.0353) (0.0442) (0.0410) (0.0554) (0.5406) (1.2428) 
  Student–Teacher Ratio –0.0975 –0.0568 –0.0327 –0.0546 –0.2054*** –0.0352 –0.0231 –0.0694 0.6101* 
 (0.0707) (0.0571) (0.0563) (0.0611) (0.0508) (0.0665) (0.0807) (0.1891) (0.1825) 
  Percent Below Basic (Math) –0.0373 –0.0212 –0.0541** –0.0580*** –0.0517*** –0.0323* 0.0002 –0.1202 –0.6203 
 (0.0219) (0.0190) (0.0165) (0.0165) (0.0124) (0.0137) (0.0161) (0.1187) (0.5372) 
  Percent Basic (Math) –0.0312 0.0127 –0.0179 –0.0516* –0.0140 0.0007 –0.0164 –0.3892** –0.0622 
 (0.0298) (0.0189) (0.0213) (0.0233) (0.0242) (0.0255) (0.0230) (0.1116) (0.1332) 
  Intercept 17.3609* 2.8745 9.1431* 8.2357** 11.7670* 4.5575 5.7733 –36.6384 –56.7997 
 (8.1862) (4.8481) (3.9827) (2.9717) (4.8849) (3.4671) (4.0938) (17.6814) (46.8718) 
  R2 0.0992 0.0672 0.0457 0.0447 0.0331 0.0288 0.0224 0.0272 0.0357 
  N 17,396 21,683 22,889 23,248 26,094 25,710 24,863 3,965 2,282 
Grade
Dependent Variables2345678910
Independent Variables          
  TAP 1.6665*** 0.4790 2.0369*** 1.8761*** 1.1691** –0.0409 0.2853 –0.2440 5.1874 
 (0.4347) (0.4237) (0.3328) (0.3300) (0.3640) (0.3877) (0.5094) (1.0872) (4.7119) 
  Pre-TAP –0.0644 –0.2224 0.3329 0.1115 1.5719*** 1.3741** 1.0797* 4.1393* 8.8730 
 (0.3816) (0.4715) (0.4102) (0.2728) (0.3684) (0.4168) (0.5098) (1.3880) (5.8275) 
  Time Elapsed Between Fall and Spring Assessment 0.0528*** 0.0594*** 0.0391*** 0.0390*** 0.0239*** 0.0336*** 0.0226** 0.0066 –0.0028 
 (0.0075) (0.0065) (0.0052) (0.0052) (0.0059) (0.0058) (0.0074) (0.0157) (0.0248) 
  Average Teacher Salary/100 –0.0722 0.0063 –0.0365 –0.0478 –0.0482 –0.0479 –0.0622 1.3367* 1.6151 
 (0.0417) (0.0452) (0.0380) (0.0353) (0.0442) (0.0410) (0.0554) (0.5406) (1.2428) 
  Student–Teacher Ratio –0.0975 –0.0568 –0.0327 –0.0546 –0.2054*** –0.0352 –0.0231 –0.0694 0.6101* 
 (0.0707) (0.0571) (0.0563) (0.0611) (0.0508) (0.0665) (0.0807) (0.1891) (0.1825) 
  Percent Below Basic (Math) –0.0373 –0.0212 –0.0541** –0.0580*** –0.0517*** –0.0323* 0.0002 –0.1202 –0.6203 
 (0.0219) (0.0190) (0.0165) (0.0165) (0.0124) (0.0137) (0.0161) (0.1187) (0.5372) 
  Percent Basic (Math) –0.0312 0.0127 –0.0179 –0.0516* –0.0140 0.0007 –0.0164 –0.3892** –0.0622 
 (0.0298) (0.0189) (0.0213) (0.0233) (0.0242) (0.0255) (0.0230) (0.1116) (0.1332) 
  Intercept 17.3609* 2.8745 9.1431* 8.2357** 11.7670* 4.5575 5.7733 –36.6384 –56.7997 
 (8.1862) (4.8481) (3.9827) (2.9717) (4.8849) (3.4671) (4.0938) (17.6814) (46.8718) 
  R2 0.0992 0.0672 0.0457 0.0447 0.0331 0.0288 0.0224 0.0272 0.0357 
  N 17,396 21,683 22,889 23,248 26,094 25,710 24,863 3,965 2,282 

Notes: Models include year fixed effects, NWEA cohort fixed effects, school-level controls for race/ethnicity, special education, free/reduced lunch status, and student-level controls for race/ethnicity and gender. Standard errors reported in parentheses. Standard errors corrected for clustering of students within schools.

*Statistically significant at the 10% level; **statistically significant at the 5% level; ***statistically significant at the 1% level.