Skip to Main Content

To test for possibly outsized negative effects on low-probability evaluation days, we calculate our main specification and drop observations with the lowest evaluation probability. For completeness, we also look at dropping the highest evaluation probability observations. Table 7 shows the effect of evaluation probability on a teacher's evaluation score after dropping observations in the bottom 5 and 10 percent of treatment, the top 5 and 10 percent of treatment, and then both the top 5 percent and bottom 5 percent simultaneously. We also calculate the difference between the full sample and the subsample, though none of these differences is statistically significant.

Table 7.

Sample Robustness Checks for Evaluation Probability

Unannounced Evaluations (Eval)
ExternalInternalExternalInternal
Eval 1Eval 2Eval 2Eval 3
2011—12DifferenceAll YearsDifferenceAll YearsDifferenceAll YearsDifference
Full sample 0.406*  0.309***  0.738***  0.173**  
 (0.240)  (0.078)  (0.169)  (0.072)  
Drop bottom 5 percent 0.461* 0.055 0.288*** −0.021 0.553*** −0.185 0.075 −0.098 
of treatment (0.239)  (0.080)  (0.161)  (0.075)  
Drop bottom 10 percent 0.509** 0.103 0.277*** −0.032 0.535*** −0.203 0.072 −0.101 
of treatment (0.238)  (0.083)  (0.163)  (0.075)  
Drop top 5 percent 0.460 0.054 0.318*** 0.009 0.799*** 0.061 0.342** 0.169 
of treatment (0.500)  (0.109)  (0.296)  (0.141)  
Drop top 10 percent 0.768 0.362 0.418*** 0.109 0.528 −0.21 0.298* 0.125 
of treatment (0.648)  (0.151)  (0.397)  (0.169)  
Drop top 5 percent and 0.455 0.049 0.319*** 0.01 0.717** −0.021 0.271* 0.098 
bottom 5 percent (0.508)  (0.116)  (0.309)  (0.145)  
Unannounced Evaluations (Eval)
ExternalInternalExternalInternal
Eval 1Eval 2Eval 2Eval 3
2011—12DifferenceAll YearsDifferenceAll YearsDifferenceAll YearsDifference
Full sample 0.406*  0.309***  0.738***  0.173**  
 (0.240)  (0.078)  (0.169)  (0.072)  
Drop bottom 5 percent 0.461* 0.055 0.288*** −0.021 0.553*** −0.185 0.075 −0.098 
of treatment (0.239)  (0.080)  (0.161)  (0.075)  
Drop bottom 10 percent 0.509** 0.103 0.277*** −0.032 0.535*** −0.203 0.072 −0.101 
of treatment (0.238)  (0.083)  (0.163)  (0.075)  
Drop top 5 percent 0.460 0.054 0.318*** 0.009 0.799*** 0.061 0.342** 0.169 
of treatment (0.500)  (0.109)  (0.296)  (0.141)  
Drop top 10 percent 0.768 0.362 0.418*** 0.109 0.528 −0.21 0.298* 0.125 
of treatment (0.648)  (0.151)  (0.397)  (0.169)  
Drop top 5 percent and 0.455 0.049 0.319*** 0.01 0.717** −0.021 0.271* 0.098 
bottom 5 percent (0.508)  (0.116)  (0.309)  (0.145)  

Notes: This table shows estimates for the effect of the likelihood of an in-class observation on teacher evaluation scores by different samples. To assess whether extremely low-probability or high-probability evaluation days are driving our main results, we drop observations based on their treatment size. The “Difference” column calculates the difference between the Full Sample and the subsample. Overall, these results suggest that there are nonlinearities in how evaluation probability affects teacher scores, though changing the treated sample does not qualitatively alter our results. Errors are clustered at the school-by-year level.

***Significant at the 1 percent level; **significant at the 5 percent level; *significant at the 10 percent level.

Close Modal

or Create an Account

Close Modal
Close Modal