Factorial designs are widely used to study multiple treatments in one experiment. While t -tests using a fully-saturated “long” model provide valid inferences, “short” model t -tests (that ignore interactions) yield higher power if interactions are zero, but incorrect inferences otherwise. Of 27 factorial experiments published in top-5 journals (2007–2017), 19 use the short model. After including interactions, over half of their results lose significance. Based on recent econometric advances, we show that power improvements over the long model are possible. We provide practical guidance for the design of new experiments and the analysis of completed experiments.

We study the impact of reforms that introduced more stringent, biometric ID requirements into India's largest social protection program, using large-scale randomized and natural experiments. Corruption fell but with substantial costs to legitimate beneficiaries, 1.5-2 million of whom lost access to benefits at some point during the reforms. At the same time, adverse effects appear to have been driven primarily by decisions about the way the transition was managed, illustrating both the risks of rapid reforms, and how the impacts of promising new technologies can be highly sensitive to the protocols governing their use.