Careful examination of the structure determining treatment choice and outcomes, as advocated by Heckman (2008), is central to the design of treatment effect estimators and, in particular, proper choice of covariates. Here, we demonstrate how causal diagrams developed in the machine learning literature by Judea Pearl and his colleagues, but not so well known to economists, can play a key role in this examination by using these methods to give a detailed analysis of the choice of efficient covariates identified by Hahn (2004).

We take a model selection approach to the question of whether a class of adaptive prediction models (artificial neural networks) is useful for predicting future values of nine macroeconomic variables. We use a variety of out-of-sample forecast-based model selection criteria, including forecast error measures and forecast direction accuracy. Ex ante or real-time forecasting results based on rolling window prediction methods indicate that multivariate adaptive linear vector autoregression models often outperform a variety of (1) adaptive and nonadaptive univariate models, (2) nonadaptive multivariate models, (3) adaptive nonlinear models, and (4) professionally available survey predictions. Further, model selection based on the in-sample Schwarz information criterion apparently fails to offer a convenient shortcut to true out-of-sample performance measures.