Detecting and characterizing causal interdependencies and couplings between different activated brain areas from functional neuroimage time series measurements of their activity constitutes a significant step toward understanding the process of brain functions. In this letter, we make the simple point that all current statistics used to make inferences about directed influences in functional neuroimage time series are variants of the same underlying quantity. This includes directed transfer entropy, transinformation, Kullback-Leibler formulations, conditional mutual information, and Granger causality. Crucially, in the case of autoregressive modeling, the underlying quantity is the likelihood ratio that compares models with and without directed influences from the past when modeling the influence of one time series on another. This framework is also used to derive the relation between these measures of directed influence and the complexity or the order of directed influence. These results provide a framework for unifying the Kullback-Leibler divergence, Granger causality, and the complexity of directed influence.
With the widespread acceptance of the network description onto brain processing functions, identification and quantification of directed interactions between brain structures that support the processing of specific brain functions (perceptual, cognitive or motor functions) are fundamental issues in neuroscience. For example, information about the functioning or dysfunctioning of the brain can be inferred from the network structure. The current theories of schizophrenia emphasize that the core aspects of the pathophysiology are due to the disconnection hypothesis (Friston, 1998) rather than deficits in specific brain areas or neurotransmitter systems. Neuroimage techniques, such as electroencephalography (EEG), magnetoencephalography (MEG), functional near infrared imaging (fNIR), and functional magnetic magnetic resonance imaging (fMRI) are commonly employed to generate time series measurements of brain structure activities to address questions of functional and effective connectivity that are fundamental for the description of these brain networks (Roebroeck, Formissamo, & Goebel, 2005; Hesse, Moller, Arnold, & Schack, 2003). The key distinction between functional and effective connectivity pertains to whether one is assessing simple statistical dependencies (functional connectivity) or trying to infer the parameters of an underlying model of connectivity (effective connectivity). The interesting issue, from our perspective, is that Granger causality and related analysis allow the in vivo study of effective connectivity through the statistical analysis of neuroimaging time series. Although we have been in motivated this work by the analysis of neuroimaging time series, all the arguments presented in this letter apply to any inference about directed causality in any time series.
Granger causality analysis (Granger, 1969; Geweke, 1982; Kim, Putrino, Ghosh, & Brown, 2011; Quinn, Coleman, & Kiyavash, 2011) and transfer entropy methods (Schreider, 2000; Massey, 1990; Kamitake, Harashima, & Miyakawa, 1984; Amblard & Michel, 2011; Quinn, Coleman, Kiyavash, & Hatsopoulos, 2011) have become increasingly used approaches to explore directed influences between brain structures using functional neuroimage data (Hinrichs, Heinze, & Schoenfeld, 2006; Seghouane, 2011). While both approaches target the same objective—measuring the effect that one brain structure has on another—the transfer entropy specifically uses the Kullback-Leibler divergence to measure the influence of extra information. The general idea of Granger causality is defined in terms of upgrading predictability. If a signal X causes a signal Y, the knowledge of the past of both X and Y should improve the prediction of the future of Y in comparison to the knowledge of the past of Y only, whereas the idea behind transfer entropy is defined in terms of influence on conditional probabilities as measured with the Kullback-Leibler divergence (Kullback & Leibler, 1951). If a signal Y does not cause a signal X, then the probability density describing the future of X conditioned on its past should not be different from the probability density describing the future of X conditioned on its past and the past of Y as measured by the Kullback-Leibler divergence.
In this letter, we make the simple point that all current statistics used to make inferences about directed influences in functional neuroimage time series are variants of the same underlying quantity. This includes directed transfer entropy, transinformation, Kullback-Leibler formulations, conditional mutual information, and Granger causality. In the autoregressive modeling framework, the relation between these information measures of directed influence, Granger causality, and the complexity of directed influence is derived in both the univariate and multivariate cases. The rest of this letter is organized as follows. After a brief formulation of the problem, the different measures of directed influence are introduced in section 2, and the different relations of these measures are described. The autoregressive modeling is introduced in section 3, and the link with Granger causality and the complexity of directed influence is established for the univariate case. The multivariate case is treated in section 4. A simulation example is described in section 5, and the conclusion is given in section 6.
2. Problem Formulation and Measures of Directed Influence
In a typical fMRI experiment, several regions of interest (ROIs) are a priori identified in the brain. Each ROI is represented in the fMRI data set by multiple voxels, where each voxel is a variable comprising a single time series reflecting changes in the underlying metabolic signal. A standard approach used to assess directed influence between two ROIs is to derive a single time series for each ROI either by averaging or extracting a principal component (Zhou, Chen, Ding, Lu, & Liu, 2009); alternatively repeated pairwise analysis can be performed on pairs of voxels.
The computation of the above measure requires estimating multivariate probability density functions. Two different approaches can be used to estimate these multivariate entropies from experimental data: nonparametric or parametric. The nonparametric approach requires the choice of a kernel and the estimation of its bandwidth (Hinrichs et al., 2006; Schreider, 2000). The parametric approach requires the selection of the appropriate model and the estimation of its parameters. We adopt the parametric approach using autoregressive modeling in this letter for its relation with Granger causality (Granger, 1969; Geweke, 1982).
3. Parametric Estimation of the Directed Influence
The derivation is given in appendix B.
Therefore, (and the Granger causality) measures the assistance of Y in predicting the future of X by asymptotically measuring the order or complexity by which Y influences X. As a result, there is an attractive complementarity between the notions’ complexity, information, and prediction when it comes to measuring the interaction or influence between time series. The parametric estimation of the proposed measure, equation 2.2, using models 3.1 and 3.3, requires estimating the parameters of these models as well as the determination of their orders. The estimation of the parameters can be derived using the least squares method or the Yule-Walker equations (Brockwell & Davis, 1991). The estimation of the orders p and q can be obtained using a model selection criterion (Seghouane & Amari, 2007; Seghouane & Bekara, 2004; Seghouane, 2010).
4. The Multivariate Case
Instead of an approach that consists of deriving a single time series for each ROI, a possible alternative to assess the directed influence between two ROIs would be to consider multivariate random variables represented by vector time series characterizing the group of voxels comprising each ROI (Barrett, Seth, & Barnett, 2010). In this case, the autoregressive modeling requires the use of vector autoregressive models.
The derivation is given in appendix C.
In comparison to equation 3.7, note the presence of m2 on the right-hand side of equation 4.6. This represents the dimension of the coefficient matrices Ai, and Bi, . In the univariate case, m=1, (22) is equal to equation 3.7. Therefore, (and the Granger causality) measures the directed influence of Y on X by asymptotically measuring the complexity by which Y influences X as measured by the number of required new parameters.
The parametric estimation of the proposed measure, equation 2.2, using models 4.1 and 4.3, requires the estimation of the parameters of these models as well as the determination of their orders. The estimation of the parameters can be obtained using the ordinary least squares method or the multivariate Yule-Walker equations (Brockwell & Davis, 1991). The estimation of orders p and q can be obtained using an appropriate model selection criterion (Hurvich & Tsai, 1993; Seghouane, 2006).
5. Simulation Examples
To verify relation 3.7 of proposition 2, we generated 50 data sets of size
using two models:
For each sample size N, the expectation of relation 3.7 was obtained by averaging relation 3.6 () over the 50 realizations. For both models, each N, and each data set, the parameter estimates of the AR and ARX models were obtained by least squares. The results of the numerical estimations closely correspond to the theoretical estimate (see Figure 1) for model 1 and (see Figure 2) for model 2.
Directed influence measures are finding increasing applications in neuroscience. In this letter, the relations between all current statistics used to make inference about directed influences have been derived. It has been established that these statistics are variants of the same underlying quantity, the optimum statistics under the Neyman-Pearson lemma. In the context of autoregressive modeling, it has been shown that asymptotically, both measures quantify the same information about the directed influence—the complexity of directed influence as measured by the order of the extra-autoregressive part of the model used to characterize the influence of the past of one time series on another. In the multivariate case, this complexity of influence is represented by the order of the extra-autoregressive part of the model used to characterize the influence of the past of one multivariate time series on another, multiplied by the dimension of the coefficient matrices. This corresponds to the number of extra parameters needed to model the influence. There is therefore an attractive complementarity among complexity, information, and prediction when it comes to measuring directed influences from neuroimage time series measurements.
Appendix A: Proof of Proposition 1
Appendix B: Proof of Proposition 2
Appendix C: Proof of Proposition 3
We thank the anonymous reviewers for their comments and suggestions, which helped to improve the quality and presentation of this letter. NICTA is funded by the Australian government as represented by the Department of Broadband, Communications and the Digital Economy and the Australian Research Council through the ICT Centre of Excellence program. This work was partly supported by the Human Frontier Science Program (grant RGY 80/2008) and the Japanese Society for Promoting Science Fellowship (number S-10738).