Comprehending the interplay between spatial and temporal characteristics of neural dynamics can contribute to our understanding of information processing in the human brain. Graph neural networks (GNNs) provide a new possibility to interpret graph structured signals like those observed in complex brain networks. In our study we compare different spatio-temporal GNN architectures and study their ability to model neural activity distributions obtained in functional MRI (fMRI) studies. We evaluate the performance of the GNN models on a variety of scenarios in MRI studies and also compare it to a VAR model, which is currently often used for directed functional connectivity analysis. We show that by learning localized functional interactions on the anatomical substrate, GNN based approaches are able to robustly scale to large network studies, even when available data are scarce. By including anatomical connectivity as the physical substrate for information propagation, such GNNs also provide a multi-modal perspective on directed connectivity analysis, offering a novel possibility to investigate the spatio-temporal dynamics in brain networks.

In our study we compare different spatial and temporal modeling techniques based on graph neural networks (GNNs) for investigating the spatio-temporal dynamics in brain networks. We show that a convolutional neural network and a recurrent neural network based approach are both very suitable to capture the temporal characteristics in functional activity distributions. Further we demonstrate that structural connectome embeddings can effectively reduce the number of parameters in GNN models, by naturally including higher order topological relations between brain areas within the structural network. We compare the prediction accuracy of the GNN based approaches to a vector auto regressive model, and we show that GNNs remain considerably more accurate when brain networks become large and available data are limited. Finally we demonstrate how these spatio-temporal GNN models can provide a multi-modal perspective on directed connectivity in brain networks.

This content is only available as a PDF.

Author notes

Handling Editor: Bratislav Misic

This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. For a full description of the license, please visit

Article PDF first page preview

Article PDF first page preview