Several recent studies have optimized deep neural networks to learn high-dimensional relationships linking structural and functional connectivity across the human connectome. However, the extent to which these models recapitulate individual-specific characteristics of resting-state functional brain networks remains unclear. A core concern relates to whether current individual predictions outperform simple benchmarks such as group averages and null conditions. Here, we consider two measures to statistically evaluate whether functional connectivity predictions capture individual effects. We revisit our previously published functional connectivity predictions for 1000 healthy adults and provide multiple lines of evidence supporting that our predictions successfully capture subtle individual-specific variation in connectivity. While predicted individual effects are statistically significant and outperform several benchmarks, we find that effect sizes are small (i.e., 8–11% improvement relative to group-average benchmarks). As such, initial expectations about individual prediction performance expressed by us and others may require moderation. We conclude that individual predictions can significantly outperform appropriate benchmark conditions and we provide several recommendations for future studies in this area. Future studies should statistically assess the individual prediction performance of their models using one of the measures and benchmarks provided here.

Several recent studies have optimized deep neural networks to learn high-dimensional relationships linking structural and functional connectivity across the human connectome. However, the extent to which these models recapitulate individual-specific characteristics of resting-state functional brain networks remains unclear. A core concern relates to whether current individual predictions outperform simple benchmarks such as group averages and null conditions. Here, we consider two measures to statistically evaluate whether functional connectivity predictions capture individual effects. We revisit our previously published functional connectivity predictions for 1000 healthy adults and provide multiple lines of evidence supporting that our predictions successfully capture subtle individual-specific variation in connectivity. While predicted individual effects are statistically significant and outperform several benchmarks, we find that effect sizes are small (i.e., 8–11% improvement relative to group-average benchmarks). As such, initial expectations about individual prediction performance expressed by us and others may require moderation. We conclude that individual predictions can significantly outperform appropriate benchmark conditions and we provide several recommendations for future studies in this area. Future studies should statistically assess the individual prediction performance of their models using one of the measures and benchmarks provided here.

This content is only available as a PDF.

Author notes

Handling Editor: Martijn van den Heuvel

This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. For a full description of the license, please visit https://creativecommons.org/licenses/by/4.0/legalcode.

Article PDF first page preview

Article PDF first page preview

Supplementary data