The sense of agency (SoA) refers to a constitutional aspect of the self describing the extent to which individuals feel in control over their actions and consequences thereof. Although the SoA has been associated with mental health and well-being, it is still unknown how interindividual variability in the SoA is embedded in the intrinsic brain organization. We hypothesized that the prospective component of an implicit SoA is associated with brain networks related to SoA and sensorimotor predictions on multiple spatial scales. We replicated previous findings by showing a significant prospective SoA as indicated by intentional binding effects. Then, using task-free fMRI and graph analysis, we analyzed associations between intentional binding effects and the intrinsic brain organization at regional, modular, and whole-brain scales. The results showed that intermodular connections of a frontoparietal module including the premotor cortex, supramarginal gyrus, and dorsal precuneus are associated with individual differences in prospective intentional binding. Notably, prospective intentional binding effects were also related to global brain modularity within a specific structural resolution range. These findings suggest that an implicit SoA generated through sensorimotor predictions relies on the intrinsic organization of the brain connectome on both local and global scales.