Cognitive control involves the flexible allocation of mental resources during goal-directed behavior and comprises three correlated but distinct domains—inhibition, shifting, and working memory. The work of Don Stuss and others has demonstrated that frontal and parietal cortices are crucial to cognitive control, particularly in normal aging, which is characterized by reduced control mechanisms. However, the structure–function relationships specific to each domain and subsequent impact on performance are not well understood. In the current study, we examined both age and individual differences in functional activity associated with core domains of cognitive control in relation to fronto-parietal structure and task performance. Participants (n = 140, aged 20–86 years) completed three fMRI tasks: go/no-go (inhibition), task switching (shifting), and n-back (working memory), in addition to structural and diffusion imaging. All three tasks engaged a common set of fronto-parietal regions; however, the contributions of age, brain structure, and task performance to functional activity were unique to each domain. Aging was associated with differences in functional activity for all tasks, largely in regions outside common fronto-parietal control regions. Shifting and inhibition showed greater contributions of structure to overall decreases in brain activity, suggesting that more intact fronto-parietal structure may serve as a scaffold for efficient functional response. Working memory showed no contribution of structure to functional activity but had strong effects of age and task performance. Together, these results provide a comprehensive and novel examination of the joint contributions of aging, performance, and brain structure to functional activity across multiple domains of cognitive control.