Estimating duration depends on the sequential integration (accumulation) of temporal information in working memory. Using fMRI, we directly compared the accumulation of information in temporal versus spatial domains. Participants estimated either the duration or distance of the dynamic trajectory of a moving dot or, in a control condition, a static line stimulus. Comparing the duration versus distance of static lines activated an extensive cortico-striatal network. By contrast, comparing the duration versus distance of dynamic trajectories, both of which required sequential integration of information, activated SMA alone. Indeed, activity in SMA, as well as right inferior occipital cortex, increased parametrically as a function of stimulus duration and also correlated with individual differences in the propensity to overestimate stimulus duration. By contrast, activity in primary visual cortex increased parametrically as a function of stimulus distance. Crucially, a direct comparison of the parametric responses to duration versus distance revealed that activity in SMA increased incrementally as a function of stimulus duration but not as a function of stimulus distance. Collectively, our results indicate that SMA responds to the active accumulation of information selectively in the temporal domain.