An individual's readiness to switch tasks (cognitive flexibility) varies over time, in part, as the result of reinforcement learning based on the statistical structure of the world around them. Consequently, the behavioral cost associated with task-switching is smaller in contexts where switching is frequent than where it is rare, but the underlying brain mechanisms of this adaptation in cognitive flexibility are not well understood. Here, we manipulated the likelihood of switches across blocks of trials in a classic cued task-switching paradigm while participants underwent fMRI. As anticipated, behavioral switch costs decreased as the probability of switching increased, and neural switch costs were observed in lateral and medial frontoparietal cortex. To study moment-by-moment adjustments in cognitive flexibility at the neural level, we first fitted the behavioral RT data with reinforcement learning algorithms and then used the resulting trial-wise prediction error estimate as a regressor in a model-based fMRI analysis. The results revealed that lateral frontal and parietal cortex activity scaled positively with unsigned switch prediction error and that there were no brain regions encoding signed (i.e., switch- or repeat-specific) prediction error. Taken together, this study documents that adjustments in cognitive flexibility to time-varying switch demands are mediated by frontoparietal cortex tracking the likelihood of forthcoming task switches.