Flexible behavior requires switching between different task conditions. It is known that such task switching is associated with costs in terms of slowed RT, reduced accuracy, or both. The neural correlates of task switching have usually been studied by requiring participants to switch between distinct task conditions that recruit different brain networks. Here, we investigated the transition of neural states underlying switching between two opposite memory-related processes (i.e., memory retrieval and memory suppression) in a memory task. We investigated 26 healthy participants who performed a think/no-think task while being in the fMRI scanner. Behaviorally, we show that it was more difficult for participants to suppress unwanted memories when a no-think was preceded by a think trial instead of another no-think trial. Neurally, we demonstrate that think–no-think switches were associated with an increase in control-related and a decrease in memory-related brain activity. Neural representations of task condition, assessed by decoding accuracy, were lower immediately after task switching compared with the nonswitch transitions, suggesting a switch-induced delay in the neural transition toward the required task condition. This suggestion is corroborated by an association between condition-specific representational strength and condition-specific performance in switch trials. Taken together, we provided neural evidence from the time-resolved decoding approach to support the notion that carryover of the previous task set activation is associated with the switching cost, leading to less successful memory suppression.