Episodic memory is associated with the encoding and retrieval of context information and with a subjective sense of reexperiencing past events. The neural correlates of episodic retrieval have been extensively studied using fMRI, leading to the identification of a “general recollection network” including medial temporal, parietal, and prefrontal regions. However, in these studies, it is difficult to disentangle the effects of context retrieval from recollection. In this study, we used fMRI to determine the extent to which the recruitment of regions in the recollection network is contingent on context reinstatement. Participants were scanned during a cued recognition test for target words from encoded sentences. Studied target words were preceded by either a cue word studied in the same sentence (thus congruent with encoding context) or a cue word studied in a different sentence (thus incongruent with encoding context). Converging fMRI results from independently defined ROIs and whole-brain analysis showed regional specificity in the recollection network. Activity in hippocampus and parahippocampal cortex was specifically increased during successful retrieval following congruent context cues, whereas parietal and prefrontal components of the general recollection network were associated with confident retrieval irrespective of contextual congruency. Our findings implicate medial temporal regions in the retrieval of semantic context, contributing to, but dissociable from, recollective experience.