Lesion and neuroimaging studies suggest that orbito-frontal cortex (OFC) supports temporal aspects of episodic memory. However, it is unclear whether OFC contributes to the encoding and/or retrieval of temporal context and whether it is selective for temporal relative to nontemporal (spatial) context memory. We addressed this issue with two complimentary studies: functional magnetic resonance imaging to measure OFC activity associated with successful temporal and spatial context memory during encoding and retrieval in healthy young participants, and a neuropsychological investigation to measure changes in spatial and temporal context memory in OFC lesion patients. Imaging results revealed that OFC contributed to encoding and retrieval of associations between objects and their temporal but not their spatial contexts. Consistent with this, OFC patients exhibited impairments in temporal but not spatial source memory accuracy. These results suggest that OFC plays a critical role in the formation and subsequent retrieval of temporal context.