Medial temporal lobe structures such as the hippocampus have been shown to play a critical role in mnemonic processes, with additional recruitment of the amygdala when memories contain emotional content. Thus far, studies that have examined the relationship between amygdala activity and memory have typically relied on emotional content of the kind that is rarely encountered in day-to-day interactions. The present event-related functional magnetic resonance imaging study investigates whether amygdala activity supports emotional memory during the more subtle social interactions that punctuate everyday life. Across four training sessions, subjects learned common first names for unfamiliar faces in the presence or absence of additional contextual information that was positive, negative, and neutral in valence (e.g., “Emily helps the homeless,” “Bob is a deadbeat dad,” “Eric likes carrots”). During scanning, subjects performed a yes/no recognition memory test on studied and novel faces. Results revealed a functional dissociation within the medial temporal lobe. Whereas a region within the right hippocampus responded strongly to all faces that had been paired with a description, regardless of its valence, activity in the right amygdala was uniquely sensitive to faces that had been previously associated with emotional descriptions (negative and positive > neutral). This pattern of activity in the amygdala was preserved even when the emotional contexts associated with faces could not be explicitly retrieved, suggesting a role for the amygdala in providing a nonspecific arousal indicator in response to viewing individuals with emotionally colored pasts.