Elucidating the neural mechanisms involved in aversive conditioning helps find effective treatments for psychiatric disorders such as anxiety disorder and phobia. Previous studies using fMRI and human subjects have reported that the amygdala plays a role in this phenomenon. However, the noxious stimuli that were used as unconditioned stimuli in previous studies (e.g., electric shock) might have been ecologically invalid because we seldom encounter such stimuli in daily life. Therefore, we investigated whether a face stimulus could be conditioned by using a voice that had negative emotional valence and was collected from a real-life environment. A skin conductance response showed that healthy subjects were conditioned by using these stimuli. In an fMRI study, there was greater amygdala activation in response to the faces that had been paired with the voice than to those that had not. The right amygdala showed transient activity in the early stage of acquisition. A psychophysiological interaction analysis indicated that the subcortical pathway from the medial geniculate body to the amygdala played a role in conditioning. Modulation of the subcortical pathway by voice stimuli preceded the transient activity in the amygdala. The finding that an ecologically valid stimulus elicited the conditioning and amygdala response suggests that our brain is automatically processing unpleasant stimuli in daily life.