Rapid identification of a familiar face requires an image-invariant representation of person identity. A varying sample of familiar faces is necessary to disentangle image-level from person-level processing. We investigated the time course of face identity processing using a multivariate electroencephalography analysis. Participants saw ambient exemplars of celebrity faces that differed in pose, lighting, hairstyle, and so forth. A name prime preceded a face on half of the trials to preactivate person-specific information, whereas a neutral prime was used on the remaining half. This manipulation helped dissociate perceptual- and semantic-based identification. Two time intervals within the post-face onset electroencephalography epoch were sensitive to person identity. The early perceptual phase spanned 110–228 msec and was not modulated by the name prime. The late semantic phase spanned 252–1000 msec and was sensitive to person knowledge activated by the name prime. Within this late phase, the identity response occurred earlier in time (300–600 msec) for the name prime with a scalp topography similar to the FN400 ERP. This may reflect a matching of the person primed in memory with the face on the screen. Following a neutral prime, the identity response occurred later in time (500–800 msec) with a scalp topography similar to the P600f ERP. This may reflect activation of semantic knowledge associated with the identity. Our results suggest that processing of identity begins early (110 msec), with some tolerance to image-level variations, and then progresses in stages sensitive to perceptual and then to semantic features.

Perception of faces has been shown to engage a domain-specific set of brain regions, including the occipital face area (OFA) and the fusiform face area (FFA). It is commonly held that the OFA is responsible for the detection of faces in the environment, whereas the FFA is responsible for processing the identity of the face. However, an alternative model posits that the FFA is responsible for face detection and subsequently recruits the OFA to analyze the face parts in the service of identification. An essential prediction of the former model is that the OFA is not sensitive to the arrangement of internal face parts. In the current fMRI study, we test the sensitivity of the OFA and FFA to the configuration of face parts. Participants were shown faces in which the internal parts were presented in a typical configuration (two eyes above a nose above a mouth) or in an atypical configuration (the locations of individual parts were shuffled within the face outline). Perception of the atypical faces evoked a significantly larger response than typical faces in the OFA and in a wide swath of the surrounding posterior occipitotemporal cortices. Surprisingly, typical faces did not evoke a significantly larger response than atypical faces anywhere in the brain, including the FFA (although some subthreshold differences were observed). We propose that face processing in the FFA results in inhibitory sculpting of activation in the OFA, which accounts for this region's weaker response to typical than to atypical configurations.

Research about the neural basis of face recognition has investigated the timing and anatomical substrates of different stages of face processing. Scalp-recorded ERP studies of face processing have focused on the N170, an ERP with a peak latency of ∼170 msec that has long been associated with the initial structural encoding of faces. However, several studies have reported earlier ERP differences related to faces, suggesting that face-specific processes might occur before N170. Here, we examined the influence of face inversion and face race on the timing of face-sensitive scalp-recorded ERPs by examining neural responses to upright and inverted line-drawn and luminance-matched white and black faces in a sample of white participants. We found that the P100 ERP evoked by inverted faces was significantly larger than that evoked by upright faces. Although this inversion effect was statistically significant at 100 msec, the inverted-upright ERP difference peaked at 138 msec, suggesting that it might represent an activity in neural sources that overlap with P100. Inverse modeling of the inversion effect difference waveform suggested possible neural sources in pericalcarine extrastriate visual cortex and lateral occipito-temporal cortex. We also found that the inversion effect difference wave was larger for white faces. These results are consistent with behavioral evidence that individuals process the faces of their own races more configurally than faces of other races. Taken together, the inversion and race effects observed in the current study suggest that configuration influences face processing by at least 100 msec.

Refreshing is the component cognitive process of directing reflective attention to one of several active mental representations. Previous studies using fMRI suggested that refresh tasks involve a component process of initiating refreshing as well as the top–down modulation of representational regions central to refreshing. However, those studies were limited by fMRI's low temporal resolution. In this study, we used EEG to examine the time course of refreshing on the scale of milliseconds rather than seconds. ERP analyses showed that a typical refresh task does have a distinct electrophysiological response as compared to a control condition and includes at least two main temporal components: an earlier (∼400 msec) positive peak reminiscent of a P3 response and a later (∼800–1400 msec) sustained positivity over several sites reminiscent of the late directing attention positivity. Overall, the evoked potentials for refreshing representations from three different visual categories (faces, scenes, words) were similar, but multivariate pattern analysis showed that some category information was nonetheless present in the EEG signal. When related to previous fMRI studies, these results are consistent with a two-phase model, with the first phase dominated by frontal control signals involved in initiating refreshing and the second by the top–down modulation of posterior perceptual cortical areas that constitutes refreshing a representation. This study also lays the foundation for future studies of the neural correlates of reflective attention at a finer temporal resolution than is possible using fMRI.

A briefly exposed visual stimulus may not be consciously perceived if it is preceded and followed by a dissimilar visual pattern or mask. Despite the subject's lack of awareness, prior behavioral studies have shown that such masked stimuli, nevertheless, engage domain-specific processes [Dehaene, S., Naccache, L., Cohen, L., Le Bihan, D., Mangin, J.-F., Poline, J.-B., et al. Cerebral mechanisms of word masking and unconscious repetition priming. Nature Neuroscience, 4 , 752–758, 2001; Bar, M., & Biederman, I. Subliminal visual priming. Psychological Science, 9 , 464–469, 1998; Dehaene, S., Naccache, L., Le Clec'H, G., Koechlin, E., Mueller, M., Dehaene-Lambertz, G., et al. Imaging unconscious semantic priming. Nature, 395 , 597–600, 1998; Whalen, P. J., Rauch, S. L., Etcoff, N. L., McInerney, S. C., Lee, M. B., & Jenike, M. A. Masked presentations of emotional facial expressions modulate amygdala activity without explicit knowledge. Journal of Neuroscience, 18 , 411–418, 1998; Marcel, A. J. Conscious and unconscious perception: Experiments on visual masking and word recognition. Cognitive Psychology, 15 , 197–237, 1983]. Masking thus provides a method for identifying language processes that are preattentive and automatic. Functional magnetic resonance imaging used in concert with masking may identify brain regions engaged by these unconscious language processes. In an adaptation design, subjects viewed a continuous stream of masked words and masked nonwords while performing an unrelated detection task, in which they were asked to make a response to a visible colored nonword stimulus (i.e., ampersands in red or blue font). Most trials were masked nonwords and masked words were presented once every 12–15 sec. The task ensured participant engagement, while the masked nonword baseline controlled for perceptual and orthographic processing. Participants were naïve to the purpose of the experiment and testing indicated that they did not consciously perceive either the words or nonwords. Masked words, but not masked nonwords, strongly activated left hemisphere language regions, including Broca's area, the angular gyrus, and the lateral temporal lobe. Differential activation of the posterior corpus callosum was also observed.

Previous studies suggested that auditory change-specific neural responses are attention-independent and reflect central auditory processing. The automaticity of the brain's response to infrequent changes in pitch within a series of auditory tone pips was examined in parallel functional magnetic resonance imaging (fMRI) and event-related potential (ERP) studies. Subjects performed a continuous perceptual-motor visual tracking task at two levels of difficulty while simultaneously hearing a series of task-irrelevant standard tone pips and infrequent pitch-deviant tones. fMRI results revealed that the unattended pitch-deviant tones strongly activated superior temporal and frontal cortical regions. These activations were significantly modulated by the tracking difficulty of the primary task. ERP results revealed that the amplitude of the scalp-negative component evoked by deviant tones (MMN) was attenuated during the more difficult tracking task. Our results demonstrate that the brain's response to task-irrelevant sensory changes is strongly influenced by intermodal attentional demands.

We investigated the necessity of biological motion for activation of the posterior superior temporal sulcus (pSTS) in circumstances in which the rapid approach of the observer to a virtual human induced the observer to make inferences about the characters intentions. Using a virtual reality environment, subjects experienced themselves walking towards a complex scene composed of animate and/or inanimate objects. During “person” trials, the scene contained a virtual human either making a simple gesture such as scratching his face (Study 1) or standing completely still (Study 2). During “object” trials, the scenes contained items such as furniture, a face portrait, and a clock, but not the virtual human. Using functional MRI to measure brain activity, we demonstrated strong activity in the pSTS while the observer approached the social scene, but only when the virtual human was making gestures. This result emphasizes the importance of biological motion in inferring the intentions of others.

An explication of the neural substrates for social perception is an important component in the emerging field of social cognitive neuroscience and is relevant to the field of cognitive neuroscience as a whole. Prior studies from our laboratory have demonstrated that passive viewing of biological motion (Pelphrey, Mitchell, et al., 2003; Puce et al., 1998) activates the posterior superior temporal sulcus (STS) region. Furthermore, recent evidence has shown that the perceived context of observed gaze shifts (Pelphrey, Singerman, et al., 2003; Pelphrey et al., 2004) modulates STS activity. Here, using event-related functional magnetic resonance imaging at 4 T, we investigated brain activity in response to passive viewing of goal- and non-goal- directed reaching-to-grasp movements. Participants viewed an animated character making reaching-to-grasp movements either toward (correct) or away (incorrect) from a blinking dial. Both conditions evoked significant posterior STS activity that was strongly right lateralized. By examining the time course of the blood oxygenation level-dependent response from areas of activation, we observed a functional dissociation. Incorrect trials evoked significantly greater activity in the STS than did correct trials, while an area posterior and inferior to the STS (likely corresponding to the MT/V5 complex) responded equally to correct and incorrect movements. Parietal cortical regions, including the superior parietal lobule and the anterior intraparietal sulcus, also responded equally to correct and incorrect movements, but showed evidence for differential responding based on the hand and arm (left or right) of the animated character used to make the reaching-to-grasp movement. The results of this study further suggest that a region of the right posterior STS is involved in analyzing the intentions of other people's actions and that activity in this region is sensitive to the context of observed biological motions.

We investigated using functional magnetic resonance imaging (fMRI) the neural processes associated with performance of a change-detection task. In this task, two versions of the same picture are presented in alternation, separated by a brief mask interval. Even when the two pictures greatly differ (e.g., as when a building is in different locations), subjects report that identification of the change is difficult and often take 30 or more seconds to identify the change. This phenomenon of “change blindness” provides a powerful and novel paradigm for segregating components of visual attention using fMRI that can otherwise be confounded in short-duration tasks. By using a response-contingent event-related analysis technique, we successfully dissociated brain regions associated with different processing components of a visual change-detection task. Activation in the calcarine cortex was associated with task onset, but did not vary with the duration of visual search. In contrast, the pattern of activation in dorsal and ventral visual areas was temporally associated with the duration of visual search. As such, our results support a distinction between brain regions whose activation is modulated by attentional demands of the visual task (extrastriate cortex) and those that are not affected by it (primary visual cortex). A second network of areas including central sulcus, insular, and inferior frontal cortical areas, along with the thalamus and basal ganglia, showed phasic activation tied to the execution of responses. Finally, parietal and frontal regions showed systematic deactivations during task performance, consistent with previous reports that these regions may be associated with nontask semantic processing. We conclude that detection of change, when transient visual cues are not present, requires activation of extrastriate visual regions and frontal regions responsible for eye movements. These results suggest that studies of change blindness can inform understanding of more general attentional processing.

We conducted two fMRI studies to investigate the sensitivity of delay-period activity to changes in memory load during a delayed-recognition task for faces. In Experiment 1, each trial began with the presentation of a memory array consisting of one, two, or three faces that lasted for 3 sec. A 15-sec delay period followed during which no stimuli were present. The delay interval concluded with a one-face probe to which subjects made a button press response indicating whether this face was part of the memory array. Experiment 2 was similar in design except that the delay period was lengthened to 24 sec, and the memory array consisted of only one or three faces. We hypothesized that memory maintenance processes that spanned the delay interval would be revealed by their sensitivity to memory load. Long delay intervals were employed to temporally dissociate phasic activity engendered by the memory array from sustained activity reflecting maintenance. Regions of interest (ROIs) were defined anatomically for the superior frontal gyri (SFG), middle frontal gyri (MFG), and inferior frontal gyri (IFG), intraparietal sulci (IPS), and fusiform gyri (FFG) on a subject-by-subject basis. The mean time course of activity was determined for all voxels within these regions and for that subset of voxels within each ROI that correlated significantly with an empirically determined reference waveform. In both experiments, memory load significantly influenced activation 6-9 sec following the onset of the memory array with larger amplitude responses for higher load levels. Responses were greatest within MFG, IPS, and FFG. In both experiments, however, these load-sensitive differences declined over successive time intervals and were no longer significant at the end of the delay interval. Although insensitive to our load manipulation, sustained activation was present at the conclusion of the delay interval within MFG and other prefrontal regions. IPS delay activity returned to prestimulus baseline levels prior to the end of the delay period in Experiment 2, but not in Experiment 1. Within FFG, delay activity returned to prestimulus baseline levels prior to the conclusion of the delay interval in both experiments. Thus, while phasic processes engendered by the memory array were strongly affected by memory load, no evidence for load-sensitive delay-spanning maintenance processes was obtained.

The perception of faces is sometimes regarded as a specialized task involving discrete brain regions. In an attempt to identi$ face-specific cortex, we used functional magnetic resonance imaging (fMRI) to measure activation evoked by faces presented in a continuously changing montage of common objects or in a similar montage of nonobjects. Bilateral regions of the posterior fusiform gyrus were activated by faces viewed among nonobjects, but when viewed among objects, faces activated only a focal right fusiform region. To determine whether this focal activation would occur for another category of familiar stimuli, subjects viewed flowers presented among nonobjects and objects. While flowers among nonobjects evoked bilateral fusiform activation, flowers among objects evoked no activation. These results demonstrate that both faces and flowers activate large and partially overlapping regions of inferior extrastriate cortex. A smaller region, located primarily in the right lateral fusiform gyrus, is activated specifically by faces.

Event-related potentials (ERPs) associated with face perception were recorded with scalp electrodes from normal volunteers. Subjects performed a visual target detection task in which they mentally counted the number of occurrences of pictorial stimuli from a designated category such as butterflies. In separate experiments, target stimuli were embedded within a series of other stimuli including unfamiliar human faces and isolated face components, inverted faces, distorted faces, animal faces, and other nonface stimuli. Human faces evoked a negative potential at 172 msec (N170), which was absent from the ERPs elicited by other animate and inanimate nonface stimuli. N170 was largest over the posterior temporal scalp and was larger over the right than the left hemisphere. N170 was delayed when faces were presented upside-down, but its amplitude did not change. When presented in isolation, eyes elicited an N170 that was significantly larger than that elicited by whole faces, while noses and lips elicited small negative ERPs about 50 msec later than N170. Distorted human faces, in which the locations of inner face components were altered, elicited an N170 similar in amplitude to that elicited by normal faces. However, faces of animals, human hands, cars, and items of furniture did not evoke N170. N170 may reflect the operation of a neural mechanism tuned to detect (as opposed to identify) human faces, similar to the “structural encoder” suggested by Bruce and Young (1986). A similar function has been proposed for the face-selective N200 ERP recorded from the middle fusiform and posterior inferior temporal gyri using subdural electrodes in humans (Allison, McCarthy, Nobre, Puce, & Belger, 1994c). However, the differential sensitivity of N170 to eyes in isolation suggests that N170 may reflect the activation of an eye-sensitive region of cortex. The voltage distribution of N170 over the scalp is consistent with a neural generator located in the occipitotemporal sulcus lateral to the fusiform/inferior temporal region that generates N200.

Event-related potentials (ERPs) were recorded from the scalp to investigate the processing of word stimuli. Three tasks were used: (1) a task comparing words that provided an anomalous or normal sentence ending, (2) a word-list task in which different word types were examined, and (3) a word-list task in which semantic priming was examined. ERPs were recorded from a 50-channel montage in an attempt to dissociate overlapping ERP features by their scalp distributions. The focus of these studies was the N400, an ERP previously associated with language processing (Kutas & Hillyard, 1980). The temporal interval typically associated with N400 (250–500 msec) was found to contain overlapping ERP features. Two of these features were common to both sentence and word-list tasks—but one appeared different. Anomalous sentence endings and words with semantic content in lists both showed coincident negative left frontotemporal and midline-anterior ERP foci, peaking at 332 msec for sentences and 316 msec for word lists. The most negative voltage obtained in the sentence task peaked at 386 msec and had a midline-posterior focus. A right frontotemporal focus developed after the midline-posterior focus and outlasted its duration. The most negative voltage for content words in lists was reached at 364 msec. The distribution of this ERP was extensive over the midline and appeared to differ from that observed in the sentence task. Modulation of language-related ERPs by word type and semantic priming was investigated using the word-list tasks, which required category-detection responses. Two novel findings were obtained: (1) The ERP distributions for words serving grammatical function and content words differed substantially in word lists. Even when devoid of any sentence context, function words presented significantly attenuated measures of N400 compared to content words. These findings support hypotheses that suggest a differential processing of content and function words. (2) Semantic priming functionally dissociated two ERP features in the 250–500 msec range. The later and most negative midline ERP feature (peaking at 364 msec) was attenuated by semantic priming. However, the earlier left frontotemporal feature (peaking at 316 msec) was enhanced by semantic priming. The isolation of this novel language-related ERF' that is sensitive to semantic manipulations has important consequences for temporal and mechanistic aspects of theories of language processing.