Although it is generally acknowledged that familiar face recognition is fast, mandatory, and proceeds outside conscious control, it is still unclear whether processes leading to familiar face recognition occur in a linear (i.e., gradual) or a nonlinear (i.e., all-or-none) manner. To test these two alternative accounts, we recorded scalp ERPs while participants indicated whether they recognize as familiar the faces of famous and unfamiliar persons gradually revealed in a descending sequence of frames, from the noisier to the least noisy. This presentation procedure allowed us to characterize the changes in scalp ERP responses occurring prior to and up to overt recognition. Our main finding is that gradual and all-or-none processes are possibly involved during overt recognition of familiar faces. Although the N170 and the N250 face-sensitive responses displayed an abrupt activity change at the moment of overt recognition of famous faces, later ERPs encompassing the N400 and late positive component exhibited an incremental increase in amplitude as the point of recognition approached. In addition, famous faces that were not overtly recognized at one trial before recognition elicited larger ERP potentials than unfamiliar faces, probably reflecting a covert recognition process. Overall, these findings present evidence that recognition of familiar faces implicates spatio-temporally complex neural processes exhibiting differential pattern activity changes as a function of recognition state.
Many of our everyday behaviors, such as recognizing acquaintances in the street, reading a book, appropriately manipulating objects, and so forth, appear to be disconcertingly effortless and automatic. For instance, we cannot look at a face and decide whether or not we recognize it; no matter how hard we try. Face recognition is fast, mandatory, and proceeds outside conscious control (Bruce & Young, 1986). This phenomenon and a number of other observations have been taken as evidence that face recognition may be achieved in an all-or-none fashion. Paradoxically, other everyday situations like the types of slips and errors in recognizing faces and several empirical findings from laboratory experiments (Hay, Young, & Ellis, 1991; Young, Hay, & Ellis, 1985) strongly suggest that several states of face recognition are possible and that the underlying mechanisms would operate in a continuous and graded manner. These two views have been conceptualized in theoretical frameworks of face recognition (Burton, Bruce, & Johnston, 1990; Bruce & Young, 1986) but there is as yet no clear-cut evidence for an all-or-none or a gradual account of the process underlying familiar face recognition.
The functional model put forward by Bruce and Young (1986) has promoted the idea that face familiarity takes place at the level of face-specific representation, referred to as face recognition units (FRUs) that store structural perceptual information of familiar faces. When the visual input from a perceived face matches its stored representation, the corresponding FRU is activated, therefore generating a feeling of familiarity. FRUs are viewed as transmitting a graded signal that is proportional to the degree of resemblance between stored representation and the current percept (Bruce & Young, 1986). FRU activation, in turn, instigates the access to all sorts of identity-specific and semantic information (person–identity nodes, PINs) about the face, and finally, retrieval of his or her name. Moreover, in Burton et al.'s (1990) model, face recognition is supported by thresholded devices located at PINs, the activation of which rapidly triggers retrieval of associated person-specific information. Accordingly, these PINs operate in an “all-or-none” (i.e., binary) fashion (Burton et al., 1990); if their signal strength exceeds a set threshold criterion, then a recognition response is given, and if it is below threshold, the face is judged as unfamiliar.
Based on these two models, face recognition entails two memory processes that might be related but are, by no means, synonymous. The first computes similarity of probe faces to stored traces in long-term memory and generally refers to a diffuse and undifferentiated feeling of familiarity, that is, a general, context-free sense of knowing that a given face has been encountered at some time in the past (i.e., Bruce & Young, 1986). The second generally refers to explicit forms of memory retrieval of specific person identity information (Burton et al., 1990), that is, a context-dependent process associated with the experience of not only knowing that a face is familiar but also remembering some relevant information that allow identification of that face. Accordingly, one can argue that face familiarity is a memory process that supports recognition without being necessarily accompanied by conscious retrieval of detailed memories (Mandler, 1980). Evidence for this argument mainly comes from neuropsychological studies on prosopagnosia, a face recognition impairment caused by selective damage in ventral occipito-temporal cortex and anterior temporal regions (Damasio, Tranel, & Damasio, 1990). Despite the failure in overt or conscious recognition of familiar faces, residual effects of facial familiarity can still be demonstrated in some prosopagnosic patients through a variety of indirect measures that do not depend on explicit recognition (Young, 1994), such as implicit learning of true and untrue face–name associations (Bruyer et al., 1983), semantic priming (Barton, Cherkasova, & Hefter, 2004), eye movement scan paths (Rizzo, Hurtig, & Damasio, 1987), as well as electrophysiological (Renault, Signoret, Debruille, Breton, & Bolgert, 1989) and skin-conductance response measures [SCR] (Tranel & Damasio, 1985). Moreover, the reverse dissociation, that is, explicit recognition without familiarity, has been supported by some evidence in patients with ventromedial frontal damage (see Tranel, Damasio, & Damasio, 1995) and in patients with delusional misidentification syndromes such as Capgras delusion (Ellis, Young, Quayle, & DePauw, 1997; Hirstein & Ramachandran, 1997). Tranel et al. (1995) reported the case of some patients with ventromedial frontal damage who failed to show discriminatory autonomic responses to familiar and unfamiliar faces that were otherwise consciously recognized. Taken collectively, these sets of findings suggest that the activation of face representation is necessary but not sufficient for conscious recognition of familiar faces. However, it is still unclear which mechanisms support overt face recognition and whether processes engaged immediately upon recognizing a familiar face occur in a linear or a nonlinear manner.
In the present study, we sought to further address the question: “what happens when a face rings a bell?” (Hay, Young, & Ellis, 1986), as reflected by scalp-recorded event-related brain potentials (ERPs). Due to their high temporal resolution, ERPs constitute a valuable means to track the temporal sequence of brain events during face processing as they occur, from perception to recognition (Tanaka, Curran, Porterfield, & Collins, 2006; Schweinberger, Pickering, Jentzsch, Burton, & Kaufmann, 2002; Bentin & Deouell, 2000; Eimer, 2000). Findings from several studies have established that the stage of perceptual face processing is indexed by a negative peak in scalp-recorded ERPs appearing at approximately 170 msec poststimulus onset over lateral occipito-temporal recording sites, known as the N170 face-sensitive component (Bentin, Allison, Puce, Perez, & McCarthy, 1996). The demonstration that the N170 is highly sensitive to stimulus manipulations that impede perceptual face processing (Jemel, Schuller, et al., 2003; Rossion et al., 2000; George, Evans, Fiori, Davidoff, & Renault, 1996) and is not sensitive to memory-related and familiarity-related factors (Tanaka et al., 2006; Schweinberger et al., 2002; Bentin & Deouell, 2000; Eimer, 2000) provided support for the claim that this ERP component reflects a fairly early stage of the visual analysis of faces. Other lines of research, however, showed reliable effects of familiarity, repetition priming, and recognition on N170 responses to faces (Jemel, Pisani, Rousselle, Crommelinck, & Bruyer, 2005; Tanskanen, Näsänen, Montez, Päällysaho, & Hari, 2005; Jemel, Pisani, Calabria, Crommelinck, & Bruyer, 2003; Caharel et al., 2002), suggesting that as early as 170 msec after stimulus onset, the brain is individuating previously encoded faces.
Nonetheless, there is now an increasing number of studies showing that the earliest time epoch at which familiar and unfamiliar face ERPs differ occurs circa 200–300 msec after stimulus onset, at the level of an occipito-temporal negative potential known as the N250 (Begleiter, Porjesz, & Wang, 1995); N250 familiarity effect typically takes the form of enhanced negative potentials in response to familiar than to unfamiliar faces (Tanaka et al., 2006; Schweinberger, Pfütze, & Sommer, 1995). This result has led some authors to suggest that this component reflects the activation of preexistent internal representation within the face recognition system (Bruce & Young, 1986). However, studies investigating repetition effects within the time range of the N250 (i.e., N250r for repetition) have cast some doubt on this interpretation. Most notably, larger N250r for repeated as compared to nonrepeated faces have been reported for familiar and unfamiliar faces (Begleiter et al., 1995; Schweinberger et al., 1995), although sometimes with reduced amplitudes for unfamiliar faces (Herzmann, Schweinberger, Sommer, & Jentzsch, 2004). These findings support the general idea that the N250r rather reflects a perceptually specific form of memory that is not restricted to preexistent memory contents (Jemel et al., 2005; Schweinberger et al., 1995).
In addition to studies on the N250, prior work has focused on ERP face familiarity effects during a latency period consistent with the N400 (Bentin & Deouell, 2000; Eimer, 2000; Smith & Halgren, 1987), a negative component that has been classically associated with the activation of semantic information of verbal as well as nonverbal material, namely, objects (Barrett & Rugg, 1990), faces (Jemel, George, Olivares, Fiori, & Renault, 1999; Barrett & Rugg, 1989; Smith & Halgren, 1987), arithmetic facts (Niedeggen & Rösler, 1999), and so forth. Enhanced N400s to famous relative to unfamiliar faces have not only been reported over centro-parietal scalp regions (Bentin & Deouell, 2000; Eimer, 2000) but also over occipito-temporal and frontal regions (Jemel et al., 2005; Smith & Halgren, 1987), consistent with depth electrode recording findings showing different generators of the N400 within the medial-temporal and orbito-frontal regions (Dietl et al., 2005; Halgren, Baudena, Heit, Clarke, & Marinkovic, 1994). Besides, a late positive component (LPC) occurring beyond 500 msec poststimulus was also found to be modulated by face familiarity, showing greater positive potentials in response to familiar than to unfamiliar faces (Bentin & Deouell, 2000; Eimer, 2000). Unlike the N250, the amplitude of the N400 to familiar faces is reduced not only when the same face has been previously presented (i.e., within-domain repetition; Jemel et al., 2005; Jemel, Calabria, Delvenne, Crommelinck, & Bruyer, 2003; Schweinberger et al., 2002; Eimer, 2000; Barrett, Rugg, & Perrett, 1988), but also when it has been preceded by the presentation of its proper name (cross-domain priming; Jemel et al., 2005) or a semantically associated face (associative priming; Schweinberger, 1996; Schweinberger et al., 1995; Barrett & Rugg, 1989). These results suggest that effects taking place within the N400 latency range would reflect the activation of not only face representation but also person-related knowledge and associated semantic information (Schweinberger et al., 1995). Interestingly, N400 differences between familiar and unfamiliar faces have been reported in the absence of a face familiarity-oriented task (Bentin & Deouell, 2000, Experiment 1; Jemel et al., 2005; Jemel, Pisani, et al., 2003; Eimer, 2000) and for faces presented at a very rapid rate (Jemel et al., 2005; Jemel, Pisani, et al., 2003). Instead, effects associated to the LPC are strongly related to the time taken to categorize an eliciting stimulus (McCarthy & Donchin, 1981), categorical decision strategies (e.g., Bentin & Deouell, 2000), as well as the difficulty of discriminating targets from nontargets (e.g., Kutas, McCarthy, & Donchin, 1977). Taken collectively, these findings suggest that the N400, as opposed to the LPC, reflects in part automatic spread of activation within the face representational system, and does not necessarily imply conscious recollection of memory representation or strategical process.
In so far as the N250, the N400, the LPC, and, to some extent, the N170, have been shown to be sensitive to face familiarity, should they also reveal whether graded or discrete patterns of memory processes support changes in face recognition state? Notwithstanding this, the present study was undertaken to investigate whether recognition of famous faces is associated to a gradual or abrupt change in these ERP responses. For this end, we used a variant of the method of ascending limits (Snodgrass & Feenan, 1990), in which the amount of visual information is incrementally increased with each successive image presentation. Such a presentation procedure can be employed not only to slow down the automatic process of familiar face recognition but also to foster the specific spatio-temporal dynamics that allegedly occur during such a process. Gradual presentation methods have been successfully employed in previous ERP and fMRI studies to unravel the brain mechanisms underlying perceptual closure (Doniger et al., 2000), object recognition (Viggiano & Kutas, 1998), and object priming (James, Humphrey, Gati, Menon, & Goodale, 2000). In the present experiment, famous and unfamiliar faces were gradually revealed by parametrically decreasing the level of noise in subsequent image frames as in James et al.'s (2000) study. ERPs elicited by noise image frames associated with the first occurrence of correct recognition responses within familiar face continua (trials of overt recognition) were compared to the ERPs elicited by 1-trial and 2-trial image frames preceding participants' overt recognition. It is worth mentioning, however, that ERP differences between recognition levels, particularly in the time range of early visual ERPs, could possibly reflect a genuine effect of recognition state but also a systematic low-level visual difference between the three consecutive recognition trials. Indeed, ERP averages at recognition would always include trial image frames with less amount of noise than trials preceding recognition. This might result in a gradual increase of early ERP components, specially the N170 (Jemel, Schuller, et al., 2003). A potential control for this caveat is to reconstruct the ERP averages at each recognition level for unfamiliar faces based on the same noise trials that served for familiar face ERPs. A potential drawback with this averaging method is that the ERPs to unfamiliar faces at recognition level would not necessarily reflect the noise trials at which participants were able to reject as familiar the faces of unfamiliar persons. Nonetheless, considering that unfamiliar faces are devoid of any memory representation, it is unlikely that ERP responses to unfamiliar faces would show differences between trials of first correct unfamiliarity decisions and prior noise trials. Moreover, computing the ERP averages for unfamiliar faces in a similar way as for familiar faces allowed us not only to control for potential confounds as outlined earlier but also to assess familiarity effect by contrasting familiar versus unfamiliar face ERPs at each recognition level. Our reasoning was that if familiar faces were recognized in an all-or-none fashion, we would observe an abrupt increase in the amplitude of ERPs at the level of recognition relative to prior trials that did not lead to overt recognition. Conversely, if it is the case that face recognition results from a graded level of activation in corresponding memory representation, then familiarity-related and recognition-level ERP indices would display a gradual change that would reach its maximum at level of recognition.
Sixteen young adults with normal or corrected-to-normal vision were paid for their participation in the EEG experiment. All participants were recruited from Université de Louvain-La-Neuve in Brussels. They were healthy, without any medication, and free from any history of neurological or psychiatric illness. Data from three participants were excluded from further analysis due to excessive noise in the EEG data. The remaining 13 subjects (5 men) were aged between 18 and 31 years (mean = 24 years). Two of them classified themselves as left-handed, and the remainder as right-handed.
Stimuli Preparation and Material
The stimuli consisted of continua of gray-scale photographs of familiar (i.e., well-known celebrities such as politicians, movie stars, musicians, etc.) and unfamiliar faces (i.e., unknown individuals) with decreasing levels of noise. Prior to the EEG experiment, we ran a pilot behavioral study to determine stimulus selection based on the following criteria: (i) celebrities' faces should be recognized by almost all participants; (ii) noise levels should produce a discrete change in subjects' recognition rate along famous face continua; and (iii) image frames of famous faces with the highest levels of noise should be systematically difficult to recognize.
Twenty-one undergraduate students (mean age = 19.14 ± 3 years) from Université de Louvain-La-Neuve received course credits for their participation in the pilot study, and none took part in the EEG study. The stimuli were images of 65 famous and 65 unfamiliar faces taken from a preexistent face database (Jemel, Calabria, et al., 2003; Jemel, Pisani, et al., 2003), all cut out from their clothing and backgrounds and equated in contrast and size so that they were 7 cm long and 6 cm wide. These face images were then subjected to nine equidistant levels of noise using the additive Gaussian noise function available in Adobe Photoshop. An example of nine noise frames of a famous face is shown in Figure 1A. During the experiment, each noise frame of famous and unfamiliar faces was presented for 33 msec on a screen monitor, with an interstimulus interval of 2.05 sec. The stimuli were presented in random order, except that immediate stimulus repetitions were not allowed. Participants were asked to press as quickly as possible a “yes” response key button if they recognize the face in the noise as belonging to a famous person, and a “no” key button if they do not recognize the face as being familiar. According to these instructions, “no” responses would be provided either when a famous face could not be recognized due to the high noise levels, or when the actual face was that of an unfamiliar face. The results of the pilot study show (see Figure 1A): (i) a gradual increase of correct recognition rates as the noise level was parametrically decreased, (ii) and the highest levels of noise did not dramatically prevent familiar face recognition to take place, as subjects' ability to recognize famous faces at the ninth level of noise was relatively good (∼32%). By taking into account the pilot study's results along with the above-mentioned criteria for stimulus selection, 50 out of 65 photographs of famous faces and five out of nine levels of noise (indicated by an arrow in Figure 1A) were selected to create the stimulus set employed in the EEG experiment. We also added a sixth noise level that was noisier than the ninth level of noise used in the pilot study and a pure noise image (see Figure 1B).
Following electrode application, participants sat in a comfortable chair in a dimly lit electrically shielded room, their head being restrained by a chin rest positioned 80 cm in front of a VGA monitor. Famous and unfamiliar face stimuli were presented according to the ascending method of limits (Snodgrass & Feenan, 1990), so that each face was gradually revealed over a sequence of seven noise frames, from the noisiest to the least noisy. A schematic presentation of the task is shown in Figure 1B. Each face continuum lasted for about 17 sec starting with a 1000-msec presentation of a written notice (“New sequence” in French) to inform participants that a new face continuum was about to start. Trials of each face continuum always started with the presentation of a pure noise image frame (Noise level 7) that was followed by the successive presentation of six decreasing levels of noise image frames (from Noise level 6 to Noise level 1) of either a famous or an unfamiliar face. Each noise frame was briefly presented for 33 msec followed by a black screen for 880 to 1100 msec. A fixation cross then appeared on the screen for 1250–1400 msec, prompting a familiarity judgment response. Participants pressed either a “yes” button if they recognized the face as familiar, or a “no” button if they did not recognize the face as familiar. As in the pilot study, nonrecognition responses were given to nonrecognized familiar faces, especially at high noise levels, and to unfamiliar faces. Left and right hand assignments for “yes” and “no” responses were counterbalanced across subjects. After a short practice block, subjects viewed in the first block 50 different sets of familiar face continua interleaved with 25 different sets of unfamiliar face continua. After a 15-min pause, the first block was repeated using the same 50 famous and 25 unfamiliar face continua. The order of presentation of famous and unfamiliar face continua was always randomized for each block.
EEG activity was continuously recorded from 64 tin electrodes mounted in an elastic cap (Quick-Cap International; El Paso, TX; montage inset in Figure 1C) according to the enhanced 10–20 system (Sharbrough et al., 1991). Vertical and horizontal EOGs were monitored respectively via a supra- to suborbital bipolar montage and a right to left canthal bipolar montage. EEG and EOG recordings were amplified (0.1–100 Hz band width, −3 dB points) and digitized at a 500-Hz sampling rate. Recordings were made with a reference to an electrode placed on the tip of the nose, and off-line average re-referenced. Impedances were always kept below 8 kΩ. Continuous EEG activity was off-line windowed for epochs of 1200 msec, including a 200-msec prestimulus baseline and a 1000-msec poststimulus epoch. EEG segments that contained nonblink eye movements, muscle artifacts, and EEG signals exceeding ±50 μV at any recording scalp site were automatically rejected. EEG segments with eye blink artifacts were corrected using the algorithm of Semlitsch, Anderer, Schuster, and Presslich (1986) available in Neuroscan software. Artifact-free EEG segments were then averaged according to condition and behavior as described below.
Data Preparation and ERPs Analysis
Before averaging, the data of each subject from the first and the second experimental blocks were first arranged to exclude all trials (i.e., six noise frame trials) associated with unrecognized famous faces and those associated with misrecognized unfamiliar faces. Continua in which the face of a famous person was initially judged as familiar but subsequently not recognized at the lowest noise level (e.g., Level 1) were also excluded. Second, noise continua of correctly recognized famous faces were then examined individually for each participant to determine the trials at which a correct familiarity response was given the first time (recognition level) within the six noise frames. These trials were extracted and referred to as trials of “recognition.” The last and second-to-the-last trials before recognition were also identified and were referred to as “1-trial” and “2-trials before recognition,” respectively. This was done separately for each subject and for each famous face continuum. Furthermore, famous face continua for which trials of first recognition responses occurred at Noise levels 6 and 5 (approximately 20%) were excluded from ERP averages. This was done because taking into account these noise levels in the ERP averages would have led to an unbalanced number of trials between the three recognition levels (i.e., “recognition” and “1-trial” and “2-trials before recognition”). In fact, for first correct familiarity responses given at Noise levels 6 and 5, 1-trial and 2-trials before recognition corresponded to pure noise images (i.e., Noise level 7) that were never included in the ERP averaging. Taken collectively, the procedures used for data preparation ensured that similar number of trials was averaged for famous faces at each recognition level, and for famous and unfamiliar faces. Indeed, the total number of trials used for famous face averages was approximately 55 trials out of 100, which is approximately similar to the number of trials used for the ERP averaging for unfamiliar faces (∼50 trials).
ERP averages for famous faces were then performed with respect to the first occurrence of correct recognition responses along noise continua (recognition level or rec-0) and with respect to 1-trial (rec-1) and 2-trials before recognition (rec-2). ERP averages for unfamiliar faces were made using the same individual noise trials as those used to derive famous face ERPs at recognition level, at 1-trial before recognition, and at 2-trials before recognition. Thus, the three ERP averages of unfamiliar faces included a similar number of trials and the same levels of noise as famous face ERPs at each recognition level. Statistical analyses of behavioral and ERP amplitude measures were all performed with the multivariate F ratio based on Wilks' lambda in a series of multivariate analyses of variance (MANOVAs) with a p < .05 level of significance. The effects of interest were familiarity and recognition level and all interactions involving these two effects. If significant effects and interactions of interest showed up in overall MANOVAs, additional analyses were conducted to further clarify simple effects.
The analyses of behavioral data were based on participants' accuracy to recognize famous faces at each noise level. First, the percentage of correct familiarity responses was calculated as a function of noise levels. Overall, participants correctly judged as familiar an average of 74.6% (SD = 10.5) of the famous face images used. As expected (Figure 2A), there was a marked effect of noise on participants' correct familiarity judgments [F(5, 8) = 132.09, p < .00001]; the rate of correct familiarity responses improved gradually as level of noise decreased in face images. Second, to determine each participant's recognition threshold, that is, the noise level at which each famous face was first recognized within noise continua, we calculated the proportion of correct familiarity responses given the first time (recognition level) along each familiar face continuum for each noise level. For example, if a famous face was recognized at Noise level 4, familiarity responses to the same face in the subsequent levels of noise (3, 2, and 1) were not counted. This was done separately for each subject and for each experimental block. It should be noted that in this analysis, the percentage of correct familiarity responses was computed with respect to the absolute number of famous faces that was recognized by each participant. As can be seen in Figure 2B, first recognition responses to famous face continua were often given at Noise levels 3 and 4 followed by Noise levels 2, 1, 5, and 6. Finally, we performed signal detection analyses using both hit (correct recognition responses to famous faces) and false alarm rates (incorrect recognition responses to unfamiliar faces) to derive the nonparametric measures of discrimination sensitivity (i.e., A′), and response criterion (i.e., B″; Macmillan & Creelman, 1991; Grier, 1971).1 As illustrated in Figure 2C, A′ scores showed a main effect of noise levels [F(5, 8) = 94.37, p < .0001], confirming the fact that participants were better at discriminating famous faces from unfamiliar ones as the noise level decreased in the face images. The mean scores of B″ plotted in Figure 2D indicate that participants used a relatively conservative criterion, which was merely affected by decreasing levels of noise [F(5, 8) = 4.31, p = .034]. However Bonferroni-corrected paired contrasts did not reveal significant B″ score differences between noise levels (all ps > .1), except between Noise levels 1 and 3 (p = .032).
Familiarity and Recognition-level Effects on P1 and N170
Figure 3 illustrates the effect of familiarity on P1 and N170 peaks separately at each recognition level. The effect of recognition level on P1 and N170 peaks is displayed in Figure 4 separately for famous and unfamiliar faces. Familiarity and recognition-level effects were assessed using mean amplitude measures in a 100–130 msec time window centered on the peak of the occipital P1 (over O1/O2, PO3/PO4 and PO5/PO6, PO7/PO8) and a 180–210 msec time window centered on the peak of the N170 face-sensitive component (over P7/P8, P5/P6 and PO7/PO8, PO5/PO6). Mean amplitude values for P1 and N170 peaks were then entered into a Familiarity (famous, unfamiliar) × Recognition level (rec-0, rec-1, rec-2) × Electrode site (four sites) × Hemisphere (left, right) repeated measures MANOVA. As evident from both figures, neither familiarity effect nor recognition-level effect modulated P1 amplitude. The four-way MANOVA analyses of P1 confirm the above observation. Familiarity [F(1, 12) = 4.2, p > .062] and recognition-level [F(2, 11) < 0.77] effects did not reach significance and none of the interactions involving these two factors were significant (Fs < 3.14, ps > .07).
Reliable amplitude differences were present on the N170 peak. The N170s elicited by famous faces were larger at recognition trials than at 1-trial and 2-trials before recognition (Figure 4). Interestingly, no N170 amplitude difference between the three recognition levels was observed for unfamiliar faces. Furthermore, as illustrated in Figure 3, at 1-trial and 2-trials before recognition, the N170s to famous faces were as large as the N170s to unfamiliar faces. However, at recognition trials, the N170s elicited by famous faces appeared to be larger than those elicited by unfamiliar faces, particularly over the right hemisphere. MANOVA analyses of N170 did not yield any significant main effect of familiarity and recognition level (Fs < 3.23, p > .079), but the interaction between the two factors was significant [F(2, 11) = 11.51, p < .002]. This interaction was not modulated by either electrode site or hemisphere, or both (Fs < 1.3, p > .3).
Follow-up analysis of familiarity and recognition level, performed using Bonferroni-corrected paired contrasts, indicated that although the N170s elicited by unfamiliar faces did not show significant amplitude differences between the three recognition levels (p > .9), the N170s elicited by famous faces were more negative at recognition trials than at both 1-trial (p < .001) and 2-trials before recognition (p < .042); no N170 amplitude difference between the two latter recognition levels was found (p > .9). Statistical results showed that familiarity effect did not modulate the amplitude of the N170s at recognition level (p = .069), at 1-trial (p > .4), and at 2-trials before recognition (p = .082). Although these results indicate that the N170s elicited by famous faces were as large as the N170s elicited by unfamiliar faces, regardless of recognition level, there is some evidence for larger N170s to famous as compared to unfamiliar faces over the right hemisphere, for recognition trials only (see Figure 3). Familiarity effect at recognition level was then assessed over individual electrode sites using paired-sample t tests. N170 amplitudes were significantly larger in response to famous than to unfamiliar faces over electrode PO8 [t(12) = 2.32, p = .039]. These differences were marginally significant over electrode PO6 [t(12) = 2.11, p = .057] and failed to reach significance level for the remaining electrodes (p > .1).
ERP Results within the N250 Time Epoch
As shown in Figure 5A, the N250 within the 240–300 msec time epoch displayed an occipito-temporal distribution and was more negative in response to famous than unfamiliar faces, more particularly at recognition trials. Visual inspection of this figure also suggests that the N250 in response to famous faces was smaller at 1-trial and 2-trials before recognition than at recognition trials. These effects were assessed using mean amplitude measurements within the N250 time range (240–300 msec) at five occipito-temporal and temporal scalp sites over the left (P7, PO7, TP7, PO5, and P5) and right (P8, PO8, TP8, PO6, and P6) hemispheres. The mean amplitude values of these measures are plotted in Figure 5B. A Familiarity × Recognition level × Electrode × Hemisphere MANOVA resulted in significant main effects of familiarity [F(1, 12) = 5.68, p = .035] and recognition level [F(2, 11) = 4.15, p = .045] and a significant Familiarity × Recognition-level interaction [F(2, 11) = 14.17, p < .001]. All other effects and interactions involving topographical factors (electrode and/or hemisphere) failed to reach significance level (all Fs < 3.8, p > .08).
Follow-up analyses of Familiarity × Recognition-level interaction indicated that the N250 was larger in response to famous than unfamiliar faces at recognition level (p < .001) and at 1-trial before recognition (p = .05). No familiarity effect was found at 2-trials before recognition (p > .6). Recognition-level effects evaluated by means of Bonferroni-corrected paired contrasts indicated that famous faces elicited more negative N250 at recognition trials than at both 1-trial (p < .024) and 2-trials (p < .005) before recognition; no N250 amplitude difference between 1-trial and 2-trials before recognition was found (p > .5). As expected, no significant N250 amplitude differences between recognition levels were found for unfamiliar faces (all p > .1).
Results of Late ERP Responses (from 300 to 900 msec)
Late ERP responses to famous and unfamiliar faces elicited at each recognition level are displayed in Figure 6A as topographical maps of mean voltage ERPs in three consecutive 200-msec time windows, starting from 300 msec and ending at 900 msec. These maps show three spatially and temporally different ERP features that were, overall, most prominent in response to famous faces at overt recognition trials: From 300 to 700 msec, scalp ERPs were characterized by a negative-going component distributed over bilateral occipito-temporal scalp sites (pos-N400) and by an extended positivity over centro-parietal sites (known as an LPC). At the 500–700 msec time window, both components tended to decrease in amplitude while a frontally distributed negativity became prominent, which lasted until 900 msec (ant-N400).
The analyses of late negative potentials (pos- and ant-N400) were conducted on mean amplitude measures of 200 msec wide range (300–500 msec, 500–700 msec, 700–900 msec) at averaged groups of anterior (left at FP1, AF7, AF3, F7, F5, and right at FP2, AF8, AF4, F8, F6) and posterior scalp sites (left at O1, PO7, PO5, P7, P5, and right at O2, PO8, PO6, P8, P6). The same time windows were used to measure mean amplitudes of the LPC over averaged groups of midline centro-parietal scalp sites (Cz, CPz, Pz, CP1, and CP2). Grand-average ERP waveforms within each of the five analyzed regions (bilateral anterior and posterior regions, central regions) are plotted in Figure 7A and B, displaying respectively familiarity effect at each recognition level and recognition-level effect for famous and unfamiliar faces. Mean amplitude values of each ERP component elicited by famous and unfamiliar faces at every recognition level are plotted in Figure 6B. The statistical significance of familiarity and recognition-level effects and the topographical profile of these effects (anterior–posterior and hemisphere) were assessed using a Familiarity (famous, unfamiliar) × Recognition level (rec-0, rec-1, rec-2) × Caudality (anterior, posterior regions) × Hemisphere (left, right) repeated measures MANOVAs applied separately to each time window. More fine-grained follow-up ANOVAs were conducted when justified by significant interactions.
Pos-N400 and Ant-N400
For the 300–500 msec time epoch, MANOVA yielded significant main effects of familiarity [F(1, 12) = 16.84, p < .001] and recognition level [F(2, 11) = 5.58, p < .025], and a Familiarity × Recognition-level interaction [F(2, 11) = 14.05, p < .001] that was further modulated by a higher-level interaction involving caudality factor [F(2, 11) = 3.99, p = .05]. Follow-up analyses confirmed that familiarity and recognition-level effects modulated the amplitude of the pos-N400. Neither familiarity effect nor recognition-level effect was significant at the level of the ant-N400 (p > .07). Bonferroni-corrected paired contrasts indicated that pos-N400 was more negative in response to famous than unfamiliar faces, and the difference was significant at recognition trials (p < .0001) and also at 1-trial before recognition (p < .001). The evaluation of recognition-level effect as a function of familiarity indicated that although there was no significant amplitude differences between recognition levels for unfamiliar faces (p > .6), pos-N400 elicited by famous faces was larger at recognition level than at 2-trials before recognition (p < .035). Pos-N400 in response to famous faces at 1-trial before recognition was not significantly different from that in response to famous faces at trials of overt recognition and at 2-trials before recognition (p > .09).
Significant main effects of familiarity and recognition level and interactions between the two effects were also seen in the 500–700 msec time epoch (all Fs > 6.41, p < .014). As shown in Figure 5, familiarity effect was the largest at recognition trials (p < .001), slightly smaller but still reliable at 1-trial before recognition (p < .035), and absent at 2-trials before recognition (p > .3). Familiarity × Recognition-level interaction also indicated a graded enhancement of negative-going potentials as a function of recognition level of famous faces. The amplitude of the N400 to famous faces was smaller at 2-trials before recognition than at 1-trial before recognition (p < .001), that was, in turn, smaller than that elicited at recognition trials (p < .02). All effects and interactions involving topographical factors, namely, hemisphere and caudality, were nonsignificant (F < 1.82, p > .2), suggesting that both pos- and ant-N400 were similarly modulated by familiarity and recognition level. Furthermore, inspection of Figure 6B suggests that recognition level differentially affected pos- and ant-N400 in response to familiar faces. Post hoc analyses revealed that the above-described graded effect of recognition level was mainly driven by the ant-N400. In the 700–900 msec time epoch, the main effect of familiarity only was significant [F(1, 12) = 9.77, p < .009]. The interaction between recognition level and familiarity did not reach significance [F(1, 12) = 3.68, p = .06], and all other interactions involving these two effects were not significant (all p > .1).
Midline centro-parietal positive potentials (LPC)
The global MANOVAs on mean amplitude measures of the LPC yielded significant main effects of familiarity from 300 to 900 msec (all Fs > 9.27, p < .01), and recognition level from 300 to 700 msec (all Fs > 8.88, p < .005). Familiarity × Recognition-level interaction was significant from 300 to 900 msec (all Fs > 5.48, p < .022). This interaction was further assessed in each 200-msec time window using Bonferroni-corrected paired contrasts to evaluate familiarity effect as a function of recognition level and recognition-level effect as a function of familiarity of faces. The analyses revealed a gradual effect of familiarity from 300 to 900 msec: The LPC was significantly larger in response to famous than unfamiliar faces at recognition trials (all p < .002) and also at 1-trial before recognition (all p < .025). However, there was no LPC familiarity effect at 2-trials before recognition (all p > .5). There were also significant effects of recognition level for famous faces from 300 to 900 msec with larger LPC responses at recognition trials than at both 1-trial and 2-trials before recognition (all p < .006). For unfamiliar faces, there were no amplitude differences between recognition levels (p > .08), except for the 300–500 msec time window that showed larger LPC at 2-trials before recognition than at 1-trial before recognition (p < .02).
The main goal of the present study was to track the spatio-temporal dynamics of electrical brain activity contingent upon overt recognition of famous faces. More specifically, we were interested in determining whether amplitude changes of specific electrophysiological markers concurrent with the face recognition process show an all-or-none or a graded effect. To test this hypothesis, we employed a gradual presentation procedure in which famous and unfamiliar faces were revealed by progressively decreasing the amount of noise in successive image frames. This procedure allowed us to follow, step by step, the neural brain events leading to overt recognition of familiar faces and to characterize in more details the changes in scalp ERP responses occurring prior to and up to overt recognition. Our main finding is that linear (e.g., gradual) and nonlinear (e.g., all-or-none) processes are possibly involved during overt recognition of familiar faces. We show that although early and mid-latency ERP face-sensitive responses encompassing the N170 and the N250 displayed an abrupt activity change at the moment of overt recognition of famous faces, late latency ERPs (LPC, pos- and ant-N400s) exhibited an incremental increase in amplitude as the point of recognition approached. Interestingly, we found that famous faces at 1-trial before recognition elicited larger late ERPs than they did at 2-trials before recognition and than unfamiliar faces. This result would likely suggest that despite not being overtly recognized, famous faces at 1-trial before recognition were associated with a weak activation of stored face representation and identity knowledge about famous faces. Finally, as expected, unfamiliar faces that are devoid of memory representation did not show any effect of recognition level on early (N170), mid (N250), and late latency ERP components.
Overall, the differential pattern of ERP amplitude changes revealed in the present study is consistent with findings from two neuroimaging studies that used morph continua to directly measure brain activity changes in face processing cortical areas during gradual transitions from a familiarized face to an unfamiliar face (Rossion, Schiltz, Robaye, Pirenne, & Crommelinck, 2001), and from one famous face to another (Rotshtein, Henson, Treves, Driver, & Dolan, 2005). Their findings suggest that some brain areas behave in an all-or-none fashion such as inferior-occipital and fusiform gyri (Rotshtein et al., 2005; Rossion et al., 2001), whereas others, such as the anterior temporal regions, are likely to conform to the gradual activation hypothesis (Rotshtein et al., 2005). Previous PET and fMRI studies have established that the fusiform gyrus can be regarded as the neural substrate underlying perceptual categorization at a more specific level (Gauthier, Tarr, Anderson, Skudlarski, & Gore, 1999) and by extension as the storehouse of face recognition units (Rotshtein et al., 2005; Henson et al., 2003). Furthermore, brain activations in more anterior regions along the temporal pole, namely, anterior middle temporal gyrus, have been commonly associated to access person-specific identity information (Leveroni et al., 2000; Gorno-Tempini et al., 1998; Kapur, Friston, Young, Frith, & Frackowiak, 1995; Sergent, Ohta, & MacDonald, 1992). Taken together, Rotshtein et al.'s (2005) and Rossion et al.'s (2001) results and our present results would likely suggest that recognition of familiar faces is indexed by a complex spatio-temporal dynamics of neurofunctional processes, some of which are activated in an all-or-none fashion, that is, FRU activation as indexed by fusiform activity in functional brain imaging studies and N250 ERP responses, whereas others are achieved in a gradual manner, that is, retrieval of PINs as reflected by the activation pattern of anterior temporal regions and late ERP responses.
Contrary to P1 that was affected by neither familiarity nor recognition level, the N170 elicited by famous faces was significantly larger at overt recognition trials than at trials preceding overt recognition. No N170 amplitude difference was found between 1-trial and 2-trials before recognition of the same famous faces. This pattern of N170 amplitude modulation would likely suggest an abrupt all-or-none change in face recognition state.2 Despite the accumulating evidence demonstrating that the N170 is permeable to high-level cognitive factors (Jemel, Pisani, et al., 2003), there is still a strong belief in much of the ERP literature that this electrophysiological component indexes the early stages of perceptual processing of faces (Bentin & Deouell, 2000; Eimer, 2000) that are insulated from top–down information flow of high-level cognitive processes, such as attention, memory, context, and so forth (Pylyshyn, 1999). There is now a growing body of evidence supporting the claim that the N170 is sensitive to both bottom–up and top–down processes. On the one hand, its amplitude is affected by the perceptual properties of the face input, such as face rotation in the picture plane (Jacques & Rossion, 2007; Rossion et al., 2000), jumbling the internal features of faces (George et al., 1996), contrast reversal (Itier & Taylor, 2002), and adding noise to faces (Jemel, Schuller, et al., 2003). On the other hand, cognitive factors were also reported to modulate N170 amplitudes, such as perceptual and identity priming (Jemel et al., 2005; Jemel, Pisani, et al., 2003), contextual priming (Bentin, Sagiv, Mecklinger, Friederici, & von Cramon, 2002), and perceptual tuning toward diagnostic information as a function of task demands (Joyce, Schyns, Gosselin, Cottrell, & Rossion, 2006; Goffaux, Jemel, Jacques, & Schyns, 2003). More relevant to the current study are two previous findings showing that the amplitude of the magnetic N170 correlates with successful recognition of individual faces (Tanskanen et al., 2005; Liu, Harris, & Kanwisher, 2002), suggesting that the N170 reflects fine perceptual mechanisms necessary for face individuation. We concur with this conclusion and further propose that previous and current N170 findings reveal two qualitatively different momentums of perceptual processing of faces. On the one side, the N170 could underlie a hypothesis-free perceptual processing of faces, regardless of their familiarity, relevance, or emotional valence. This mechanism may be favored in much of ERP studies in which the presentation of familiar and unfamiliar faces is randomized within the experimental blocks. In this context, the presentation of a face belonging to either a familiar or an unfamiliar person triggers the bottom–up visual routines that need to be performed to access the face recognition system. On the other side, the N170 may also reflect a hypothesis-driven process that encloses some a priori knowledge about the perceptual appearance of known faces. Such a process is needed most under challenging visual conditions, when the information conveyed by face inputs cannot be resolved in a purely bottom–up manner (Jemel, Pisani, et al., 2003). The latter interpretation builds on our earlier findings, showing that preactivation of face representation by a face prime or a printed name prime of a famous person modulated in a top–down fashion the N170 to a degraded, two-tone black-and-white image of the same famous face (Jemel et al., 2005; Jemel, Pisani, et al., 2003). In the current study, the evidence for the preactivation of face representation before the advent of overt recognition is readily supported by our late ERP data showing reliable familiarity and recognition effects at 1-trial before recognition. These findings suggest that some guesses about the most likely interpretation of the input image (i.e., the identity of the face) have been generated, that were, in turn, retro-injected in a top–down fashion to guide perceptual processing of a subsequently presented noise image frame. This interpretation is in line with the notion that information conveyed by top–down processes through feedback projections might be used to test for the presence of specific patterns in the incoming signals and to tune the activity of visual areas, providing them with information about background knowledge, context, or expectations—notions that are supposedly extracted from higher cortical regions (Bar, 2003).
Similar to the N170, the N250 component displayed an all-or-none amplitude increase at the moment of overt recognition of famous faces. Although unfamiliar faces did not show any N250 amplitude variations as a function of recognition level, famous faces elicited a larger N250 at overt recognition than at 1-trial and 2-trials before recognition. The current findings are consistent with a variety of evidence linking this ERP component to the activation of perceptual memory traces pertaining to individual familiar faces and previously encoded faces (Schweinberger et al., 1995, 2002), and further elucidate the critical role of this processing stage in the face recognition process. More specifically, the finding that familiarity effect was only observed at recognition level strongly supports the idea that the N250 is tied to memory processes that give rise to explicit recognition of familiar faces. It is possible that the N250 reflects a binary process, which indicates whether the perceptual input matches or not face representation stored in memory. Recent studies have also shown that it also reflects the building-up process of newly acquired face representations (Kaufmann, Schweinberger, & Burton, 2009; Tanaka et al., 2006). Kaufmann et al. (2009) recorded N250 responses to repeated presentation of newly learned faces in four different blocks. Their results showed that the magnitude of the N250 amplitude increase across repetition blocks was the largest between the first and second presentation of learned faces, suggesting that the building up of face representation occurred somehow in an all-or-none fashion. These findings as well as ours suggest that the processes underlying the N250 are in either an active or a resting state. By contrast, the pattern of N250 familiarity effect across recognition levels is compatible with the gradual activation account. N250 was significantly more negative in response to famous than unfamiliar faces both at recognition level and at 1-trial before recognition. No N250 amplitude difference between famous and unfamiliar faces was found at 2-trials before recognition.
The analyses of late ERP responses revealed that overt recognition was not associated with an abrupt qualitative change but rather with a gradual increase in electrical brain activity. Negative ERPs corresponding to the N400-like component were larger in response to famous faces than to unfamiliar faces (familiarity effect), and at trials with overt recognition responses than at trials preceding recognition of famous faces (recognition-level effect). Interestingly, these familiarity- and recognition-related effects on late negative ERPs were the largest at overt recognition, smaller but yet present at 1-trial before recognition, and absent at 2-trials before recognition. Contrary to previous reports (Bentin & Deouell, 2000; Eimer, 2000; Smith & Halgren, 1987), the late negative potentials elicited in our study displayed a bilateral distribution over occipito-temporal (pos-N400) and anterior scalp regions (ant-N400). Effects of familiarity and recognition were first observed at the level of the pos-N400 from 300 to 700 msec and then involved ant-N400 from 500 to 900 msec. The involvement of both posterior and anterior scalp regions during recognition of famous faces has been previously evidenced in some studies (Jemel et al., 1999, 2005; Begleiter et al., 1995). Although the present study cannot directly relate the pos- and ant-N400s to specific cognitive components of the face recognition system, it is possible that the pos-N400 reflects the spread of activation within the face representation system, whereas the ant-N400 would likely index the access to semantic knowledge related to familiar materials. Consistent with this interpretation, an ERP old/new experiment investigating recognition memory of pre-experimentally unknown faces (Paller, Gonsalves, Grabowecky, Bozic, & Yamada, 2000) showed that faces associated with biographical information during the study phase produced larger positive potentials than new faces (i.e., old–new ERP effect) over posterior and anterior scalp regions, whereas studied faces without any semantic information gave rise to ERP old/new effects over posterior scalp regions only. These results suggest that retrieval of visual face information involves the activation of occipito-temporal regions, and probably, the fusiform gyrus (Rotshtein et al., 2005; Henson et al., 2003), whereas retrieval of person-specific semantic information is likely to involve anterior brain regions, as already suggested by neuroimaging studies (Leveroni et al., 2000; Gorno Tempini et al., 1998). Furthermore, it is worth noting that recognizing a familiar face in everyday situation implies access to pre-experimentally existing memory traces, a process that inarguably goes beyond recognition of prior occurrence of a face as investigated in ERP old/new experiments (Paller et al., 2000).
The fact that N400s were more negative in response to famous than unfamiliar faces at 1-trial prior overt recognition likely suggests that, at some point, participants had covertly recognized the famous face, even though their behavioral response was telling us the opposite statement. In this respect, we interpret the forgoing results as a sign of covert recognition. Covert face recognition has been extensively studied in prosopagnosic patients (see, for a review, Young, 1994) but has also been evidenced in neurologically intact participants (Jenkins, Burton, & Ellis, 2002; Morrison, Bruce, & Burton, 2000; Ellis, Young, & Koenken, 1993; Wallace & Farah, 1992). It has been shown, for example, that subliminally presented familiar faces that were not overtly recognized do, nevertheless, give rise to greater SCR responses as compared to unknown faces (Ellis et al., 1993) and to repetition priming effects (Morrison et al., 2000). According to some theoretical accounts (Farah, O'Reilly, & Vecera, 1993), covert face recognition in the absence of conscious awareness can be simulated in healthy subjects by providing to an intact recognition system an impoverished visual input. Degrading the representation underlying recognition per se can lead to impaired overt recognition but would allow the occurrence of covert face recognition (for a similar view, see Burton, Young, Bruce, Johnston, & Ellis, 1991).
Graded effects of familiarity on centro-parietal positive potentials overlapped in time with those observed for pos- and ant-N400s, suggesting that processes underlying LPC and N400s took place in parallel and perhaps in an interactive manner. Furthermore, an LPC increase with recognition level of famous faces was not gradual as no LPC amplitude difference was found between 1-trial and 2-trials before recognition. Consistent with early works (e.g., Kutas et al., 1977), this LPC can be linked to the process of decision-making and would correspond to the stage at which information provided by face recognition system is evaluated. It should, however, be mentioned that the only recognition-level effect found for unfamiliar faces occurred at the level of the LPC within the 300–500 msec time window. We found that unfamiliar faces at 2-trials before recognition elicited larger positive-going potentials than at 1-trial before recognition. In addition, we also found that famous and unfamiliar faces at 2-trials before recognition elicited similar LPC amplitudes, which suggests that participants were at a similar state of uncertainty regarding the identity of faces.
In closing, the present results provide a detailed description of the electrophysiological indices underlying the access to face representation and illustrate the complex interplay between low- and higher-order cognitive processes during familiar face recognition. The cognitive processes underlying recognition of familiar faces involve concurrently different spatio-temporal ERP components, which showed differential pattern activity changes as a function of recognition state. These results complement previous neuroimaging studies by providing precise temporal information of the sequence of brain events underpinning the process of familiar face recognition.
B. J. is supported by a research scholarship from the Fonds de la recherche en santé du Quebec (FRSQ). Funding for this study was supplied by a postdoctoral FSR 2000 prize of the Catholic University of Louvain awarded to B. J.
Reprint requests should be sent to Boutheina Jemel, Research Laboratory in Neuroscience and Cognitive Electrophysiology, Hôpital Rivière-des-Prairies, 7070 Boulevard Perras, Montréal, QC, Canada H1E 1A4, or via e-mail: Boutheina.Jemel@umontreal.ca.
Values of A′ can vary between 0 and 1, with higher values indicating greater discriminability and a value of 0.5 indicating chance performances. Values of B″ can vary from 1 to −1, with negative values corresponding to a more liberal basis for responding (a tendency to report “familiarity judgments” frequently, thereby yielding to a very high hit rate but a high false alarm rate) and positive values corresponding to a more conservative basis for responding (a tendency to report “familiarity judgments” only when stimulus strength was high, thereby yielding to a low hit rate but almost no false alarms). B″ with a 0-value represents a neutral bias.
As discussed briefly in the Introduction, it is worth considering a potential methodological caveat that might have led to the current N170 result. In fact, the ERP averages corresponding to trials of overt recognition included systematically image frames with lower noise levels than trials of prior levels of recognition. Considering this caveat, it remains possible that enhanced N170 amplitude merely mirrors the effect of decreased noise levels in face images at recognition trials (Jemel, Schuller, et al., 2003). Nevertheless, we have two reasons to disclose such explanation. First, although the same number of trials and the same image noise frames served to make the ERPs for famous and unfamiliar faces, we did not find N170 amplitude differences between recognition levels for unfamiliar faces. Second, although the level of noise was incrementally decreased from 2-trials before recognition until recognition trials, N170 amplitude did not yield a systematic incremental increase in amplitude as the moment of recognition approached. We thus conclude that these arguments provide further support for a genuine effect of recognition on the N170 component.