Abstract

In our everyday life, we continuously get to know people, dominantly through their faces. Several neuroscientific experiments showed that familiarization changes the behavioral processing and underlying neural representation of faces of others. Here, we propose a model of the process of how we actually get to know someone. First, the purely visual familiarization of unfamiliar faces occurs. Second, the accumulation of associated, nonsensory information refines person representation, and finally, one reaches a stage where the effortless identification of very well-known persons occurs. We offer here an overview of neuroimaging studies, first evaluating how and in what ways the processing of unfamiliar and familiar faces differs and, second, by analyzing the fMRI adaptation and multivariate pattern analysis results we estimate where identity-specific representation is found in the brain. The available neuroimaging data suggest that different aspects of the information emerge gradually as one gets more and more familiar with a person within the same network. We propose a novel model of familiarity and identity processing, where the differential activation of long-term memory and emotion processing areas is essential for correct identification.

INTRODUCTION

The identification of the surrounding objects is an important ability of any living being. For social animals, such as the Homo sapiens, the identification of conspecific individuals is unarguably one of the most frequently accomplished and most important tasks in human life, which has obvious evolutionary benefits. In our everyday life, we continuously and effortlessly detect, discriminate, categorize at different levels, and, finally, recognize and identify other human beings. We do so by evaluating simultaneously many different static and dynamic sensory features, such as body shape, movement, voice, smell, and, above all, the face of other individuals. Person identification, therefore, is a multimodal process (Blank, Wieland, & von Kriegstein, 2014), involving various sensory but also semantic and contextual information as well. Still, most of our knowledge about person identification comes from facial identification, and correspondingly, the most influential models are also built on cognitive models of face identification. Therefore, in the following pages, we largely concentrate on the process of face-based person identification.

Determining the facial identity of a person is computationally a very challenging endeavor. This is because of the facts that (1) every face shares the very same features, which is why identifying the faces of different individuals requires special processing mechanisms (for a review, see Peterson & Rhodes, 2005), and (2) the facial features of a given individual show an enormous variability across time and space because of changes in illumination, viewpoint, facial expressions, hairstyles, makeup, and age, making the task of grouping the faces of the same individual together very difficult (Andrews, Jenkins, Cursiter, & Burton, 2015; Jenkins, White, van Montfort, & Burton, 2011).

Reflecting these difficulties, an extensive network of subcortical and cortical areas has been allocated to face processing in the human brain (Figure 1; for reviews, see Rapcsak, 2019; Duchaine & Yovel, 2015; Gobbini & Haxby, 2007; Calder & Young, 2005; Haxby, Hoffman, & Gobbini, 2000). In this review, we modify the classification of the influential model of familiar face processing (Gobbini & Haxby, 2007), which describes recognition as the result of the interaction of a core network and an extended system.

Figure 1. 

The neural network involved in face identity processing. Yellow: areas of the core face processing network; blue: semantic processing; yellow: recognition memory; green: areas involved in social aspects, personality traits, attitudes, and theory of mind; red: emotional processing. AMY = amygdala.

Figure 1. 

The neural network involved in face identity processing. Yellow: areas of the core face processing network; blue: semantic processing; yellow: recognition memory; green: areas involved in social aspects, personality traits, attitudes, and theory of mind; red: emotional processing. AMY = amygdala.

Currently, it is considered that occipital, fusiform, and superior temporal areas are parts of the core face processing system. According to most face processing theories, the core network is largely responsible for encoding visual appearances and is activated by any face-like stimulus. The face-sensitive visual areas of the inferior occipital gyrus (occipital face area [OFA]) are thought to be the gateway of early, view-specific face processing steps (Pitcher, Walsh, & Duchaine, 2011; but for a different conclusion, see Ambrus, Amado, Krohn, & Kovács, 2019; Ambrus, Dotzer, Schweinberger, & Kovács, 2017; Ambrus, Windel, Burton, & Kovács, 2017; Rossion, 2014). The fusiform face area (FFA) seems to have a higher-order, more image-invariant face processing and is currently allocated to an intermediate level in face identification. The STS is involved in the encoding of dynamic visual information such as head motion (Duchaine & Yovel, 2015; Collins & Olson, 2014; Axelrod & Yovel, 2013; Chan & Downing, 2011; Rajimehr, Young, & Tootell, 2009; but see Yovel & O'Toole, 2016, for the dynamic person recognition hub interpretation of the STS).

The extended system, on the other hand, encodes person knowledge about familiar faces and their affective aspects. It is composed of the anterior temporal lobe (ATL; Rajimehr et al., 2009) being situated in between the processing of person-related semantic knowledge (Lambon Ralph, 2014) and identity memory (Collins & Olson, 2014; Lambon Ralph, 2014) and where view-independent facial and semantic identity-specific information are encoded (Duchaine & Yovel, 2015). The relatively late discovered face-sensitive area of the frontal cortex, the inferior frontal face area (iFFA; Collins & Olson, 2014; Axelrod & Yovel, 2013; Chan & Downing, 2011), is associated with the STS, and it is part of a dorsal face-selective pathway (Duchaine & Yovel, 2015) being also responsible for dynamic face properties and eye-gaze information.

To the extended system belongs the “theory of mind” network (Frith & Frith, 1999), such as the medial pFC (MPFC) as well as areas around the TPJ and inferior parietal lobule (IPL). It potentially encodes personality traits, attitudes, and mental states of others. Another part of the extended network is the precuneus/posterior cingulate cortex (PC/PCC) and medial temporal lobe (MTL) structures (such as the entorhinal and parahippocampal cortex and the hippocampus), and these are related to person-specific long-term episodic and recognition memories. Finally, the system, responsible for affective processing and emotional responses (amygdala, insula, and the striatal reward system), is also considered a separate part of the extended system (for a review, see Ramon & Gobbini, 2017).

Despite extensive research, which tested the functional role of the face processing network, we have relatively little knowledge about the ways in which identity representations are created and maintained in our brain. In this review, we summarize the process of how we actually get to know someone, with special attention to the ways in which we acquire the representation of newly encountered faces. We start with an overall description of the behavioral process of how initially unfamiliar faces gradually become the basis of a stable and robust representation of person identification. We then critically evaluate and localize the different stages of familiarization and identification within the face network, based on the accumulated univariate and multivariate fMRI data.1 Finally, we propose an updated framework-level model (Guest & Martin, 2020) of how face-based person identification is performed by the human brain and discuss open questions. The major aims of the current summary are (1) to revise the currently available models; (2) to integrate the newer, spatially more specific fMRI data with that of the older, univariate studies; and (3) to locate the directions of future studies.

THE PROCESS OF GETTING TO KNOW SOMEONE

Imagine the following situation. You are commuting every morning the same time with the same train to your workplace. After a while, you start to realize that other people around you do the same and you recognize a few faces as familiar. A blond man always reads his newspaper, or a brunette woman always wears colorful scarfs. Once you initiate a quick chat with that blond man, you learn that he lives in the same city and has the same profession as you. One evening, as your train is canceled, you have to wait another hour at the railway station. You decide to invite the blond man for a beer to the corner pub. He is called Jules and has two daughters, the same age as your own daughters. You like the conversation with the man, and you decide to invite him over to your house next week for a dinner with his family. From this point onward, you slowly develop a friendship with him, and in a year or two, he becomes a very good friend of yours. During this process, an originally unfamiliar person (and his face) first becomes somewhat familiar and is eventually very well known to you. So, we are curious to ask: What are the steps of this cognitive process? What happens in the human brain during this time? How does the processing within the face processing system change by time? How and where does the brain develop eventually the representation and abstract concept of Jules? And finally, how can we test this whole process and its various steps in the laboratory?

In cognitive neurosciences, one approaches these questions from two different angles. First, recognition memory, a type of declarative memory, refers to the capacity of the observer to decide if the encountered item/object/person is familiar or not. Recognition memory is widely considered as consisting of two functionally and anatomically different components: familiarity and recollection (Rugg & Yonelinas, 2003). Familiarity is based on the vague feeling of “knowing” that one has met the person before, without necessarily recalling any specific episodic, contextual, or associative knowledge. Recollection, on the other hand, helps us in identifying details of an encountered episode as well as provides us with additional contextual and associated (nonsensory, semantic) information about the remembered person. Although the first process is assumed to depend largely on anterior temporal and perirhinal cortical functions, it is widely agreed that recollection relies on MTL structures, such as the hippocampus, as well as on prefrontal and parietal areas (for a review, see Brown & Banks, 2015; but see Merkow, Burke, & Kahana, 2015, for a different conclusion regarding the role of the hippocampus). Second, facial (or person) identity recognition has been studied intensively in the past by neuroimaging and electrophysiological methods in the healthy and brain-lesioned brains. Three major classes of such studies exist. First, several authors measured and compared the absolute magnitude of the average, univariate neural activations to unfamiliar faces to that of faces with varying degrees of familiarity. Second, fMRI adaptation (fMRIa) experiments make use of the fact that the neural signal is reduced for repeated images (Grill-Spector & Malach, 2001; Grill-Spector et al., 1999). The manipulation of certain stimulus properties, such as the identity of a person, and the measure of the release of the signal from adaptation can reveal if an area is sensitive to that particular feature or not. Third, the recent development of multivariate pattern analysis (MVPA) techniques (Haxby, Connolly, & Guntupalli, 2014; Haxby et al., 2001) enabled the detection of distributed stimulus selectivity maps of facial identity within an area.

Unarguably, the identification of a person involves both the processing of visual and nonvisual sensory information as well as the matching of this input to previously existing person knowledge, which is stored in the declarative memory. In Figure 2, we illustrate the process of how the identity-specific information is becoming more and more elaborate at various stages of familiarity. The bottom–up (sensory) versus top–down (associative, predictive, mnemonic) duality of the identification process is illustrated by the arrows on the two sides of the faces. First, a face can be judged as familiar on the basis of repeated visual presentations. At this level, typically, no additional information can be recalled about the person, reflecting the vague feeling of merely “knowing” someone or recalling the fact that the person has already been encountered. This level, nonetheless, involves already both the encoding of simple visual features (e.g., “she has dark hair”) and the creation of more complex, abstract concepts (e.g., “it is a young, female person”). In the dual-process accounts of recognition memory (Renoult, Irish, Moscovitch, & Rugg, 2019; Rugg & Yonelinas, 2003; Yonelinas, 2002), this is the level that is typically referred to as “familiarity,” and in the current review, we designate it as visual, perceptual familiarity.

Figure 2. 

The basic stages of the process of how the central nervous system accumulates information as we get to know someone. For details, see the text. Image credits: The three upper face images were artificially generated by Generative Adversarial Networks (www.thispersondoesnotexist.com). Alicia Vikander: via Wikimedia; Angelina Jolie: imdb.com under the Creative Commons Attribution-Share Alike 2.0 generic license (creativecommons.org/licenses/by/2.0/).

Figure 2. 

The basic stages of the process of how the central nervous system accumulates information as we get to know someone. For details, see the text. Image credits: The three upper face images were artificially generated by Generative Adversarial Networks (www.thispersondoesnotexist.com). Alicia Vikander: via Wikimedia; Angelina Jolie: imdb.com under the Creative Commons Attribution-Share Alike 2.0 generic license (creativecommons.org/licenses/by/2.0/).

At the next level, to which we refer here as “contextual familiarity,” we gain additional associative or contextual information about a person, helping successful recall. This would be a situation, for example, when we always meet someone in the same context and under the same circumstances. If every morning one buys a coffee in the very same corner café, then the barista will be familiar after a while. In addition to the simple visual exposures, nonvisual sensory features such as the voice or the perfume of the person as well as the association of spatial and nonspatial contexts might help identification at this stage. However, the association of the person to the context is crucial at this level. Consider, for example, the situation when you meet the abovementioned barista accidentally and unexpectedly one evening in a foyer before a theater show. It is not unlikely that we would be confused and unsure who this person is, although we clearly would have the feeling that we do know the person from somewhere (Mandler, 1980).

In addition, and independently of the previous, personal episodic and/or additional semantic information might get available. This would happen if we connect the person to a particular memorable event in our past or if we gather some biographical semantic information about her or him. At these intermediate levels, we gain more and more knowledge about the specific characteristics of a person, including personality traits or attitudes either via real-life interpersonal relationships (Figure 2, examples in the right column) or via one-way parasocial interactions with media personalities (Figure 2, examples in the left column).

Finally, the biographical information becomes very detailed, and a unique and specific label, a name, is associated to the person, enabling stable, fast, and accurate recognition and identification. Importantly, the affective nature of the information is assumed to lead to better recognition and identity memory performance across all these levels (Mattarozzi, Colonnello, Russo, & Todorov, 2019).

Note that this framework is similar to the influential interactive activation and competition (IAC) model of face recognition (Burton, Bruce, & Hancock, 1999) in the sense that both the IAC and the current model consider person identification as a multistage process where different types of information can lead to different levels of familiarity, recognition, and identification. The most important difference, on the other hand, is that the IAC describes a situation where the processing of an already familiar face occurs. We, on the other hand, propose both a model of the gradual process of how we get to know someone, leading to the representation of a familiar face, and we also provide details of the process of how the representation changes as a face gets more and more familiar.

Please also note that the process of how we are getting to know someone is not necessarily as linear and hierarchical as the above description suggests, and each level is not the prerequisite to the more advanced ones. Rather, the presented model describes the process from the perspective of the hierarchy of the available information, starting with pure visual perceptual data and then adding more and more additional information in the form of associated contexts, semantic information, or personally relevant episodic memories (Figure 2).

In the following sections, we overview the available neuroimaging data according to the above-described model. We argue that most of the available studies, intended to test face recognition, in fact give us information only about particular steps of the process, and so far, no study tested the neural correlates of the process in its entirety. We also argue that only a limited amount of study tested so far how the brain represents a given familiar individual person. Finally, the review of the available data suggests that person identification does not occur in a given area or node as early face processing theories described (Bruce & Young, 1986); rather, it is the result of the gradual activation of more and more brain areas in a large, distributed and recurrent network as one progresses across the different stages of the face familiarization and identification process.

DIFFERING NEURAL PROCESSING OF FAMILIAR AND UNFAMILIAR FACES

By now, it is generally acknowledged that there are enormous perceptual differences between unfamiliar and familiar faces as well as great differences in their neural processing (for reviews, see Young & Burton, 2017, 2018; Ramon & Gobbini, 2017; Johnston & Edmonds, 2009; Gobbini & Haxby, 2007). For example, whereas the matching of nonstandardized, naturally variable face images of a familiar person (i.e., “telling faces together”) is an easy and automatic task, performance with unfamiliar faces is surprisingly poor (for a review, see Jenkins & Burton, 2011). This suggests that the specificity of the responsible neurons to low-level features, such as viewpoint or lightning, is qualitatively different for unfamiliar and familiar faces (for a review, see Johnston & Edmonds, 2009). Therefore, if one is interested in the process of how unfamiliar faces become familiar, the first question is to test where and how unfamiliar and familiar face representations differ neurally. In the following section, we summarize the data available on unfamiliar versus familiar face representations for the three above-described steps. First, we review the experiments that test the effect of a brief, premeasurement experimental familiarization with originally novel faces, testing the first perceptual familiarization steps. Second, we compare the representation of unfamiliar faces to that of faces, familiarized via prior exposures, applied together with associated contextual, semantic biographical, or other additional information. These experiments test various processes of the second, contextual familiarization stage. Next, we test how the representation of famous individuals, whom we know via long-term exposure to various media sources or via other, one-sided parasocial relationships, differs from unfamiliar or experimentally familiarized faces. Finally, we turn to personally familiar faces, as they should have the most robust person identity representation, because of the long and intensive real-life experiences; the potentially large amount of associative, episodic, biographical semantic information; and the large influence of affective value (for a specific review on the processing of personally familiar faces, see Ramon & Gobbini, 2017).

Earlier studies applied simple univariate whole-brain and ROI methods, compared the magnitude of neural activations for familiar and unfamiliar stimuli, and provided us with unequivocal results regarding the location and mechanism of familiar face processing. However, with the increasing availability of MVPA techniques, recently several studies tested if familiarity or face identity information can be read out directly from a given cortical region. As these techniques give us more specific information and better spatial resolution than typical univariate methods, in the following sections, we summarize univariate and multivariate results separately. Please note that the direct comparison of familiar and unfamiliar faces does not only reflect familiarity per se, but it might theoretically also include some degree of identity-specific recollection as well. However, the most accepted dual-process signal detection models of recognition memory (Yonelinas, Aly, Wang, & Koen, 2010; Yonelinas, 1994) assume that, whereas familiarity is a signal detection process, recollection is a threshold retrieval process. In other words, individuals might not recollect any information about an item that is familiar to them. For this reason, we discuss studies comparing various familiarized faces with unfamiliar faces separately from studies directly testing identity-specific information.

Experimental Familiarization

Familiarization with initially unfamiliar novel faces can be carried out in two fundamentally different ways. First, face images are presented repeatedly, and the participants perform either a related (intentional) memory encoding task or another, unrelated (incidental encoding) task. These paradigms test purely sensory effects and are considered a good model of the first steps of the learning process (Figure 2; visual familiarity). Alternatively, fictional biographical, episodic, social, or affective information can be made available to the observers during the learning period. This additional information is administered either acoustically or visually along the faces. The richer and deeper experience with the identities is claimed to facilitate the creation of a robust and long-lasting representation in this case, modeling contextual familiarity better.

Visual Familiarization

The univariate comparison of signal magnitude for visually familiar versus unfamiliar faces has been reviewed by Natu and O'Toole (2011); therefore, we just briefly summarize the results and add the more recent publications here. Typically, these studies apply versions of recognition memory paradigms where a study phase with repeated presentations of a set of faces is followed by a subsequent retrieval phase in the scanner and the participant's task is to decide (1) if the item either has been seen or not or (2) whether the previously seen item elicits the vivid feeling of remembering or it is just “known” as old without any additionally recalled information (Tulving, 1985). The neural activations for the newly learned and unfamiliar faces are then compared to each other. These studies revealed an enhanced activation for learned faces in the MPFC (Leube, Erb, Grodd, Bartels, & Kircher, 2003; Leveroni et al., 2000) and in the parietal lobe over the PC/PCC as well as in the TPJ (Gobbini & Haxby, 2006; Kosaka et al., 2003; Leube et al., 2003; Leveroni et al., 2000), the hippocampus, and the inferior frontal gyrus (IFG; Bobes, Lage Castellanos, Quiñones, García, & Valdes-Sosa, 2013). Common to these studies is the absence of enhanced activations in core face-selective regions (e.g., in the OFA and FFA) for visually familiarized faces. In fact, many studies reported smaller responses to familiar versus unfamiliar faces in these regions (Gobbini & Haxby, 2006; Kosaka et al., 2003; Schwartz et al., 2003; Leveroni et al., 2000). Another few studies, however, found elevated activations for visually familiarized faces in the core face processing network as well. Katanoda, Yoshikawa, and Sugishita (2000) using blocks of learned faces and contrasting them with a combination of novel and learned faces found bilateral fusiform activation for previously learned ones. Apps and Tsakiris (2013) presented a set of faces either several times from different viewpoints to induce familiarity or only once while undergoing fMRI. Next, they created a predictive coding (Friston, 2012) computational model to fit the behavioral data and found that the FFA activity covaried with the model parameter, signaling the updating of familiarity of each facial identity.

So far, only a few studies used MVPA to decode familiarity from the neural responses to visually familiarized faces. Natu and O'Toole (2015) used a visual familiarization paradigm with 30 originally unfamiliar female faces. By changing the number of repetitions, authors manipulated the degree of familiarization and tested if a classification algorithm could discriminate the neural patterns elicited by unfamiliar and familiar faces or not. They discovered that the OFA, the FFA, and the PC contained neural activity patterns, useful for discriminating highly familiar and unfamiliar faces. These results are therefore in agreement with those univariate studies that identified differential activations in the core face processing network as well. Rissman, Greely, and Wagner (2010) and Uncapher, Boyd-Meredith, Chow, Rissman, and Wagner (2015) used a model of eyewitness identification tests and compared the neural patterns for faces correctly recognized as “old” (familiar) or “new” (unfamiliar). They found that the parahippocampal and fusiform gyri, as well as the IPL and the ventrolateral pFC, contained diagnostic signal to the classifier. More recently, Hahn and O'Toole (2017) went beyond testing familiarity for faces and presented short video clips of previously familiarized or unfamiliar persons walking toward the camera. They discovered that the OFA, the STS, and the fusiform body area (Peelen & Downing, 2005) provided information about person familiarity when the face of the approaching person came into clear view.

Familiarization via Associated Biographical and Contextual Information

In real life, purely perceptual familiarity is typically quickly followed by semantic biographical, contextual, personality, or other associative information. Such person-specific knowledge affects our judgments and impressions and eventually modulates our social interactions with the target persons. It also seems that the associated contextual information shifts face representation from image-based toward view-invariant concepts within the core network, suiting person identification purposes (Schwartz & Yovel, 2016, 2019). The first, testing the effect of this contextual familiarization with neuroimaging methods, was Todorov, Gobbini, Evans, and Haxby (2007). Authors presented originally unfamiliar faces along sentences regarding affective episodic information about person-specific personality traits. They found that only areas, such as the MPFC and the STS, implicated previously in theory-of-mind tasks showed stronger activations to the familiarized versus novel faces. Tsukiura et al. (2010) associated unfamiliar faces with names and job titles. The successful encoding of these properties led to a stronger response in the ATL. Later, Cloutier, Kelley, and Heatherton (2011) compared directly the effects of a purely visual perceptual and person knowledge-based familiarity training. Purely perceptual familiarity activated the PC/PCC, whereas familiarity with person knowledge led to an increased activation in a number of areas, including MPFC, PC/PCC, and STS, and a reduced activation in the right FFA.

The only study so far testing this type of familiarization with MVPA was by Goesaert and Op de Beeck (2013). The authors associated eight of nine faces with personal (names and semantic biographical or episodic) information and with images of scenes, signaling unique spatial contexts. They found that the familiarized and unfamiliar faces were associated with different neural patterns in the early occipital visual areas and in the ventral temporal face regions, including OFA, FFA, and ATL.

Famous Faces

In comparison with the brief experimental familiarization processes described under the previous point, we have a far longer lasting and more thorough experience with our favorite celebrities. The repeated and typically extremely variable exposure to an actor, spreading over several years and different media roles, might lead to a more stable person representation. Therefore, it is hardly surprising that several univariate neuroimaging studies compared the magnitude of neural activity to famous faces to the response to unfamiliar faces (for a review, see Natu & O'Toole, 2011). These studies showed larger activity for famous when compared to unfamiliar faces in both the core and extended face processing systems, including inferior occipital (Ishai, Schmidt, & Boesiger, 2005); fusiform (Bobes et al., 2013; Nielson et al., 2010; Ishai et al., 2005; Gobbini, Leibenluft, Santiago, & Haxby, 2004; Grill-Spector, Knouf, & Kanwisher, 2004; Sergent, Ohta, & MacDonald, 1992); anterior middle temporal (Gorno-Tempini & Price, 2001; Leveroni et al., 2000; Gorno-Tempini et al., 1998); medial temporal, such as hippocampal and parahippocampal (Bar, Aminoff, & Ishai, 2008; Leveroni et al., 2000); and superior temporal (Ishai et al., 2005; Leveroni et al., 2000) areas as well as the MPFC, the PC, the TPJ (Nielson et al., 2010; Gobbini et al., 2004; Leveroni et al., 2000), and the amygdala (Elfgren et al., 2006; Bernard et al., 2004).

Is the familiarity of famous faces different from that of perceptually familiarized faces? Gimbel, Brewer, and Maril (2017) compared directly the activity of the MTL for briefly, visually familiarized and famous faces. They found distinct neural processes in the MTL with larger activation for visually familiarized faces in the perirhinal and for famous faces in the parahippocampal cortex. Contrary to these results, Barense, Henson, and Graham (2011) found significantly larger activity for famous compared to unfamiliar faces in the left perirhinal cortex and ATL as well as in the bilateral anterior hippocampus, a result in correspondence with another study showing differential hippocampal activation during the recognition of remote and recent celebrities (Bernard et al., 2004). Interestingly, so far, no neuroimaging study evaluated the difference of neural activation patterns for famous and unfamiliar faces, using MVPA.

Personally Familiar Faces

Unarguably, we have the most experience with the deepest knowledge about and the most intimate social interactions with people familiar to us personally (for reviews, see Ramon & Gobbini, 2017; Sugiura, 2014; Natu & O'Toole, 2011). Univariate neuroimaging studies testing personal familiarity compared such faces against either famous or unfamiliar ones. These studies found a large number of areas from both the core and extended parts of the face processing network. This network of personally familiar face processing includes the MPFC, ACC and PCC, and the anterior paracingulate cortex (Góngora, Castro-Laguardia, Pérez, Valdés-Sosa, & Bobes, 2019; Visconti di Oleggio Castello, Halchenko, Guntupalli, Gors, & Gobbini, 2017; Bobes et al., 2013; Gobbini et al., 2004; Leibenluft, Gobbini, Harrison, & Haxby, 2004); the right iFFA (Visconti di Oleggio Castello et al., 2017); the IPL and the TPJ (Sugiura, Mano, Sasaki, & Sadato, 2011; Platek et al., 2006); the PC (Visconti di Oleggio Castello et al., 2017; Sugiura et al., 2011; Gobbini et al., 2004); the middle and superior temporal cortices (Visconti di Oleggio Castello et al., 2017; Sugiura et al., 2011; Platek et al., 2006; Gobbini et al., 2004; Leibenluft et al., 2004); inferior temporal regions, such as the FFA and ATL (Visconti di Oleggio Castello et al., 2017; Ramon, Vizioli, Liu-Shuang, & Rossion, 2015; Sugiura et al., 2011; Gobbini et al., 2004; Leibenluft et al., 2004); the insula (Góngora et al., 2019; Platek et al., 2006; Leibenluft et al., 2004); and MTL regions, such as the amygdala, hippocampus, and perirhinal cortex (Ramon et al., 2015; Sugiura et al., 2011; Taylor et al., 2009), showing stronger neural responses to personally familiar faces.

In an interesting recent study, Vila et al. (2019) compared the activations specifically for loved (and thereby personally highly familiar) faces, such as the face of one's father, mother, partner, and best friend, with unfamiliar faces and found that a subset of the abovementioned areas, including ACC, medial OFC, PCC, and the inferior frontal cortex, was activated, emphasizing the role of these areas in the emotional processing of faces. It is worth mentioning that essentially the same network of areas has been found to be activated by the special case of self-faces as well when contrasted with familiar faces (for reviews, see Devue & Brédart, 2011; Keenan, Wheeler, Gallup, & Pascual-Leone, 2000).

Multivariate analyses, when compared to the above univariate studies, found differential patterns of neural activations for personally familiar faces in a somewhat more restricted network. Ramon et al. (2015) found significant differences between multivoxel patterns elicited by personally familiar and unfamiliar faces in the bilateral amygdala and the left FFA. Visconti di Oleggio Castello et al. (2017) designed an MVPA study to dissociate familiarity and identity information. They trained classifiers on a set of personally familiar (including own) and unfamiliar faces and found significant familiarity classification across identities in bilateral middle temporal, right anterior fusiform, right inferior frontal, TPJ, PC, and MPFC areas.

Familiarity-Specific Neural Activations

Overall, these results are in agreement with a recent meta-analysis of univariate studies on multimodal face and voice-based person recognition (Blank et al., 2014) and show that both the core and extended face processing networks, as well as several other areas, are sensitive to the familiarity of a face. Table 1 summarizes, in a somewhat simplified form, which brain areas are sensitive to familiarity across the various stages of familiarization.

Table 1. 
Brain Regions Encoding Facial Familiarity
Network PlaceAreaPerceptual FamiliarizationContextual FamiliarizationFamousPersonally Familiar
Core face network OFA   
MTG/STS 
FFA U, M U, M 
IFG       U, M 
  
Semantic person knowledge ATL   
  
Memory retrieval PC/PCC U, M 
Parahippocampal     
Entorhinal/perirhinal     
Hippocampus   
  
Personality, trait, attitude TPJ   U, M 
IPL     
MPFC/ACC U, M 
  
Emotion Amygdala     U, M 
Insula       U, M 
Network PlaceAreaPerceptual FamiliarizationContextual FamiliarizationFamousPersonally Familiar
Core face network OFA   
MTG/STS 
FFA U, M U, M 
IFG       U, M 
  
Semantic person knowledge ATL   
  
Memory retrieval PC/PCC U, M 
Parahippocampal     
Entorhinal/perirhinal     
Hippocampus   
  
Personality, trait, attitude TPJ   U, M 
IPL     
MPFC/ACC U, M 
  
Emotion Amygdala     U, M 
Insula       U, M 

U = univariate differences, showing larger responses for perceptually, contextually familiarized, famous, or personally familiar faces when compared to unfamiliar ones; M = multivariate pattern decoding revealing significant information regarding the familiarity of a face from the area.

The comparison of the brain areas activated differentially for unfamiliar and familiar faces across the different familiarization conditions suggests that most, but not all, of the areas of this network are active at every stage of the process. After showing that various familiarization steps lead to differential neural activations across this network, the next important question is if these areas encode only familiarity per se or they also play a role in the identification of a familiar person.

IDENTIFICATION: DIFFERENTIAL FACE IDENTITY REPRESENTATION

Traditional univariate fMRI data analysis applies the subtraction logic by comparing the averaged BOLD activities for different stimulus conditions. Unfortunately, because of the low spatial resolution of fMRI, such techniques are unable to reveal the neural representation of individual faces or people. To overcome this problem, many studies have used adaptation techniques, where repeated stimulus presentations lead to reduced BOLD signal (fMRIa) and then certain stimulus properties, such as the identity of a person, are modulated and the release of the signal from this adaptation is measured. The logic behind these experiments is that, if a cortical area encodes the manipulated feature of the stimuli (in this case, facial identity), then it should show some recovery from the adapted state if the identity is altered across repetitions (Kar & Krekelberg, 2016; Grill-Spector, Henson, & Martin, 2006; Krekelberg, Boynton, & van Wezel, 2006). More recent studies, on the other hand, analyzed the fine-grained pattern of neural activity via MVPA. Such analysis techniques are able to estimate reliably if the pattern of the neural representations, evoked by different identities, differs from each other or not. In the following sections, similarly to the previous one, we summarize where in the brain person-specific representations are found by these two methods. We evaluate studies of unfamiliar, experimentally familiarized, famous, and personally familiar faces separately.

Unfamiliar Faces

Adaptation Technique

Already early neuroimaging studies tested identity representations by fMRIa methods, concentrating predominantly on the core areas of the face processing network, specifically on the FFA. They typically found a reduced fMRI signal (fMRIa) for repeated unfamiliar identities when compared to alternating different identities along the fusiform gyrus (Baseler, Harris, Young, & Andrews, 2014; Winston, Henson, Fine-Goulden, & Dolan, 2004; Hoffman & Haxby, 2000). In addition, many studies suggested that this representation is viewpoint, illumination, and face-shape dependent (Xu & Biederman, 2010; Fox, Moon, Iaria, & Barton, 2009; Ewbank & Andrews, 2008; Gilaie-Dotan & Malach, 2007; Loffler, Yourganov, Wilkinson, & Wilson, 2005; Andrews & Ewbank, 2004; Yovel & Kanwisher, 2004; Grill-Spector et al., 1999). Similar, image-dependent identity representations were also found in the STS (Davies-Thompson, Gouws, & Andrews, 2009; Fox et al., 2009; but see Baseler et al., 2014, for a larger response to repeated faces in the posterior STS [pSTS]) and in the OFA (Davies-Thompson et al., 2009; Yovel & Kanwisher, 2004; Hoffman & Haxby, 2000).

However, the image (in)dependence of unfamiliar face representation within the core network areas is still under debate. Many studies found that the fMRIa in the FFA is invariant to spatial frequency (Eger, Schyns, & Kleinschmidt, 2004), to viewpoint (Pourtois, Schwartz, Seghier, Lazeyras, & Vuilleumier, 2005) and to the specific image (Weibert et al., 2016) of the presented unfamiliar face. Some of these fMRIa studies also described differences in image dependence between unfamiliar and famous faces. Henson, Shallice, and Dolan (2000) found significant repetition enhancement for the repetition of the identical images of unfamiliar faces and could only describe fMRIa for famous face repetitions in the FFA. Later, Eger, Schweinberger, Dolan, and Henson (2005) compared the representation across different images of the same individuals, varying simultaneously in many factors, such as illumination, viewpoint, expression, or hairstyle, similarly to the naturally variable “ambient” faces of Burton (2013) and Jenkins et al. (2011). They found similar fMRIa for unfamiliar and famous faces along the fusiform gyrus with the exception of medial anterior right FFA, which showed greater signal reduction for famous than unfamiliar faces.

MVPA

Relatively few studies applied MVPA methods to test the neural representations of unfamiliar faces so far. Natu et al. (2010) examined neural response patterns for viewpoint-independent facial identity representations. Although they failed to find significant identity discrimination in the FFA alone, it became possible to decode facial identity from the neural activation pattern when the fusiform gyrus was considered together with inferior and lateral occipital areas.

Gratton, Sreenivasan, Silver, and D'Esposito (2013) used a female face, a male face, and four gradual in-between morph images. They applied a technique that enables the measurement of stimulus tuning curves for individual voxels. They found a shape-dependent encoding of the morph continuum in FFA, OFA, and STS and only a broader, categorical representation of identities in the ATL. Goesaert and Op de Beeck (2013) used a contextual familiarity training paradigm. Before the training, at a time point when the faces were still unfamiliar, no identity information could be extracted from the joined face region, composed of OFA, FFA, and ATL. Zhang et al. (2016) tested identity encoding across facial expressions and also found only a limited and partial discrimination capacity in the FFA and ATL.

Hence, the abovementioned studies confirm the results of the univariate studies, suggesting that the core areas and the ATL are rather sensitive to visual dimensions of unfamiliar faces than to the identity per se. However, Dobs, Schultz, Bülthoff, and Gardner (2018), using dynamic faces, had a different conclusion. They tested the potential segregation of processing for facial expression and identity (Haxby et al., 2000) in a stream of dynamic avatar faces while participants performed either an expression task or an identity task. Attending to identity increased decoding capacity in the right FFA and STS, arguing for the role of these areas in identification and supporting those univariate results that debate the independent processing of changeable and invariant face attributes of faces (Xu & Biederman, 2010).

Experimental Familiarization

To the best of our knowledge, no fMRIa study tested individual face representation with experimentally familiarized faces so far. Therefore, in the next chapters, we concentrate on the available MVPA results from the field of visual and contextual familiarization.

Visual Familiarization

Kriegeskorte, Formisano, Sorger, and Goebel (2007) were the first to test the effect of familiarization by MVPA. The authors used a male face and a female face, which were briefly familiarized visually before the experiments. They found that the response patterns were distinct for facial identities in the ATL. Nestor, Plaut, and Behrmann (2011) used four male faces with four different facial expressions and a brief familiarization phase, at least 1 day before scanning. They found that the bilateral FFA and the anterior middle temporal gyrus (MTG) are sensitive to individuation across variations of expressions. Unfortunately, both abovementioned studies applied only a few different identities, with limited stimulus variability. Anzellotti, Fairhall, and Caramazza (2014) tried to overcome these limitations and used five briefly familiarized 3-D faces, seen from five orientations each together with a simple identity detection task. An orientation-independent face discrimination was found in the OFA, FFA, and ATL. Guntupalli, Wheeler, and Gobbini (2017) used a similar approach but helped familiarization training by presenting short video clips of four, naturally behaving, unfamiliar persons. They found that the identity representation of the FFA is modulated by head views whereas that of the right iFFA proved to be view invariant.

Familiarization via Associated Biographical and Contextual Information

As mentioned above, Goesaert and Op de Beeck (2013) associated faces with personal and spatial context information in an MVPA study. For the familiarized faces, they could decode identity information from the OFA, FFA, and ATL. Verosky, Todorov, and Turk-Browne (2013) presented 16 originally unfamiliar faces from multiple viewpoints and associated them with names or with names and an additional brief semantic description of biographical facts or with episodic events. They found significantly larger correlations of the neural patterns (Haxby et al., 2001) of a given identity with itself as compared to the pattern evoked by another identity across runs in the bilateral FFA. This indicates the presence of identity representation within the ventral temporal cortex, as a result of brief contextual association. Notably, however, the addition of biographical information did not increase the strength of pattern correlations for a given identity in the FFA, supporting the intermediate role of the area in identity processing. Wang et al. (2017) associated biographical details to four male faces. They first trained a classifier to discriminate individual faces and tested if this performance generalized to other cues, such as names, places, or diagnostic objects. Authors found that the ATL contained information about the learned persons on an abstract conceptual level, generalizing across multiple sources, confirming the idea that the region acts like a semantic “hub” of person-related knowledge (for reviews, see Lambon Ralph, 2014; Patterson, Nestor, & Rogers, 2007).

Two further MVPA studies went beyond decoding face-based person identification. van den Hurk, Gentile, and Jansma (2011) presented blocks of words, related to previously presented specific faces or object categories. Authors could successfully decode the person-specific semantic context from FFA, even in the absence of face stimuli.

Finally, Hassabis et al. (2014) trained participants to learn personality traits of four imagery protagonists and scanned them while they imagined these persons in various fictional situations. Thus, they did not test face identification, rather the representation of persons without their immediate presence. Corresponding to the high-level nature of this task, the analysis revealed that the pattern of the MPFC activity could reliably discriminate between the four protagonists, confirming the results of previous univariate face identification and familiarity studies.

Famous Faces

Adaptation Technique

fMRIa studies of famous faces largely concentrated on the question if the representation is dependent on low-level stimulus features in the FFA or not. Most of these studies found a significant fMRIa in the FFA for repeated when compared to alternating famous face identities (Henson et al., 2000), and it seems that the effect is similar for whole faces and internal facial features (Andrews, Davies-Thompson, Kingstone, & Young, 2010). Overall, these studies suggest that the FFA representation of famous faces is size (Ewbank, Henson, Rowe, Stoyanova, & Calder, 2013), viewpoint (Ewbank & Andrews, 2008), and image independent (Weibert et al., 2016; Davies-Thompson, Newling, & Andrews, 2013; Eger et al., 2005; Rotshtein, Henson, Treves, Driver, & Dolan, 2005; but see Davies-Thompson et al., 2009, and Yang, Susilo, & Duchaine, 2016, suggesting image variance). In addition, significant fMRIa was found in the OFA (Ewbank et al., 2013), in the left iFFA (Sugiura et al., 2011), in the ATL (Yang et al., 2016), and in the MTL (Weibert et al., 2016).

MVPA

Whereas fMRIa studies are numerous, so far, only two MVPA studies tested identity representation for famous faces. Axelrod and Yovel (2015) were the first to apply MVPA to test the encoding of famous faces. They used eight different images of two famous male persons and tested where face identity can be decoded. They found that only the right FFA showed significant decoding performance, supporting the pivotal role of the area in face recognition. Jeong and Xu (2016) used eight famous faces, varying in viewpoint, hairstyle, expression, and age. Unlike the previous study, these authors failed to find significant face identity representations in the core network areas. The only place where face identity could be successfully decoded from was the IPL. Very recently, Tsantani, Kriegeskorte, McGettigan, and Garrido (2019) measured fMRI activation patterns for the faces and voices of 12 highly familiar famous individuals to identify modality-general, person-identity representations in the brain. They found significant face-identity discrimination within the right OFA, FFA, pSTS, and PC/PCC, whereas cross-modal person-identity discrimination was only present in pSTS. This suggests that the pSTS can generalize across highly variable identity-specific face and voice information in a modality-general and invariant manner.

Personally Familiar Faces

Adaptation Technique

Although many prior studies tested the differential activation of personally familiar faces from that of famous or unfamiliar faces in general (for a review, see Ramon & Gobbini, 2017), only a handful of studies appeared so far on the specific representation of personally familiar faces. Ramon, Dricot, and Rossion (2010) used fMRIa with personally familiar faces. They created morph continua between personally familiar faces and tested the sensitivity of the neurons for minor physical changes between faces falling on the same or on opposite sides of identity category, signaling categorical perception and differential identity representation (Rotshtein et al., 2005). Surprisingly, the posterior STS and the precentral gyrus over the frontal cortex (Rajimehr et al., 2009), but not the FFA, showed such a pattern. Sugiura et al. (2011) compared unfamiliar, famous, and personally familiar faces, using fMRIa. They found that episodic and semantic memory-related structures, such as the right pSTS, left amygdala, bilateral parahippocampal gyrus, and ATL, were preferentially activated for personally familiar faces. Finally, Szpunar, St Jacques, Robbins, Wig, and Schacter (2014) used a mental simulation experiment with friends from social media sources, very similar to that of Hassabis et al. (2014). They modulated the content, as well as the frequency of imagery, simulating future social/nonsocial events and measured the associated content-specific fMRIa. The repeated simulation of people in social events led to a significant fMRIa in anterior/ventral MPFC areas.

MVPA

There are two available MVPA studies testing the individual representation of personally familiar faces. Visconti di Oleggio Castello et al. (2017) measured identity decoding for the faces of four friends, each with three images from different viewpoints. They found significant identity classification performance not only in the MPFC, PC, bilateral MTG/STS, TPJ, and IFG but also in the early visual cortex as well as in the OFA and FFA. Thornton and Mitchell (2017) used mental simulations of personally familiar adults. Merely thinking about familiar others elicited person-specific neural activity patterns in several areas of the core and extended networks, including the STS, IFG, MPFC, PC/PCC, and TPJ.

Encoding of Facial Identity

Table 2 summarizes the results of the identity-specific fMRIa and MVPA studies. Overall, it seems that identity-specific information can only be found in a network of areas, overlapping with the previously described network that responds differentially for familiar and unfamiliar faces (Table 1). Identity, just like familiarity per se, seems to be encoded across the entire network, but (unlike familiarity) to a different degree within the different familiarization conditions. Whereas personally familiar faces activate almost every brain area, the identity representation of unfamiliar faces is restricted to the core network areas and experimentally familiarized and famous faces lay between these two conditions.

Table 2. 
Brain Regions Encoding Facial Identity
 AreaUnfamiliarPerceptual FamiliarizationContextual FamiliarizationFamousPersonally Familiar
Visual EV         
  
Core face network OFA A, M 
MTG/STS A, M   A, M 
Fusiform gyrus A, M A, M 
IFG/precentral gyrus     A, M 
  
Semantic person knowledge ATL (M) A, M 
  
Memory retrieval PC/PCC         
MTL       
  
Personality, trait, attitude TPJ         
IPL         
MPFC/ACC       A, M 
  
Emotion Amygdala         
 AreaUnfamiliarPerceptual FamiliarizationContextual FamiliarizationFamousPersonally Familiar
Visual EV         
  
Core face network OFA A, M 
MTG/STS A, M   A, M 
Fusiform gyrus A, M A, M 
IFG/precentral gyrus     A, M 
  
Semantic person knowledge ATL (M) A, M 
  
Memory retrieval PC/PCC         
MTL       
  
Personality, trait, attitude TPJ         
IPL         
MPFC/ACC       A, M 
  
Emotion Amygdala         

A = significant fMRIa for unfamiliar, perceptually, contextually familiarized, famous or personally familiar faces; EV = early visual cortex; M = multivariate pattern decoding, revealing significant identity specific from the area.

HOW DO THE BRAIN RESPONSES CHANGE AS WE GET TO KNOW SOMEONE?

Familiarity Is Encoded across the Entire Face-Identification Network

Several areas of the familiarity-sensitive network (Figure 3), such as large parts of the core face network, parts of the long-term memory, and personality systems, seem to be activated differentially for all four tested familiarity types (i.e., visually and semantically familiarized, famous, and personally familiar faces), when compared to unfamiliar faces. Importantly, these areas largely correspond to the results of a recent functional connectivity study (Ramot, Walsh, & Martin, 2019), which found that face memory performance is predicted best by the strength of connections between areas of the core system and long-term memory as well as the social processing systems.

Figure 3. 

Familiarity encoding at the different stages of familiarization process in the different areas of the facial PIN. The model builds upon and modifies the models of Gobbini and Haxby (2007), Duchaine and Yovel (2015), Visconti di Oleggio Castello et al. (2017), and Rapcsak (2019). Colored shadings signal differential activations for familiar and unfamiliar faces, whereas gray shading signals the lack of familiarity specificity within an area. EV = early visual cortex; AMY = amygdala; INS = insula.

Figure 3. 

Familiarity encoding at the different stages of familiarization process in the different areas of the facial PIN. The model builds upon and modifies the models of Gobbini and Haxby (2007), Duchaine and Yovel (2015), Visconti di Oleggio Castello et al. (2017), and Rapcsak (2019). Colored shadings signal differential activations for familiar and unfamiliar faces, whereas gray shading signals the lack of familiarity specificity within an area. EV = early visual cortex; AMY = amygdala; INS = insula.

Although for identifying a person, faces, voices, names, and biographical information can similarly be used as input information, all the previously reviewed studies described the situation when persons are recognized on the basis of their faces.2 Therefore, in the following sections, to be comparable to the influential model of Gobbini and Haxby (2007), we describe a network where the bottom–up sensory input is exclusively composed of human face-like stimuli and is preprocessed by early visual areas (early visual cortex).

Regarding the core network, it seems that only the IFG face area shows familiarity sensitivity: A recent study found differential (and identity-independent) encoding of personally familiar and unfamiliar faces (Visconti di Oleggio Castello et al., 2017), whereas no such result was found for any other familiarization conditions so far. This emphasizes the role of IFG at the upper end of the hierarchy within the core system (Axelrod & Yovel, 2013). Similarly, the ATL face area seems to encode familiarity better if associated contextual or biographical semantic factors are also involved. This is in line with those theories that argue that the ATL has a high-level, view-independent, and multimodal identity representation that is affected by prior experiences (Duchaine & Yovel, 2015; Blank et al., 2014). Whether the ATL is part of the core (Rapcsak, 2019) or extended network (Duchaine & Yovel, 2015) is currently under debate, but the current review is rather in correspondence with the idea that it is a specially developed area, responding differently to facial familiarization and identity information.

An interesting finding is that the areas typically involved in the encoding of theory of mind (Frith & Frith, 1999) and recognition memory (Rugg & Yonelinas, 2003) show sensitivity to familiarity under every tested familiarity condition. This supports the idea that not only the initial visual description but also the associated nonvisual information, such as the associated episodes and personality traits, is activated immediately by seeing a familiar face (Gobbini & Haxby, 2007) in an automatic fashion.

Probably every face, but the faces of our favorite celebrities and personally familiar faces specially, elicits increased emotional attachments and processing. This is typically related to the differential activation of the amygdala, insula, and limbic areas (Bobes et al., 2013; Natu & O'Toole, 2011; Gobbini & Haxby, 2007). The fact that only studies applying famous or personally familiar faces found familiarity-dependent responses in the amygdala suggests that the short practice of typical experimental familiarizations is not sufficient to activate these emotion processing areas differentially.

PIN: A Person-Identity Network

Figure 4 summarizes how identity-specific representation spreads across the “person identification network” (PIN) as an originally unfamiliar face gets well known to the observer. The results of the studies testing identity specificity showed some correspondence with the encoding of familiarity, but the two activation maps are not entirely overlapping.

Figure 4. 

Person identity-specific activation changes in the brain as a person gets known to an observer. For detailed figure captions, see Figure 3. AMY = amygdala.

Figure 4. 

Person identity-specific activation changes in the brain as a person gets known to an observer. For detailed figure captions, see Figure 3. AMY = amygdala.

Considering the core system, the data suggest that unfamiliar and familiar faces elicit a differential activation within most of its areas and that unfamiliar faces lead to identity-specific activations in the core, but not in the various parts of the extended system. There is a change, however, in the nature of the representation within the core areas: Although the image (in)dependence of the identity-specific activations of unfamiliar faces is currently under debate, it is clearer that the representation of various familiar faces is rather low level and image independent (Schwartz & Yovel, 2016, 2019; Jenkins & Burton, 2011).

On the other hand, the ATL and the IFG seem to have identity-specific activations but no differential activation for familiarity for perceptually familiarized faces. Thus, it seems that these areas might play a role in the recall of the identity of a person without the feeling of familiarity.

Another difference between the familiar and unfamiliar face processing is that the ATL has identity-specific representation at every familiarity condition, except the unfamiliar faces. This is in line with the evidences, suggesting that the ATL has a more complex representation than the areas of the core system and it is potentially involved in the identification of familiar faces. Therefore, this analysis suggests that the ATL, instead of being a part of the core network (Duchaine & Yovel, 2015), represents rather a separate functional entity, responsible for higher-level identification processes (Borghesani et al., 2019; Gainotti, 2011) and potentially playing a role in interfacing bottom–up, sensory representations and previously stored top–down knowledge (Wang et al., 2017).

Long-term memory and theory-of-mind areas showed differential encoding of familiarity from perceptual familiarization levels onward, but identity-specific encoding could only be detected in these areas for famous and personally familiar faces or from contextual familiarization levels, respectively. We suggest that the difference in the encoding of familiarity and identity within these areas explains the feeling of familiarity without identification when we encounter the same face repeatedly (Mandler, 1980).

Perhaps, the most surprising result of the current summary is that neither long-term memory nor emotion-related areas play a role in the encoding of identity for experimentally familiarized faces, restricting the future development of theory-level (Guest & Martin, 2020) models of facial identification. The PC and the neighboring PCC of the medial parietal lobe are known to be involved in long-term memory retrieval processes (Gilmore, Nelson, & McDermott, 2015; Wagner, Shannon, Kahn, & Buckner, 2005; Burgess, Maguire, Spiers, & O'Keefe, 2001), self-representation, and perspective-taking in social contexts (Cavanna & Trimble, 2006) as well as in identity-specific imagery (Simmons-Martin, 1983), whereas some consider it as part of the social brain as well (Frith, 2007). All these processes are the most efficient and intensive for very well-known, personally familiar individuals, and this may explain that these areas contain identity-specific information only for these.

Regions of the MTL, such as the hippocampus, parahippocampus, and perirhinal cortex, have long been implicated in long-term recognition memory processes (van Strien, Cappaert, & Witter, 2009; Yonelinas, 2002; Squire & Zola-Morgan, 1991). In the pioneering human single-cell recording study of Quiroga, Reddy, Kreiman, Koch, and Fried (2005), image-invariant, famous person-specific neurons were found in the MTL (including the hippocampus, entorhinal, and parahippocampal areas as well as the amygdala), strongly supporting this conclusion. It has been argued that areas around the perirhinal cortex are particularly important to familiarity discrimination, whereas the more complex contextual, associative, and recollective aspects of memory rely more on the hippocampus (for a review, see Brown & Banks, 2015; but see Bar et al., 2008, for a different conclusion regarding the role of the parahippocampal cortex in contextual associations).

Reviewing the available data supports this conclusion. Although the MTL, as a whole, was sensitive to familiarity across many familiarization conditions (see Table 1), identity-specific representation within these areas can only be revealed for famous and personally familiar faces. Overall, this suggests the differential role of various MTL regions in different stages of familiarization and emphasizes that a strong, long-term, and intimate experience is necessary for eliciting identity-specific representations in areas related to long-term memory. Whether this memory-related identity representation is either because of encoding, recall, or both is an open question so far.

Consider encountering suddenly your romantic partner or someone whom you strongly dislike. Unarguably, faces of personally familiar persons are superior to any other face in eliciting a spontaneous and vivid emotional reaction. The immediate and automatic selection of the appropriate social interaction may explain why identity specificity has been found in the amygdala for personally familiar faces. It is worth noting, however, that the signal magnitude profile of the amygdala is rather complex and less predictive regarding familiarity. In addition, the current analysis shows that the amygdala is differentially activated for famous faces but does not seem to play a role in their identification. Several studies found weaker activations in the amygdala for perceptually familiarized, when compared to unfamiliar, (Schwartz et al., 2003; Dubois et al., 1999) or for personally familiar, when compared to famous, faces (Gobbini et al., 2004; Leibenluft et al., 2004). Therefore, although its role in the encoding of face identity is evident (Mormann et al., 2015), the exact role of amygdala in signaling familiarity per se and the interaction of the two process are yet unclear.

It is important to emphasize that there is no clear hierarchy between the processing of familiarity and identity across the areas at a given familiarization level, as proposed by earlier models of person identification (Ellis, Jones, & Mosdell, 1997; Bruce & Young, 1986). In other words, one cannot conclude that certain areas, for example, the core system, merely encode familiarity, whereas the activation of later areas enables the successful identification of a person. First, familiarity information is present in the entire network for both famous and personally familiar faces. In addition, for personally familiar faces, identity information is also available across the entire network. Second, the core areas are active for both familiarity and identification at every level of familiarization. Third, as one progresses from unfamiliar to personally familiar faces, gradually more and more areas encode facial identity. This altogether argues against the strict serial information processing during identification and the existence of a single “person identification node” (PIN; Bruce & Young, 1986). We therefore recommend that it is more accurate to describe PIN as a “person identification network,” better corresponding to the framework-level (Guest & Martin, 2020), multistage familiarity model of Kramer, Young, and Burton (2018). Implementing this and other models of familiarity and face identification in the frame of the gradually spreading familiarity and identity within the network would therefore be an exciting and theoretically very important topic. We hypothesize here that the gradually spreading specificity across this network might explain why increasing familiarity with a face leads to better behavioral performance in many tasks.

The idea of a PIN is also in accordance with the model of Blank et al. (2014), who found no evidences for a single voice–face multimodal hub of person identity recognition either (for a similar conceptualization of the PIN for face and voice integration, see also Biederman et al., 2018). Finally, it is also in line with the results of the few available MVPA electrophysiological studies, which evaluated the temporal emergence of face identity representations and showed that both early (and therefore dominantly bottom–up and perceptual) and later (and therefore potentially top–down and conceptual) information contribute to the correct representation of facial identity (Ambrus, Kaiser, Cichy, & Kovács, 2019; Dobs, Isik, Pantazis, & Kanwisher, 2019; Nemrodov, Niemeier, Patel, & Nestor, 2018; Vida, Nestor, Plaut, & Behrmann, 2017; Nemrodov, Niemeier, Mok, & Nestor, 2016).

The current analysis of the newly available neuroimaging data adds one important aspect to the distributed neural model of familiar face recognition of Gobbini and Haxby (2007) that has been less emphasized. We propose that perceptually, contextually familiarized, famous, or personally familiar faces activate the same network but to a different extent. The data suggest that the differential activation of the functional units of the extended system best reflects the different types of facial familiarity. Specifically, the parietal and medial temporal areas, involved in the retrieval of long-term memories, and the areas involved in the emotional components of face processing are only activated for the most well-known, famous and personally familiar faces. This is a surprising conclusion as the processing of identity and emotion is traditionally considered as separate and relatively independent streams in both cognitive (Bruce & Young, 1986) and neural framework (Duchaine & Yovel, 2015) models of face processing. A promising direction of future studies might be to systematically alter the emotional component of experimentally familiarized faces and test how this affects familiarity and identity-specific encoding across the entire network.

Another surprising prediction of the model is that the processing of familiarity is not limited to earlier areas of the core network; rather, it modulates the entire PIN. Previous models suggest that the feeling of familiarity is generated in early, modality-specific areas, corresponding to parts of the core network for faces and in parts of the STS for voices (Blank et al., 2014; Bruce & Young, 1986), whereas identity-specific information develops only in the ATL and later areas. The current analysis, on the other hand, questions the hierarchy between familiarity and identity encoding: Familiarity has been found activating not only earlier core-network areas but also later, memory- and personality-related areas for every familiarity type. We propose that the widespread familiarity activation across many areas of the network and across different familiarity types is in line with the well-known behavioral fact that both healthy participants and prosopagnosic patients can develop the feeling of familiarity without the ability of correct identification of a face (for a review, see Blank et al., 2014).

Overall, the activation patterns across the various familiarization conditions show that the different steps of the familiarization/identification process are related to different activations within the areas of the network and that one can study these steps selectively by the appropriate stimuli and paradigms. Above all, the current review emphasizes the importance of separating the different types of familiarity when one studies person recognition and identification as well as the very careful control and testing of the level of familiarity within the tested population (Biederman et al., 2018).

LIMITATIONS, OPEN QUESTIONS, AND FUTURE DIRECTIONS

By reading the current summary, one might get the impression that the process of getting to know someone is always a serial one where person-specific visual, sensory contextual, and, finally, semantic information build a gradually more and more complex representation. Although sometimes this might happen in real life indeed, we are not claiming that these levels are always building hierarchically on each other. There are many other, less linear ways on how one can get to know someone. Consider, for example, a situation when you first “hear about someone” or when you meet them only online, virtually. You learn that a certain colleague works in another department at the same university and that she has the same hobby as you, also loves Star Trek, and is called Caroline. You can learn specific information about people, without ever meeting them or being exposed to their faces. Once you are introduced to them at a social gathering, a conference, for example, you connect the previously learned information to their faces. In this case, you are only exposed to a face after obtaining person-specific information. Unfortunately, to the best of our knowledge, the neural correlates of the process, where close emotional and personal information might be available without associated face knowledge, have not yet been compared to the more common process where facial information plays the dominant role in recognizing and identifying someone. Varying systematically what information is available and testing identity representations is a promising future direction for neuroimaging and electrophysiological experiments.

Another related issue (addressed briefly under the Familiarity Is Encoded across the Entire Face-Identification Network section) is that the current model describes a situation where the model input is always a static face image. This is because most of the available data, as well as most of the previous models, are based on faces. However, people are definitely more than their faces, and person identification involves the integration of multimodal sensory information (faces, voices, smells) together with a biography (potentially also involving episodic memories), emotional aspects, and personality traits. Thus, the current model is the best suited for face-based person identification, and its elaboration by the abovementioned aspects is necessary to account for the complex nature of person recognition and identification in the future.

Another open question regards the role of real-life personal interactions in building identity representations. Under laboratory conditions, familiarity and identification processes are tested frequently by experimentally familiarized or famous faces. Both conditions resemble the digital age parasocial relationships (Horton & Wohl, 1956), where the observers encounter people exclusively in the mass media or via the Internet. These relationships, however, are unidirectional with no opportunity for mutual interactions, typical for real-life interpersonal relationships. The direct comparison of parasocial and real-life interactions will elaborate further the model and might potentially explain the differences between contextual, famous, and personally familiar person representations.

Finally, the role of context needs to be clarified in the future. Context facilitates visual perception for nonface objects (Bar, 2004). In the current model, context is defined broadly, and under the category of contextual familiarization, we included studies where either spatial or nonvisual personal contexts (e.g., biographical data) were added to the faces. However, temporal or spatial perceptual, semantic, and conceptual contexts might modulate face representations in different cortical regions. Thus, a better definition of context may refine the model further, and the comparison of contextual effects for object and face recognition might help to advance our general knowledge about abstract neural representations.

Conclusions

The present review suggests that different aspects of the information (facial identity, contextual/semantic, social and emotional) emerge gradually in the brain as one gets more and more familiar with a person. Despite this gradual change, the same “person identity network” or PIN is active at every level of the familiarization, but to a different extent. It shows that the process of getting to know someone gradually changes the neural representation in the extended face processing system. It also implies that the typical experimental familiarization paradigms, using short-term trainings and various versions of the traditional learning paradigms, do not lead to the same neural representation of identities (and potentially objects as well) as the longer-term, sometimes several yearlong, exposures to famous persons or personally familiar people. The review of the available neuroimaging data also suggests that the major difference between the personally familiar and other faces is the specific activation of parts of the extended system, especially of the emotion and long-term memory-related areas.

Acknowledgments

The author was supported by a Deutsche Forschungsgemeinschaft grant (grant KO 3918/5-1). The author thanks Geza Ambrus, Charlotta Eick, and Chenglin Li for their comments on the paper; Linda König for her help with the figures and text; and the two reviewers of the first version of the paper for their constructive comments. There is no conflict of interests.

Reprint requests should be sent to Gyula Kovács, Biological Psychology and Cognitive Neurosciences, Institute of Psychology, Friedrich Schiller University Jena, Germany, or via e-mail: gyula.kovacs@uni-jena.de.

Notes

1. 

There is an enormous amount of available behavioral data, testing various aspects of face recognition and identification. Here, we restrict our review to studies that used fMRI, as they reflect the most direct information regarding the neural correlates of such cognitive processes (for reviews of the behavioural results, see Young & Burton, 2018; Burton, Jenkins, & Schweinberger, 2011; Jenkins & Burton, 2011; Johnston & Edmonds, 2009).

2. 

The two interesting exceptions to the rule of applying faces as inputs are the studies of Thornton and Mitchell (2017) and Hassabis et al. (2014) who both asked participants merely to think of specific persons. The future analysis of the network activations, depending on the nature of the bottom–up sensory input and its comparison with top–down activations, elicited by episodic or semantic cueing, imagery, attention, or predictions, represents a potentially very important future direction in studying person identification and shall refine the current model further.

REFERENCES

REFERENCES
Ambrus
,
G. G.
,
Amado
,
C.
,
Krohn
,
L.
, &
Kovács
,
G.
(
2019
).
TMS of the occipital face area modulates cross-domain identity priming
.
Brain Structure Function
,
224
,
149
157
.
Ambrus
,
G. G.
,
Dotzer
,
M.
,
Schweinberger
,
S. R.
, &
Kovács
,
G.
(
2017
).
The occipital face area is causally involved in the formation of identity-specific face representations
.
Brain Structure and Function
,
222
,
4271
4282
.
Ambrus
,
G. G.
,
Kaiser
,
D.
,
Cichy
,
R. M.
, &
Kovács
,
G.
(
2019
).
The neural dynamics of familiar face recognition
.
Cerebral Cortex
,
29
,
4775
4784
.
Ambrus
,
G. G.
,
Windel
,
F.
,
Burton
,
A. M.
, &
Kovács
,
G.
(
2017
).
Causal evidence of the involvement of the right occipital face area in face-identity acquisition
.
Neuroimage
,
148
,
212
218
.
Andrews
,
S.
,
Jenkins
,
R.
,
Cursiter
,
H.
, &
Burton
,
A. M.
(
2015
).
Telling faces together: Learning new faces through exposure to multiple instances
.
Quarterly Journal of Experimental Psychology
,
68
,
2041
2050
.
Andrews
,
T. J.
,
Davies-Thompson
,
J.
,
Kingstone
,
A.
, &
Young
,
A. W.
(
2010
).
Internal and external features of the face are represented holistically in face-selective regions of visual cortex
.
Journal of Neuroscience
,
30
,
3544
3552
.
Andrews
,
T. J.
, &
Ewbank
,
M. P.
(
2004
).
Distinct representations for facial identity and changeable aspects of faces in the human temporal lobe
.
Neuroimage
,
23
,
905
913
.
Anzellotti
,
S.
,
Fairhall
,
S. L.
, &
Caramazza
,
A.
(
2014
).
Decoding representations of face identity that are tolerant to rotation
.
Cerebral Cortex
,
24
,
1988
1995
.
Apps
,
M. A. J.
, &
Tsakiris
,
M.
(
2013
).
Predictive codes of familiarity and context during the perceptual learning of facial identities
.
Nature Communications
,
4
,
2698
.
Axelrod
,
V.
, &
Yovel
,
G.
(
2013
).
The challenge of localizing the anterior temporal face area: A possible solution
.
Neuroimage
,
81
,
371
380
.
Axelrod
,
V.
, &
Yovel
,
G.
(
2015
).
Successful decoding of famous faces in the fusiform face area
.
PLoS One
,
10
,
e0117126
.
Bar
,
M.
(
2004
)
Visual objects in context
.
Nature Reviews Neuroscience
,
5
,
617
629
.
Bar
,
M.
,
Aminoff
,
E.
, &
Ishai
,
A.
(
2008
).
Famous faces activate contextual associations in the parahippocampal cortex
.
Cerebral Cortex
,
18
,
1233
1238
.
Barense
,
M. D.
,
Henson
,
R. N. A.
, &
Graham
,
K. S.
(
2011
).
Perception and conception: Temporal lobe activity during complex discriminations of familiar and novel faces and objects
.
Journal of Cognitive Neuroscience
,
23
,
3052
3067
.
Baseler
,
H. A.
,
Harris
,
R. J.
,
Young
,
A. W.
, &
Andrews
,
T. J.
(
2014
).
Neural responses to expression and gaze in the posterior superior temporal sulcus interact with facial identity
.
Cerebral Cortex
,
24
,
737
744
.
Bernard
,
F. A.
,
Bullmore
,
E. T.
,
Graham
,
K. S.
,
Thompson
,
S. A.
,
Hodges
,
J. R.
, &
Fletcher
,
P. C.
(
2004
).
The hippocampal region is involved in successful recognition of both remote and recent famous faces
.
Neuroimage
,
22
,
1704
1714
.
Biederman
,
I.
,
Shilowich
,
B. E.
,
Herald
,
S. B.
,
Margalit
,
E.
,
Maarek
,
R.
,
Meschke
,
E. X.
, et al
(
2018
).
The cognitive neuroscience of person identification
.
Neuropsychologia
,
116
,
205
214
.
Blank
,
H.
,
Wieland
,
N.
, &
von Kriegstein
,
K.
(
2014
).
Person recognition and the brain: merging evidence from patients and healthy individuals
.
Neuroscience & Biobehavioral Reviews
,
47
,
717
734
.
Bobes
,
M. A.
,
Lage Castellanos
,
A.
,
Quiñones
,
I.
,
García
,
L.
, &
Valdes-Sosa
,
M.
(
2013
).
Timing and tuning for familiarity of cortical responses to faces
.
PLoS One
,
8
,
e76100
.
Borghesani
,
V.
,
Narvid
,
J.
,
Battistella
,
G.
,
Shwe
,
W.
,
Watson
,
C.
,
Binney
,
R. J.
, et al
(
2019
).
“Looks familiar, but I do not know who she is”: The role of the anterior right temporal lobe in famous face recognition
.
Cortex
,
115
,
72
85
.
Brown
,
M. W.
, &
Banks
,
P. J.
(
2015
).
In search of a recognition memory engram
.
Neuroscience & Biobehavioral Reviews
,
50
,
12
28
.
Bruce
,
V.
, &
Young
,
A.
(
1986
).
Understanding face recognition
.
British Journal of Psychology
,
77
,
305
327
.
Burgess
,
N.
,
Maguire
,
E. A.
,
Spiers
,
H. J.
, &
O'Keefe
,
J.
(
2001
).
A temporoparietal and prefrontal network for retrieving the spatial context of lifelike events
.
Neuroimage
,
14
,
439
453
.
Burton
,
A. M.
(
2013
).
Why has research in face recognition progressed so slowly? The importance of variability
.
Quarterly Journal of Experimental Psychology
,
66
,
1467
1485
.
Burton
,
A. M.
,
Bruce
,
V.
, &
Hancock
,
P. J. B.
(
1999
).
From pixels to people: A model of familiar face recognition
.
Cognitive Science
,
23
,
1
31
.
Burton
,
A. M.
,
Jenkins
,
R.
, &
Schweinberger
,
S. R.
(
2011
).
Mental representations of familiar faces
.
British Journal of Psychology
,
102
,
943
958
.
Calder
,
A. J.
, &
Young
,
A. W.
(
2005
).
Understanding the recognition of facial identity and facial expression
.
Nature Reviews Neuroscience
,
6
,
641
651
.
Cavanna
,
A. E.
, &
Trimble
,
M. R.
(
2006
).
The precuneus: A review of its functional anatomy and behavioural correlates
.
Brain
,
129
,
564
583
.
Chan
,
A. W.-Y.
, &
Downing
,
P. E.
(
2011
).
Faces and eyes in human lateral prefrontal cortex
.
Frontiers in Human Neuroscience
,
5
,
51
.
Cloutier
,
J.
,
Kelley
,
W. M.
, &
Heatherton
,
T. F.
(
2011
).
The influence of perceptual and knowledge-based familiarity on the neural substrates of face perception
.
Social Neuroscience
,
6
,
63
75
.
Collins
,
J. A.
, &
Olson
,
I. R.
(
2014
).
Knowledge is power: How conceptual knowledge transforms visual cognition
.
Psychonomic Bulletin & Review
,
21
,
843
860
.
Davies-Thompson
,
J.
,
Gouws
,
A.
, &
Andrews
,
T. J.
(
2009
).
An image-dependent representation of familiar and unfamiliar faces in the human ventral stream
.
Neuropsychologia
,
47
,
1627
1635
.
Davies-Thompson
,
J.
,
Newling
,
K.
, &
Andrews
,
T. J.
(
2013
).
Image-invariant responses in face-selective regions do not explain the perceptual advantage for familiar face recognition
.
Cerebral Cortex
,
23
,
370
377
.
Devue
,
C.
, &
Brédart
,
S.
(
2011
).
The neural correlates of visual self-recognition
.
Consciousness and Cognition
,
20
,
40
51
.
Dobs
,
K.
,
Isik
,
L.
,
Pantazis
,
D.
, &
Kanwisher
,
N.
(
2019
).
How face perception unfolds over time
.
Nature Communications
,
10
,
1258
.
Dobs
,
K.
,
Schultz
,
J.
,
Bülthoff
,
I.
, &
Gardner
,
J. L.
(
2018
).
Task-dependent enhancement of facial expression and identity representations in human cortex
.
Neuroimage
,
172
,
689
702
.
Dubois
,
S.
,
Rossion
,
B.
,
Schiltz
,
C.
,
Bodart
,
J. M.
,
Michel
,
C.
,
Bruyer
,
R.
, et al
(
1999
).
Effect of familiarity on the processing of human faces
.
Neuroimage
,
9
,
278
289
.
Duchaine
,
B.
, &
Yovel
,
G.
(
2015
).
A revised neural framework for face processing
.
Annual Review of Vision Science
,
1
,
393
416
.
Eger
,
E.
,
Schweinberger
,
S. R.
,
Dolan
,
R. J.
, &
Henson
,
R. N.
(
2005
).
Familiarity enhances invariance of face representations in human ventral visual cortex: fMRI evidence
.
Neuroimage
,
26
,
1128
1139
.
Eger
,
E.
,
Schyns
,
P. G.
, &
Kleinschmidt
,
A.
(
2004
).
Scale invariant adaptation in fusiform face-responsive regions
.
Neuroimage
,
22
,
232
242
.
Elfgren
,
C.
,
van Westen
,
D.
,
Passant
,
U.
,
Larsson
,
E. M.
,
Mannfolk
,
P.
, &
Fransson
,
P.
(
2006
).
fMRI activity in the medial temporal lobe during famous face processing
.
Neuroimage
,
30
,
609
616
.
Ellis
,
H. D.
,
Jones
,
D. M.
, &
Mosdell
,
N.
(
1997
).
Intra- and inter-modal repetition priming of familiar faces and voices
.
British Journal of Psychology
,
88
,
143
156
.
Ewbank
,
M. P.
, &
Andrews
,
T. J.
(
2008
).
Differential sensitivity for viewpoint between familiar and unfamiliar faces in human visual cortex
.
Neuroimage
,
40
,
1857
1870
.
Ewbank
,
M. P.
,
Henson
,
R. N.
,
Rowe
,
J. B.
,
Stoyanova
,
R. S.
, &
Calder
,
A. J.
(
2013
).
Different neural mechanisms within occipitotemporal cortex underlie repetition suppression across same and different-size faces
.
Cerebral Cortex
,
23
,
1073
1084
.
Fox
,
C. J.
,
Moon
,
S. Y.
,
Iaria
,
G.
, &
Barton
,
J. J. S.
(
2009
).
The correlates of subjective perception of identity and expression in the face network: An fMRI adaptation study
.
Neuroimage
,
44
,
569
580
.
Friston
,
K.
(
2012
).
Prediction, perception and agency
.
International Journal of Psychophysiology
,
83
,
248
252
.
Frith
,
C. D.
(
2007
).
The social brain?
Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences
,
362
,
671
678
.
Frith
,
C. D.
, &
Frith
,
U.
(
1999
).
Interacting minds—A biological basis
.
Science
,
286
,
1692
1695
.
Gainotti
,
G.
(
2011
).
What the study of voice recognition in normal subjects and brain-damaged patients tells us about models of familiar people recognition
.
Neuropsychologia
,
49
,
2273
2282
.
Gilaie-Dotan
,
S.
, &
Malach
,
R.
(
2007
).
Sub-exemplar shape tuning in human face-related areas
.
Cerebral Cortex
,
17
,
325
338
.
Gilmore
,
A. W.
,
Nelson
,
S. M.
, &
McDermott
,
K. B.
(
2015
).
A parietal memory network revealed by multiple MRI methods
.
Trends in Cognitive Sciences
,
19
,
534
543
.
Gimbel
,
S. I.
,
Brewer
,
J. B.
, &
Maril
,
A.
(
2017
).
I know I've seen you before: Distinguishing recent-single-exposure-based familiarity from pre-existing familiarity
.
Brain Research
,
1658
,
11
24
.
Gobbini
,
M. I.
, &
Haxby
,
J. V.
(
2006
).
Neural response to the visual familiarity of faces
.
Brain Research Bulletin
,
71
,
76
82
.
Gobbini
,
M. I.
, &
Haxby
,
J. V.
(
2007
).
Neural systems for recognition of familiar faces
.
Neuropsychologia
,
45
,
32
41
.
Gobbini
,
M. I.
,
Leibenluft
,
E.
,
Santiago
,
N.
, &
Haxby
,
J. V.
(
2004
).
Social and emotional attachment in the neural representation of faces
.
Neuroimage
,
22
,
1628
1635
.
Goesaert
,
E.
, &
Op de Beeck
,
H. P.
(
2013
).
Representations of facial identity information in the ventral visual stream investigated with multivoxel pattern analyses
.
Journal of Neuroscience
,
33
,
8549
8558
.
Góngora
,
D.
,
Castro-Laguardia
,
A. M.
,
Pérez
,
J.
,
Valdés-Sosa
,
P.
, &
Bobes
,
M. A.
(
2019
).
Anatomical connections underlying personally-familiar face processing
.
PLoS One
,
14
,
e0222087
.
Gorno-Tempini
,
M. L.
, &
Price
,
C. J.
(
2001
).
Identification of famous faces and buildings: A functional neuroimaging study of semantically unique items
.
Brain
,
124
,
2087
2097
.
Gorno-Tempini
,
M. L.
,
Price
,
C. J.
,
Josephs
,
O.
,
Vandenberghe
,
R.
,
Cappa
,
S. F.
,
Kapur
,
N.
, et al
(
1998
).
The neural systems sustaining face and proper-name processing
.
Brain
,
121
,
2103
2118
.
Gratton
,
C.
,
Sreenivasan
,
K. K.
,
Silver
,
M. A.
, &
D'Esposito
,
M.
(
2013
).
Attention selectively modifies the representation of individual faces in the human brain
.
Journal of Neuroscience
,
33
,
6979
6989
.
Grill-Spector
,
K.
,
Henson
,
R.
, &
Martin
,
A.
(
2006
).
Repetition and the brain: Neural models of stimulus-specific effects
.
Trends in Cognitive Sciences
,
10
,
14
23
.
Grill-Spector
,
K.
,
Knouf
,
N.
, &
Kanwisher
,
N.
(
2004
).
The fusiform face area subserves face perception, not generic within-category identification
.
Nature Neuroscience
,
7
,
555
562
.
Grill-Spector
,
K.
,
Kushnir
,
T.
,
Edelman
,
S.
,
Avidan
,
G.
,
Itzchak
,
Y.
, &
Malach
,
R.
(
1999
).
Differential processing of objects under various viewing conditions in the human lateral occipital complex
.
Neuron
,
24
,
187
203
.
Grill-Spector
,
K.
, &
Malach
,
R.
(
2001
).
fMR-adaptation: A tool for studying the functional properties of human cortical neurons
.
Acta Psychologica
,
107
,
293
321
.
Guest
,
O.
, &
Martin
,
A. E.
(
2020
).
How computational modeling can force theory building in psychological science
.
PsyArXiv
.
Guntupalli
,
J. S.
,
Wheeler
,
K. G.
, &
Gobbini
,
M. I.
(
2017
).
Disentangling the representation of identity from head view along the human face processing pathway
.
Cerebral Cortex
,
27
,
46
53
.
Hahn
,
C. A.
, &
O'Toole
,
A. J.
(
2017
).
Recognizing approaching walkers: Neural decoding of person familiarity in cortical areas responsive to faces, bodies, and biological motion
.
Neuroimage
,
146
,
859
868
.
Hassabis
,
D.
,
Spreng
,
R. N.
,
Rusu
,
A. A.
,
Robbins
,
C. A.
,
Mar
,
R. A.
, &
Schacter
,
D. L.
(
2014
).
Imagine all the people: How the brain creates and uses personality models to predict behavior
.
Cerebral Cortex
,
24
,
1979
1987
.
Haxby
,
J. V.
,
Connolly
,
A. C.
, &
Guntupalli
,
J. S.
(
2014
).
Decoding neural representational spaces using multivariate pattern analysis
.
Annual Review of Neuroscience
,
37
,
435
456
.
Haxby
,
J. V.
,
Gobbini
,
M. I.
,
Furey
,
M. L.
,
Ishai
,
A.
,
Schouten
,
J. L.
, &
Pietrini
,
P.
(
2001
).
Distributed and overlapping representations of faces and objects in ventral temporal cortex
.
Science
,
293
,
2425
2430
.
Haxby
,
J. V.
,
Hoffman
,
E. A.
, &
Gobbini
,
M. I.
(
2000
).
The distributed human neural system for face perception
.
Trends in Cognitive Sciences
,
4
,
223
233
.
Henson
,
R.
,
Shallice
,
T.
, &
Dolan
,
R.
(
2000
).
Neuroimaging evidence for dissociable forms of repetition priming
.
Science
,
287
,
1269
1272
.
Hoffman
,
E. A.
, &
Haxby
,
J. V.
(
2000
).
Distinct representations of eye gaze and identity in the distributed human neural system for face perception
.
Nature Neuroscience
,
3
,
80
84
.
Horton
,
D.
, &
Wohl
,
R.
(
1956
).
Mass communication and para-social interaction: Observations on intimacy at a distance
.
Psychiatry
,
19
,
215
229
.
Ishai
,
A.
,
Schmidt
,
C. F.
, &
Boesiger
,
P.
(
2005
).
Face perception is mediated by a distributed cortical network
.
Brain Research Bulletin
,
67
,
87
93
.
Jenkins
,
R.
, &
Burton
,
A. M.
(
2011
).
Stable face representations
.
Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences
,
366
,
1671
1683
.
Jenkins
,
R.
,
White
,
D.
,
van Montfort
,
X.
, &
Burton
,
A. M.
(
2011
).
Variability in photos of the same face
.
Cognition
,
121
,
313
323
.
Jeong
,
S. K.
, &
Xu
,
Y.
(
2016
).
Behaviorally relevant abstract object identity representation in the human parietal cortex
.
Journal of Neuroscience
,
36
,
1607
1619
.
Johnston
,
R. A.
, &
Edmonds
,
A. J.
(
2009
).
Familiar and unfamiliar face recognition: A review
.
Memory
,
17
,
577
596
.
Kar
,
K.
, &
Krekelberg
,
B.
(
2016
).
Testing the assumptions underlying fMRI adaptation using intracortical recordings in area MT
.
Cortex
,
80
,
21
34
.
Katanoda
,
K.
,
Yoshikawa
,
K.
, &
Sugishita
,
M.
(
2000
).
Neural substrates for the recognition of newly learned faces: A functional MRI study
.
Neuropsychologia
,
38
,
1616
1625
.
Keenan
,
J. P.
,
Wheeler
,
M. A.
,
Gallup
,
G. G.
, &
Pascual-Leone
,
A.
(
2000
).
Self-recognition and the right prefrontal cortex
.
Trends in Cognitive Sciences
,
4
,
338
344
.
Kosaka
,
H.
,
Omori
,
M.
,
Iidaka
,
T.
,
Murata
,
T.
,
Shimoyama
,
T.
,
Okada
,
T.
, et al
(
2003
).
Neural substrates participating in acquisition of facial familiarity: An fMRI study
.
Neuroimage
,
20
,
1734
1742
.
Kramer
,
R. S. S.
,
Young
,
A. W.
, &
Burton
,
A. M.
(
2018
).
Understanding face familiarity
.
Cognition
,
172
,
46
58
.
Krekelberg
,
B.
,
Boynton
,
G. M.
, &
van Wezel
,
R. J. A.
(
2006
).
Adaptation: From single cells to BOLD signals
.
Trends in Neurosciences
,
29
,
250
256
.
Kriegeskorte
,
N.
,
Formisano
,
E.
,
Sorger
,
B.
, &
Goebel
,
R.
(
2007
).
Individual faces elicit distinct response patterns in human anterior temporal cortex
.
Proceedings of the National Academy of Sciences, U.S.A.
,
104
,
20600
20605
.
Lambon Ralph
,
M. A.
(
2014
).
Neurocognitive insights on conceptual knowledge and its breakdown
.
Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences
,
369
,
20120392
.
Leibenluft
,
E.
,
Gobbini
,
M. I.
,
Harrison
,
T.
, &
Haxby
,
J. V.
(
2004
).
Mothers' neural activation in response to pictures of their children and other children
.
Biological Psychiatry
,
56
,
225
232
.
Leube
,
D. T.
,
Erb
,
M.
,
Grodd
,
W.
,
Bartels
,
M.
, &
Kircher
,
T. T. J.
(
2003
).
Successful episodic memory retrieval of newly learned faces activates a left fronto-parietal network
.
Brain Research Cognitive Brain Research
,
18
,
97
101
.
Leveroni
,
C. L.
,
Seidenberg
,
M.
,
Mayer
,
A. R.
,
Mead
,
L. A.
,
Binder
,
J. R.
, &
Rao
,
S. M.
(
2000
).
Neural systems underlying the recognition of familiar and newly learned faces
.
Journal of Neuroscience
,
20
,
878
886
.
Loffler
,
G.
,
Yourganov
,
G.
,
Wilkinson
,
F.
, &
Wilson
,
H. R.
(
2005
).
fMRI evidence for the neural representation of faces
.
Nature Neuroscience
,
8
,
1386
1390
.
Mandler
,
G.
(
1980
).
Recognizing: The judgment of previous occurrence
.
Psychological Review
,
87
,
252
271
.
Mattarozzi
,
K.
,
Colonnello
,
V.
,
Russo
,
P. M.
, &
Todorov
,
A.
(
2019
).
Person information facilitates memory for face identity
.
Psychological Research
,
83
,
1817
1824
.
Merkow
,
M. B.
,
Burke
,
J. F.
, &
Kahana
,
M. J.
(
2015
).
The human hippocampus contributes to both the recollection and familiarity components of recognition memory
.
Proceedings of the National Academy of Sciences, U.S.A.
,
112
,
14378
14383
.
Mormann
,
F.
,
Niediek
,
J.
,
Tudusciuc
,
O.
,
Quesada
,
C. M.
,
Coenen
,
V. A.
,
Elger
,
C. E.
, et al
(
2015
).
Neurons in the human amygdala encode face identity, but not gaze direction
.
Nature Neuroscience
,
18
,
1568
1570
.
Natu
,
V. S.
,
Jiang
,
F.
,
Narvekar
,
A.
,
Keshvari
,
S.
,
Blanz
,
V.
, &
O'Toole
,
A. J.
(
2010
).
Dissociable neural patterns of facial identity across changes in viewpoint
.
Journal of Cognitive Neuroscience
,
22
,
1570
1582
.
Natu
,
V.
, &
O'Toole
,
A. J.
(
2011
).
The neural processing of familiar and unfamiliar faces: A review and synopsis
.
British Journal of Psychology
,
102
,
726
747
.
Natu
,
V. S.
, &
O'Toole
,
A. J.
(
2015
).
Spatiotemporal changes in neural response patterns to faces varying in visual familiarity
.
Neuroimage
,
108
,
151
159
.
Nemrodov
,
D.
,
Niemeier
,
M.
,
Mok
,
J. N. Y.
, &
Nestor
,
A.
(
2016
).
The time course of individual face recognition: A pattern analysis of ERP signals
.
Neuroimage
,
132
,
469
476
.
Nemrodov
,
D.
,
Niemeier
,
M.
,
Patel
,
A.
, &
Nestor
,
A.
(
2018
).
The neural dynamics of facial identity processing: Insights from EEG-based pattern analysis and image reconstruction
.
eNeuro
,
5
,
ENEURO.0358-17.2018
.
Nestor
,
A.
,
Plaut
,
D. C.
, &
Behrmann
,
M.
(
2011
).
Unraveling the distributed neural code of facial identity through spatiotemporal pattern analysis
.
Proceedings of the National Academy of Sciences, U.S.A.
,
108
,
9998
10003
.
Nielson
,
K. A.
,
Seidenberg
,
M.
,
Woodard
,
J. L.
,
Durgerian
,
S.
,
Zhang
,
Q.
,
Gross
,
W. L.
, et al
(
2010
).
Common neural systems associated with the recognition of famous faces and names: An event-related fMRI study
.
Brain and Cognition
,
72
,
491
498
.
Patterson
,
K.
,
Nestor
,
P. J.
, &
Rogers
,
T. T.
(
2007
).
Where do you know what you know? The representation of semantic knowledge in the human brain
.
Nature Reviews Neuroscience
,
8
,
976
987
.
Peelen
,
M. V.
, &
Downing
,
P. E.
(
2005
).
Selectivity for the human body in the fusiform gyrus
.
Journal of Neurophysiology
,
93
,
603
608
.
Peterson
,
M. A.
, &
Rhodes
,
G.
(
2005
).
Perception of faces, objects, and scenes: Analytic and holistic processes
.
New York
:
Oxford University Press
.
Pitcher
,
D.
,
Walsh
,
V.
, &
Duchaine
,
B.
(
2011
).
The role of the occipital face area in the cortical face perception network
.
Experimental Brain Research
,
209
,
481
493
.
Platek
,
S. M.
,
Loughead
,
J. W.
,
Gur
,
R. C.
,
Busch
,
S.
,
Ruparel
,
K.
,
Phend
,
N.
, et al
(
2006
).
Neural substrates for functionally discriminating self-face from personally familiar faces
.
Human Brain Mapping
,
27
,
91
98
.
Pourtois
,
G.
,
Schwartz
,
S.
,
Seghier
,
M. L.
,
Lazeyras
,
F.
, &
Vuilleumier
,
P.
(
2005
).
Portraits or people? Distinct representations of face identity in the human visual cortex
.
Journal of Cognitive Neuroscience
,
17
,
1043
1057
.
Quiroga
,
R. Q.
,
Reddy
,
L.
,
Kreiman
,
G.
,
Koch
,
C.
, &
Fried
,
I.
(
2005
).
Invariant visual representation by single neurons in the human brain
.
Nature
,
435
,
1102
1107
.
Rajimehr
,
R.
,
Young
,
J. C.
, &
Tootell
,
R. B. H.
(
2009
).
An anterior temporal face patch in human cortex, predicted by macaque maps
.
Proceedings of the National Academy of Sciences, U.S.A.
,
106
,
1995
2000
.
Ramon
,
M.
,
Dricot
,
L.
, &
Rossion
,
B.
(
2010
).
Personally familiar faces are perceived categorically in face-selective regions other than the fusiform face area
.
European Journal of Neuroscience
,
32
,
1587
1598
.
Ramon
,
M.
, &
Gobbini
,
M. I.
(
2017
).
Familiarity matters: A review on prioritized processing of personally familiar faces
.
Visual Cognition
,
26
,
179
195
.
Ramon
,
M.
,
Vizioli
,
L.
,
Liu-Shuang
,
J.
, &
Rossion
,
B.
(
2015
).
Neural microgenesis of personally familiar face recognition
.
Proceedings of the National Academy of Sciences, U.S.A.
,
112
,
E4835
E4844
.
Ramot
,
M.
,
Walsh
,
C.
, &
Martin
,
A.
(
2019
).
Multifaceted integration: Memory for faces is subserved by widespread connections between visual, memory, auditory, and social networks
.
Journal of Neuroscience
,
39
,
4976
4985
.
Rapcsak
,
S. Z.
(
2019
).
Face recognition
.
Current Neurology and Neuroscience Reports
,
19
,
41
.
Renoult
,
L.
,
Irish
,
M.
,
Moscovitch
,
M.
, &
Rugg
,
M. D.
(
2019
).
From knowing to remembering: The semantic–episodic distinction
.
Trends in Cognitive Sciences
,
23
,
1041
1057
.
Rissman
,
J.
,
Greely
,
H. T.
, &
Wagner
,
A. D.
(
2010
).
Detecting individual memories through the neural decoding of memory states and past experience
.
Proceedings of the National Academy of Sciences, U.S.A.
,
107
,
9849
9854
.
Rossion
,
B.
(
2014
).
Understanding face perception by means of prosopagnosia and neuroimaging
.
Frontiers in Bioscience
,
6
,
258
307
.
Rotshtein
,
P.
,
Henson
,
R. N. A.
,
Treves
,
A.
,
Driver
,
J.
, &
Dolan
,
R. J.
(
2005
).
Morphing Marilyn into Maggie dissociates physical and identity face representations in the brain
.
Nature Neuroscience
,
8
,
107
113
.
Rugg
,
M. D.
, &
Yonelinas
,
A. P.
(
2003
).
Human recognition memory: A cognitive neuroscience perspective
.
Trends in Cognitive Sciences
,
7
,
313
319
.
Schwartz
,
C. E.
,
Wright
,
C. I.
,
Shin
,
L. M.
,
Kagan
,
J.
,
Whalen
,
P. J.
,
McMullin
,
K. G.
, et al
(
2003
).
Differential amygdalar response to novel versus newly familiar neutral faces: A functional MRI probe developed for studying inhibited temperament
.
Biological Psychiatry
,
53
,
854
862
.
Schwartz
,
L.
, &
Yovel
,
G.
(
2016
).
The roles of perceptual and conceptual information in face recognition
.
Journal of Experimental Psychology: General
,
145
,
1493
1511
.
Schwartz
,
L.
, &
Yovel
,
G.
(
2019
).
Learning faces as concepts rather than percepts improves face recognition
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
45
,
1733
1747
.
Sergent
,
J.
,
Ohta
,
S.
, &
MacDonald
,
B.
(
1992
).
Functional neuroanatomy of face and object processing. A positron emission tomography study
.
Brain
,
115
,
15
36
.
Simmons-Martin
,
A.
(
1983
).
Salient features from the literature, with implications for parent–infant programming
.
American Annals of the Deaf
,
128
,
107
117
.
Squire
,
L. R.
, &
Zola-Morgan
,
S.
(
1991
).
The medial temporal lobe memory system
.
Science
,
253
,
1380
1386
.
Sugiura
,
M.
(
2014
).
Neuroimaging studies on recognition of personally familiar people
.
Frontiers in Bioscience
,
19
,
672
686
.
Sugiura
,
M.
,
Mano
,
Y.
,
Sasaki
,
A.
, &
Sadato
,
N.
(
2011
).
Beyond the memory mechanism: Person-selective and nonselective processes in recognition of personally familiar faces
.
Journal of Cognitive Neuroscience
,
23
,
699
715
.
Szpunar
,
K. K.
,
St Jacques
,
P. L.
,
Robbins
,
C. A.
,
Wig
,
G. S.
, &
Schacter
,
D. L.
(
2014
).
Repetition-related reductions in neural activity reveal component processes of mental simulation
.
Social Cognitive and Affective Neuroscience
,
9
,
712
722
.
Taylor
,
M. J.
,
Arsalidou
,
M.
,
Bayless
,
S. J.
,
Morris
,
D.
,
Evans
,
J. W.
, &
Barbeau
,
E. J.
(
2009
).
Neural correlates of personally familiar faces: Parents, partner and own faces
.
Human Brain Mapping
,
30
,
2008
2020
.
Thornton
,
M. A.
, &
Mitchell
,
J. P.
(
2017
).
Consistent neural activity patterns represent personally familiar people
.
Journal of Cognitive Neuroscience
,
29
,
1583
1594
.
Todorov
,
A.
,
Gobbini
,
M. I.
,
Evans
,
K. K.
, &
Haxby
,
J. V.
(
2007
).
Spontaneous retrieval of affective person knowledge in face perception
.
Neuropsychologia
,
45
,
163
173
.
Tsantani
,
M.
,
Kriegeskorte
,
N.
,
McGettigan
,
C.
, &
Garrido
,
L.
(
2019
).
Faces and voices in the brain: A modality-general person-identity representation in superior temporal sulcus
.
Neuroimage
,
201
,
116004
.
Tsukiura
,
T.
,
Mano
,
Y.
,
Sekiguchi
,
A.
,
Yomogida
,
Y.
,
Hoshi
,
K.
,
Kambara
,
T.
, et al
(
2010
).
Dissociable roles of the anterior temporal regions in successful encoding of memory for person identity information
.
Journal of Cognitive Neuroscience
,
22
,
2226
2237
.
Tulving
,
E.
(
1985
).
Memory and consciousness
.
Canadian Psychology
,
26
,
1
12
.
Uncapher
,
M. R.
,
Boyd-Meredith
,
J. T.
,
Chow
,
T. E.
,
Rissman
,
J.
, &
Wagner
,
A. D.
(
2015
).
Goal-directed modulation of neural memory patterns: Implications for fMRI-based memory detection
.
Journal of Neuroscience
,
35
,
8531
8545
.
van den Hurk
,
J.
,
Gentile
,
F.
, &
Jansma
,
B. M.
(
2011
).
What's behind a face: Person context coding in fusiform face area as revealed by multivoxel pattern analysis
.
Cerebral Cortex
,
21
,
2893
2899
.
van Strien
,
N. M.
,
Cappaert
,
N. L. M.
, &
Witter
,
M. P.
(
2009
).
The anatomy of memory: An interactive overview of the parahippocampal-hippocampal network
.
Nature Reviews Neuroscience
,
10
,
272
282
.
Verosky
,
S. C.
,
Todorov
,
A.
, &
Turk-Browne
,
N. B.
(
2013
).
Representations of individuals in ventral temporal cortex defined by faces and biographies
.
Neuropsychologia
,
51
,
2100
2108
.
Vida
,
M. D.
,
Nestor
,
A.
,
Plaut
,
D. C.
, &
Behrmann
,
M.
(
2017
).
Spatiotemporal dynamics of similarity-based neural representations of facial identity
.
Proceedings of the National Academy of Sciences, U.S.A.
,
114
,
388
393
.
Vila
,
J.
,
Morato
,
C.
,
Lucas
,
I.
,
Guerra
,
P.
,
Castro-Laguardia
,
A. M.
, &
Bobes
,
M. A.
(
2019
).
The affective processing of loved familiar faces and names: Integrating fMRI and heart rate
.
PLoS One
,
14
,
e0216057
.
Visconti di Oleggio Castello
,
M.
,
Halchenko
,
Y. O.
,
Guntupalli
,
J. S.
,
Gors
,
J. D.
, &
Gobbini
,
M. I.
(
2017
).
The neural representation of personally familiar and unfamiliar faces in the distributed system for face perception
.
Scientific Reports
,
7
,
12237
.
Wagner
,
A. D.
,
Shannon
,
B. J.
,
Kahn
,
I.
, &
Buckner
,
R. L.
(
2005
).
Parietal lobe contributions to episodic memory retrieval
.
Trends in Cognitive Sciences
,
9
,
445
453
.
Wang
,
Y.
,
Collins
,
J. A.
,
Koski
,
J.
,
Nugiel
,
T.
,
Metoki
,
A.
, &
Olson
,
I. R.
(
2017
).
Dynamic neural architecture for social knowledge retrieval
.
Proceedings of the National Academy of Sciences, U.S.A.
,
114
,
E3305
E3314
.
Weibert
,
K.
,
Harris
,
R. J.
,
Mitchell
,
A.
,
Byrne
,
H.
,
Young
,
A. W.
, &
Andrews
,
T. J.
(
2016
).
An image-invariant neural response to familiar faces in the human medial temporal lobe
.
Cortex
,
84
,
34
42
.
Winston
,
J. S.
,
Henson
,
R. N. A.
,
Fine-Goulden
,
M. R.
, &
Dolan
,
R. J.
(
2004
).
fMRI-adaptation reveals dissociable neural representations of identity and expression in face perception
.
Journal of Neurophysiology
,
92
,
1830
1839
.
Xu
,
X.
, &
Biederman
,
I.
(
2010
).
Loci of the release from fMRI adaptation for changes in facial expression, identity, and viewpoint
.
Journal of Vision
,
10
,
36
.
Yang
,
H.
,
Susilo
,
T.
, &
Duchaine
,
B.
(
2016
).
The anterior temporal face area contains invariant representations of face identity that can persist despite the loss of right FFA and OFA
.
Cerebral Cortex
,
26
,
1096
1107
.
Yonelinas
,
A. P.
(
1994
).
Receiver-operating characteristics in recognition memory: Evidence for a dual-process model
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
20
,
1341
1354
.
Yonelinas
,
A. P.
(
2002
).
The nature of recollection and familiarity: A review of 30 years of research
.
Journal of Memory and Language
,
46
,
441
517
.
Yonelinas
,
A. P.
,
Aly
,
M.
,
Wang
,
W. C.
, &
Koen
,
J. D.
(
2010
).
Recollection and familiarity: Examining controversial assumptions and new directions
.
Hippocampus
,
20
,
1178
1194
.
Young
,
A. W.
, &
Burton
,
A. M.
(
2017
).
Recognizing faces
.
Current Directions in Psychological Science
,
26
,
212
217
.
Young
,
A. W.
, &
Burton
,
A. M.
(
2018
).
Are we face experts?
Trends in Cognitive Sciences
,
22
,
100
110
.
Yovel
,
G.
, &
Kanwisher
,
N.
(
2004
).
Face perception: Domain specific, not process specific
.
Neuron
,
44
,
889
898
.
Yovel
,
G.
, &
O'Toole
,
A. J.
(
2016
).
Recognizing people in motion
.
Trends in Cognitive Sciences
,
20
,
383
395
.
Zhang
,
H.
,
Japee
,
S.
,
Nolan
,
R.
,
Chu
,
C.
,
Liu
,
N.
, &
Ungerleider
,
L. G.
(
2016
).
Face-selective regions differ in their ability to classify facial expressions
.
Neuroimage
,
130
,
77
90
.