Abstract

In two fMRI experiments (n = 44) using tasks with different demands—approach–avoidance versus one-back recognition decisions—we measured the responses to the social value of faces. The face stimuli were produced by a parametric model of face evaluation that reduces multiple social evaluations to two orthogonal dimensions of valence and power [Oosterhof, N. N., & Todorov, A. The functional basis of face evaluation. Proceedings of the National Academy of Sciences, U.S.A., 105, 11087–11092, 2008]. Independent of the task, the response within regions of the occipital, fusiform, and lateral prefrontal cortices was sensitive to the valence dimension, with larger responses to low-valence faces. Additionally, there were extensive quadratic responses in the fusiform gyri and dorsal amygdala, with larger responses to faces at the extremes of the face valence continuum than faces in the middle. In all these regions, participants' avoidance decisions correlated with brain responses, with faces more likely to be avoided evoking stronger responses. The findings suggest that both explicit and implicit face evaluation engage multiple brain regions involved in attention, affect, and decision making.

INTRODUCTION

Faces convey information about person identity, membership in social categories such as gender and age, and the cognitive and emotional states of others. Moreover, faces are evaluated on a large number of social dimensions, including trustworthiness, competence, friendliness, dominance, and threat (Todorov, Said, & Verosky, 2011). These evaluations are made after extremely brief exposure to faces (Todorov, Pakrashi, & Oosterhof, 2009; Bar, Neta, & Linz, 2006; Willis & Todorov, 2006) and affect important social outcomes ranging from sentencing decisions to electoral success (Olivola & Todorov, 2010; Ballew & Todorov, 2007; Eberhardt, Davies, Purdie-Vaughns, & Johnson, 2006; Todorov, Mandisodza, Goren, & Hall, 2005; Blair, Judd, & Chapleau, 2004; Montepare & Zebrowitz, 1998; Mueller & Mazur, 1996; Zebrowitz & McDonald, 1991; Mazur, Mazur, & Keating, 1984).

The main objective of this article is to identify task-invariant brain responses to the social value of faces. To test for such responses, we conducted two experiments with different task demands. We manipulated the task between experiments rather than within experiment to avoid cross-task contamination, in which participation in one task might affect responses in a subsequent task through the priming of specific goals or other mechanisms. In the first experiment, participants performed a forced-choice task that required them to decide whether to approach or avoid individual faces. This is an explicit task of face evaluation, because it requires participants to evaluate faces on an interpersonal dimension that is highly correlated with the perceived valence of faces (Todorov, 2008; Adolphs, Tranel, & Damasio, 1998). In the second experiment, participants performed a one-back recognition task that required them to decide whether each face's identity is the same as or different from the previous face. This is an implicit task with respect to face evaluation, because it does not require participants to evaluate the faces. All experimental conditions, including face stimuli, were the same in the two experiments.

Because social judgments from faces are highly correlated, we used a validated model of face evaluation, which reduces multiple social evaluations to two underlying dimensions (Oosterhof & Todorov, 2008; Todorov, Said, Engell, & Oosterhof, 2008). Specifically, previous work has found that more than 80% of the variance in a large set of social judgments of faces can be accounted for by two principal components, one interpreted as face valence and the other interpreted as face power (Oosterhof & Todorov, 2008; Todorov, Said, et al., 2008). Oosterhof and Todorov (2008) used a statistical model of face representation to build a computer-generated imagery model of face evaluation. This model can generate an unlimited number of faces and can parametrically manipulate these faces along the two orthogonal dimensions of valence and power. In the two experiments, we investigate the brain regions responsible for face evaluation by presenting participants with faces that systematically varied along these two orthogonal dimensions (see Figure 1).

Figure 1. 

Two-dimensional space of faces varying on valence and power. Faces were created from one identity. Units are in standard deviations.

Figure 1. 

Two-dimensional space of faces varying on valence and power. Faces were created from one identity. Units are in standard deviations.

We were particularly interested in the amygdala response to the variation of the faces, because this region (a) has been implicated in a number of previous studies on face evaluation (Kennedy, Glascher, Tyszka, & Adolphs, 2009; Said, Baron, & Todorov, 2009; Todorov & Engell, 2008; Engell, Haxby, & Todorov, 2007; Winston, Strange, O'Doherty, & Dolan, 2002; Adolphs et al., 1998) and (b) is likely involved in bottom–up attentional effects driven by the affective significance of stimuli (Vuilleumier, 2005). Given that some studies have reported that the amygdala responds linearly to the valence of faces, with greater responses to low- than high-valence faces (Todorov & Engell, 2008; Engell et al., 2007; Winston et al., 2002), whereas other studies have reported quadratic, nonmonotonic response profiles, with greater activation to the extremes of face valence compared with faces in the middle of the continuum (Said, Dotsch, & Todorov, 2010; Said, Baron, et al., 2009), we model both linear and quadratic responses to the dimensions.

To identify regions in the brain that responded similarly to the faces in both tasks, we performed a conjunction analysis for all effects that were significant in both experiments. We also explored the relationship between the brain responses in these conjunction regions and the behavioral responses from the two experiments. We were specifically interested in testing whether avoidance decisions predicted brain responses, because these decisions are highly related to face evaluation (Todorov, 2008).

METHODS

Face Model and Face Stimuli

Each participant was shown faces that were generated from 1 of 20 white faces created in FaceGen (Singular Inversions, Toronto, Ontario, Canada). Each of the 20 faces was then used as an “anchor” face to create an additional set of 16 faces that varied along the dimensions of valence and power (Figure 1). Each participant viewed only the 16 variations created from a single anchor face. To avoid racial stereotype effects, all anchor faces were White. We used male faces, because FaceGen generates faces without scalp hair and male bald faces look more natural than female bald faces. Using the approach described in Oosterhof and Todorov (2008), we created models of face valence and face power, which were defined as the first two principal components derived from a principal components analysis of social judgments (see Table S6 in Oosterhof & Todorov, 2008). The two vectors can be added to any face to change its predicted value on valence and power, respectively (for details, see Said et al., 2010). The 16 versions were constructed by varying the valence and power controls over (−3, −1, +1, +3) SD units independently.

fMRI Participants

Twenty-three participants (12 women, mean age = 20.9, SD = 2.9) participated in Experiment 1. One participant was excluded from the analysis for moving his head more than 5 mm from its initial position in the y and z directions. Twenty-five participants (13 women, mean age = 21.3, SD = 3.1) participated in Experiment 2. Three participants were excluded for moving their head more than 5 mm in any spatial direction over the course of the experiment.

fMRI Task

In Experiment 1, participants performed a forced-choice task in which they decided on each trial whether they would like to approach or avoid the face. In Experiment 2, participants performed a one-back recognition task in which they decided whether each face was the same as or different from the preceding face. The decisions did not influence the stimulus presentation. Participants responded with a button box in their right hand.

Each participant was presented with a Type 1–Index 1 sequence of faces (Aguirre, 2007; Finney & Outhwaite, 1956). This type of sequence ensured that every face was preceded by every other face an equal number of times for each participant. Each face was presented for 1450 msec and followed by a 50-msec fixation intertrial interval. Sixty-eight rest trials, each 3000-msec long, were distributed throughout the sequence and were included in the order counterbalancing. To increase participant comfort, the experiment was divided into eight “runs” of 4 min and 20 sec each. The number of trials corresponding to each type of face as well as the number of rests were equally balanced among Runs 1–2, Runs 3–4, Runs 5–6, and Runs 7–8. To allow time for the MRI signal to reach steady-state equilibrium and for the participants to orient to the task, each run began with 11 dummy trials, which were excluded from the analysis. The 11 dummy faces were the same as the last 11 faces of the previous run, thus reinstating the adaptation effect for the first valid trial of every run.

fMRI Data Acquisition

The BOLD signal was used as a measure of neural activation (Kwong et al., 1992; Ogawa, Lee, Nayak, & Glynn, 1990). EPIs were acquired with a Siemens 3.0 Tesla Allegra scanner (Siemens, Erlangen, Germany). Functional data were obtained at a resolution of 3 × 3 × 4 mm3 (TR = 2000 msec, TE = 30 msec, flip angle = 80°, field of view = 192 × 192 mm2). For the purpose of cross-participant spatial registration, a whole-brain high-resolution T1-weighted structural scan was acquired at the end of each experiment (TR = 2500 msec, TE = 33 msec, flip angle = 8°, 2 × 1 × 1 mm3 voxel size).

fMRI Data Analysis

Image analysis was performed with Analysis of Functional Neuroimages (AFNI; Cox, 1996). After discarding the first five functional images from each run to allow the MRI signal to reach steady-state equilibrium, the remaining images were slice time-corrected and then motion-corrected to the last image of the last run using a six-parameter 3-D motion correction algorithm. Next, the functional data were cleaned of outliers using the AFNI program 3dDespike, smoothed with an 8-mm FWHM Gaussian kernel, and then normalized to percent signal change from the mean.

The anatomical data were aligned to the unsmoothed functional data using the AFNI program align_epi_anat.py and then transformed to Talairach space using the function @auto_tlrc. The transformation matrix created by the prior step was used to put each functional data set into Talairach space.

To determine the effects of valence and power, the general linear model was used. The set of regressors consisted of (1) linear effects of valence, (2) quadratic effects of valence, (3) linear effects of power, (4) quadratic effects of power, (5) linear interaction effects, (6) quadratic interaction effects, (7) a “stimulus on” regressor, indicating whenever a face was present, (8) a dummy regressor for the first 11 trials, and (9–14) six regressors that accounted for participant head motion. All regressors except the dummy regressors and the motion regressors were orthogonal to each other. Regressors 1–8 were then convolved with a canonical γ-variate hemodynamic response function before being entered into the model.

To determine cluster size significance, we performed Monte Carlo simulations of null hypothesis data using the AFNI program AlphaSim. The program first generates multiple images of the null hypothesis data and then accounts for the spatial correlation in the real data by smoothing each null image with a Gaussian kernel corresponding to the smoothness of the residual image from the group t test. We then used the program to determine the distribution of cluster sizes defined by a voxelwise threshold of p < .001. Whole-brain simulations revealed that less than 5% of null clusters exceeded 810 mm3 (90 functional voxels), which we, therefore, used as our cutoff for significance. Because we had an a priori hypothesis about the amygdala, we performed separate simulations in both the left and right amygdalae. Both simulations revealed a cluster size cutoff of 27 mm3 or three contiguous voxels.

Next, we tested for the significance of coefficients extracted from the polynomial regression on face effects. Specifically, the coefficients were submitted to single sample t tests to determine if they were significant. A t test was performed for each of the following variables: linear valence, quadratic valence, linear power, quadratic power, linear interaction, and quadratic interaction. Voxel clusters identified by these analyses were later used in ROI analyses (see below).

To illustrate the face effects, we defined functional ROIs (fROIs) on the basis of the analyses described above. From each fROI, we extracted the average regression coefficient for each of the 16 faces (Figure 1), which varied on four levels of valence and four levels of power. These coefficients were obtained with a general linear model applied to each participant. The model contained one regressor for each of the 16 faces, a dummy regressor for the first 11 trials of each run, and six motion regressors.

Analysis across Experiments

To identify regions in the brain that responded similarly to the faces in both experiments, that is, independent of the task of the participants, we performed a conjunction analysis for all effects that were significant in both experiments. Specifically, for the main linear and quadratic effects as well as interaction effects, voxels were thresholded at p < .001 in both experiments, and clusters that consisted of voxels surviving the threshold in both experiments were identified. The resulting clusters had a conjoint probability of .000001. All fusiform and amygdala clusters were intersected with anatomical masks from the AFNI Talairach atlas.

RESULTS

Behavioral Results

Both avoidance (Experiment 1) and recognition (Experiment 2) decisions were submitted to a 4 (Valence) × 4 (Power) repeated measures ANOVA.

Experiment 1

As shown in Figure 2, at each level of power, participants were more likely to make avoidance decisions for low- than high-valence faces, F(3, 63) = 190.70, p < .001. The probability of avoidance decisions increased monotonically with the increase in face negativity. The linear component of face valence was large and significant, F(1, 21) = 263.45, p < .001. There was also a significant quadratic component, F(1, 21) = 29.34, p < .001, indicating that avoidance decisions were more sensitive to changes in valence at the negative than at the positive end of the valence continuum.

Figure 2. 

Behavioral responses in the approach–avoidance task of Experiment 1. Each subplot shows the fraction of “avoidance” responses at each level of valence. (A) Faces with power/dominance = −3 SD. (B) Faces with power/dominance = −1 SD. (C) Faces with power/dominance = 1 SD. (D) Faces with power/dominance = 3 SD.

Figure 2. 

Behavioral responses in the approach–avoidance task of Experiment 1. Each subplot shows the fraction of “avoidance” responses at each level of valence. (A) Faces with power/dominance = −3 SD. (B) Faces with power/dominance = −1 SD. (C) Faces with power/dominance = 1 SD. (D) Faces with power/dominance = 3 SD.

The analysis also revealed a significant main effect of Power, F(3, 63) = 12.14, p < .001, modified by a significant interaction with Valence, F(9, 189) = 17.26, p < .001. This interaction indicated that the effect of valence on avoidance decisions increased in magnitude with the increase in the power of faces.

Experiment 2

In the one-back recognition task, participants had an average hit rate of .74 (SD = .16), an average false positive rate of .09 (SD = .04), and an average d-prime of 2.1 (SD = .47). We used hit rate instead of d-prime as our measure because d-prime was undefined for many of the cells because of ceiling performance. The analysis revealed that both main effects were significant, F(3, 63) = 10.52, p < .001 and F(3, 63) = 7.02, p < .001, for Valence and Power, respectively. There were no other significant effects.

For both dimensions, the hit rate was higher for faces at the extremes than at the middle of the continuum. Correspondingly, the quadratic components for both dimensions were significant, F(1, 21) = 18.43, p < .001 and F(1, 21) = 19.71, p < .001, for Valence and Power, respectively. The linear component for valence was also significant, F(1, 21) = 7.89, p < .011, indicating higher hit rate for high- than low-valence faces.

fMRI Results

Experiment 1

Large regions of the brain showed a significant negative linear response to face valence, with greater activation to low- than high-valence faces (Figure 3A). The largest clusters showing this response encompassed the fusiform gyri, lateral occipital cortex, and the inferior parietal lobules (Table 1). We also found several clusters showing activation in bilateral DLPFC.

Figure 3. 

Responses to face valence in Experiment 1 for regions showing a significant linear effect. (A) Axial, coronal, and sagittal slices at x = 32, y = -2, z = −12 showing brain regions with a linear response to face valence, thresholded at voxelwise p < .001. Blue–green color coding indicates that brain responses in these areas decreased with face valence. (B) Response in the bilateral fusiform gyri to valence, overlaid with a best linear fit (least squares). ROI was functionally defined by its linear response to valence at voxelwise p < .001 and intersected with an anatomical mask of the fusiform gyrus extracted from the AFNI Talairach atlas. Note that this region overlapped heavily with regions showing a quadratic response (see Figure 4).

Figure 3. 

Responses to face valence in Experiment 1 for regions showing a significant linear effect. (A) Axial, coronal, and sagittal slices at x = 32, y = -2, z = −12 showing brain regions with a linear response to face valence, thresholded at voxelwise p < .001. Blue–green color coding indicates that brain responses in these areas decreased with face valence. (B) Response in the bilateral fusiform gyri to valence, overlaid with a best linear fit (least squares). ROI was functionally defined by its linear response to valence at voxelwise p < .001 and intersected with an anatomical mask of the fusiform gyrus extracted from the AFNI Talairach atlas. Note that this region overlapped heavily with regions showing a quadratic response (see Figure 4).

Table 1. 

Locations of Clusters Showing Linear and Quadratic Effects of Face Valence and Power and Their Linear Interaction in Experiment 1


Volume (mm3)
x
y
z
Regions Showing Linear Effects of Face Valence 
Right fusiform gyrus, occipital, and inferior parietal lobule 42,849 46.5 −67.5 −15.5 
Left fusiform gyrus, occipital 23,922 −31.5 −76.5 −18.5 
Left inferior parietal lobule 6993 −40.5 −49.5 53.5 
Right inferior frontal gyrus 6372 40.5 19.5 2.5 
Right middle frontal gyrus 4725 52.5 10.5 41.5 
Left insula 2862 −34.5 16.5 5.5 
Left middle frontal gyrus 2511 −43.5 46.5 20.5 
Left middle frontal gyrus 1593 −55.5 13.5 35.5 
Right inferior frontal gyrus 1053 52.5 4.5 20.5 
Left precentral gyrus 918 −43.5 −10.5 53.5 
 
Region Showing Quadratic Effects of Face Valence 
Left fusiform gyrus, left lateral occipital lobe 9234 −40.5 −76.5 −18.5 
Right middle occipital gyrus 5643 52.5 −67.5 −6.5 
Right fusiform gyrus 2997 31.5 −52.5 −18.5 
Right amygdala 999 19.5 1.5 −12.5 
 
Regions Showing Linear Effects of Face Power 
Right lingual gyrus 7506 7.5 −85.5 −12.5 
Left middle frontal gyrus 1161 −25.5 −58.5 8.5 
 
Region Showing Quadratic Effects of Face Power 
Left fusiform gyrus 1944 −37.5 −52.5 −18.5 
Right middle occipital gyrus 999 −46.5 −76.5 6.5 
 
Regions Showing a Linear Interaction of Face Valence and Power 
Left posterior middle temporal gyrus 1647 −52.5 −70.5 20.5 
Right middle occipital gyrus 999 34.5 −85.5 14.5 
Left anterior middle temporal gyrus 972 −64.5 −28.5 −3.5 

Volume (mm3)
x
y
z
Regions Showing Linear Effects of Face Valence 
Right fusiform gyrus, occipital, and inferior parietal lobule 42,849 46.5 −67.5 −15.5 
Left fusiform gyrus, occipital 23,922 −31.5 −76.5 −18.5 
Left inferior parietal lobule 6993 −40.5 −49.5 53.5 
Right inferior frontal gyrus 6372 40.5 19.5 2.5 
Right middle frontal gyrus 4725 52.5 10.5 41.5 
Left insula 2862 −34.5 16.5 5.5 
Left middle frontal gyrus 2511 −43.5 46.5 20.5 
Left middle frontal gyrus 1593 −55.5 13.5 35.5 
Right inferior frontal gyrus 1053 52.5 4.5 20.5 
Left precentral gyrus 918 −43.5 −10.5 53.5 
 
Region Showing Quadratic Effects of Face Valence 
Left fusiform gyrus, left lateral occipital lobe 9234 −40.5 −76.5 −18.5 
Right middle occipital gyrus 5643 52.5 −67.5 −6.5 
Right fusiform gyrus 2997 31.5 −52.5 −18.5 
Right amygdala 999 19.5 1.5 −12.5 
 
Regions Showing Linear Effects of Face Power 
Right lingual gyrus 7506 7.5 −85.5 −12.5 
Left middle frontal gyrus 1161 −25.5 −58.5 8.5 
 
Region Showing Quadratic Effects of Face Power 
Left fusiform gyrus 1944 −37.5 −52.5 −18.5 
Right middle occipital gyrus 999 −46.5 −76.5 6.5 
 
Regions Showing a Linear Interaction of Face Valence and Power 
Left posterior middle temporal gyrus 1647 −52.5 −70.5 20.5 
Right middle occipital gyrus 999 34.5 −85.5 14.5 
Left anterior middle temporal gyrus 972 −64.5 −28.5 −3.5 

Regions with linear effects all had negative linear coefficients, and regions with quadratic effects all had positive quadratic coefficients. All cluster sizes are significant at p < .05, corrected for multiple comparisons.

Plots illustrating the responses to individual faces in the fusiform gyrus showed that, although there was an overall downward trend as face valence increased, the function was curvilinear, with strong responses to both very low and very high valence faces (Figure 3B). In fact, several areas showed quadratic responses to face valence with stronger responses to faces at the extreme than at the middle of the continuum (Figure 4A). These areas included the bilateral fusiform gyri, bilateral occipital cortex, and the right amygdala (Table 1). The responses to face valence in the fusiform and amygdala are illustrated in Figure 4B and C, respectively.

Figure 4. 

Responses to face valence in Experiment 1 for regions showing a significant quadratic effect. (A) Axial, coronal, and sagittal slices at x = 32, y = -2, z = −12, showing brain regions with a quadratic response to face valence, thresholded at voxelwise p < .001. Red color coding indicates that brain response in these areas was highest for the extremes of valence. (B) Response in the bilateral fusiform gyri to valence, overlayed with a best-fit (least squares) second-order polynomial. ROI was functionally defined by its quadratic response to valence at voxelwise p < .001 and intersected with an anatomical mask of the fusiform gyrus extracted from the AFNI Talairach atlas. (C) Response in the amygdala. This region was also functionally defined by its quadratic response and intersected with an anatomical mask of the amygdala from the AFNI Talairach atlas.

Figure 4. 

Responses to face valence in Experiment 1 for regions showing a significant quadratic effect. (A) Axial, coronal, and sagittal slices at x = 32, y = -2, z = −12, showing brain regions with a quadratic response to face valence, thresholded at voxelwise p < .001. Red color coding indicates that brain response in these areas was highest for the extremes of valence. (B) Response in the bilateral fusiform gyri to valence, overlayed with a best-fit (least squares) second-order polynomial. ROI was functionally defined by its quadratic response to valence at voxelwise p < .001 and intersected with an anatomical mask of the fusiform gyrus extracted from the AFNI Talairach atlas. (C) Response in the amygdala. This region was also functionally defined by its quadratic response and intersected with an anatomical mask of the amygdala from the AFNI Talairach atlas.

The effects of face power were generally less extensive than the effects of face valence (Table 1). We found two clusters with a negative linear response to power, one in the right lingual gyrus and the other in the posterior cingulate. Two small clusters showed a quadratic response to power, with greater responses to extreme than to moderate power. The clusters were in the left fusiform gyrus and in the left middle occipital gyrus.

Three clusters showed a linear interaction between valence and power (Table 1). These were in the left anterior middle temporal gyrus, the left posterior middle temporal gyrus, and the right middle occipital gyrus. In the left anterior and left posterior middle temporal gyri, there was a larger response to low- than to high-valence high-power faces (1 and 3 SD), whereas there was a larger response to high- than to low-valence low-power faces (−1 and −3 SD). In the middle occipital gyrus, there was a larger response to low- than to high-valence faces, and this effect increased as a function of increasing face power. This pattern of responses resembled the pattern of the participants' avoidance decisions.

Experiment 2

As in Experiment 1, we found regions in bilateral fusiform, occipital, and pFC that showed a negative linear response to face valence (Figure 5A and Table 2). The linear trends in the fusiform and occipital cortex were typically accompanied by quadratic components, so that the function relating neural response to valence was nonmonotonic (Figure 5B). A test for quadratic responses across the brain showed several active regions, including the bilateral fusiform gyri, bilateral occipital cortex, and the bilateral dorsal amygdala/ventral putamen (Figure 6 and Table 2). The quadratic regression coefficients in these areas were always positive, indicating that the response was stronger for the extremes than for the moderate levels of valence. There were no significant voxels responding linearly or quadratically to face power at a voxelwise threshold of p < .001.

Figure 5. 

Responses to face valence in Experiment 2 for regions showing a significant linear effect. (A) Axial, coronal, and sagittal slices at x = −24, y = −4, z = −13 showing brain regions with a linear response to face valence, thresholded at voxelwise p < .001. Blue–green color coding indicates that brain responses in these areas decreased with face valence. (B) Response in the bilateral fusiform gyri to valence, overlayed with a best linear fit (least squares). ROI was functionally defined by its linear response to valence at voxelwise p < .001 and intersected with an anatomical mask of the fusiform gyrus extracted from the AFNI Talairach atlas. Note that this region overlapped heavily with regions showing a quadratic response (see Figure 6).

Figure 5. 

Responses to face valence in Experiment 2 for regions showing a significant linear effect. (A) Axial, coronal, and sagittal slices at x = −24, y = −4, z = −13 showing brain regions with a linear response to face valence, thresholded at voxelwise p < .001. Blue–green color coding indicates that brain responses in these areas decreased with face valence. (B) Response in the bilateral fusiform gyri to valence, overlayed with a best linear fit (least squares). ROI was functionally defined by its linear response to valence at voxelwise p < .001 and intersected with an anatomical mask of the fusiform gyrus extracted from the AFNI Talairach atlas. Note that this region overlapped heavily with regions showing a quadratic response (see Figure 6).

Table 2. 

Locations of Clusters Showing Linear and Quadratic Effects of Face Valence and Linear and Quadratic Interactions between Face Valence and Face Power in Experiment 2


Volume (mm3)
x
y
z
Regions Showing Linear Effects of Face Valence 
Right fusiform gyrus, occipital 16,416 46.5 −64.5 −15.5 
Right middle frontal gyrus 11,367 46.5 31.5 32.5 
Left fusiform gyrus, occipital 10,962 −49.5 −61.5 −15.5 
Left middle frontal gyrus 6858 −40.5 52.5 −0.5 
Left superior parietal lobule 3699 −28.5 −76.5 44.5 
Right middle frontal gyrus 3618 37.5 13.5 53.5 
Left superior frontal gyrus 3105 −1.5 43.5 50.5 
Left cingulate gyrus 2538 −1.5 −13.5 32.5 
Right middle temporal gyrus 1944 55.5 −40.5 −12.5 
Left superior frontal gyrus 945 −31.5 13.5 50.5 
Left middle frontal gyrus 810 −43.5 4.5 53.5 
 
Regions Showing Quadratic Effects of Face Valence 
Bilateral fusiform gyri, bilateral lateral occipital lobe, hippocampus, left dorsal amygdala, left putamen 98,226 37.5 −79.5 −15.5 
Right dorsal amygdala and putamen 5940 19.5 1.5 −6.5 
Left superior parietal lobule 3996 −16.5 −58.5 62.5 
Right postcentral gyrus 2349 19.5 −34.5 71.5 
Right postcentral gyrus 1512 46.5 −19.5 47.5 
Right middle cingulate cortex 1026 4.5 −34.5 44.5 
Left putamen 972 −28.5 −13.5 8.5 
 
Regions Showing a Linear Interaction of Face Valence and Power 
Left fusiform gyrus 2133 −28.5 −85.5 −15.5 
Left superior occipital gyrus 1188 −40.5 −79.5 35.5 
Superior medial frontal gyrus 1134 −52.5 22.5 23.5 
Right medial frontal gyrus 864 1.5 43.5 41.5 
Right middle temporal gyrus 810 64.5 −13.5 −6.5 
 
Regions Showing a Quadratic Interaction of Face Valence and Power 
Right inferior frontal gyrus 9855 52.5 19.5 −0.5 
Right supramarginal gyrus 2187 58.5 −46.5 26.5 

Volume (mm3)
x
y
z
Regions Showing Linear Effects of Face Valence 
Right fusiform gyrus, occipital 16,416 46.5 −64.5 −15.5 
Right middle frontal gyrus 11,367 46.5 31.5 32.5 
Left fusiform gyrus, occipital 10,962 −49.5 −61.5 −15.5 
Left middle frontal gyrus 6858 −40.5 52.5 −0.5 
Left superior parietal lobule 3699 −28.5 −76.5 44.5 
Right middle frontal gyrus 3618 37.5 13.5 53.5 
Left superior frontal gyrus 3105 −1.5 43.5 50.5 
Left cingulate gyrus 2538 −1.5 −13.5 32.5 
Right middle temporal gyrus 1944 55.5 −40.5 −12.5 
Left superior frontal gyrus 945 −31.5 13.5 50.5 
Left middle frontal gyrus 810 −43.5 4.5 53.5 
 
Regions Showing Quadratic Effects of Face Valence 
Bilateral fusiform gyri, bilateral lateral occipital lobe, hippocampus, left dorsal amygdala, left putamen 98,226 37.5 −79.5 −15.5 
Right dorsal amygdala and putamen 5940 19.5 1.5 −6.5 
Left superior parietal lobule 3996 −16.5 −58.5 62.5 
Right postcentral gyrus 2349 19.5 −34.5 71.5 
Right postcentral gyrus 1512 46.5 −19.5 47.5 
Right middle cingulate cortex 1026 4.5 −34.5 44.5 
Left putamen 972 −28.5 −13.5 8.5 
 
Regions Showing a Linear Interaction of Face Valence and Power 
Left fusiform gyrus 2133 −28.5 −85.5 −15.5 
Left superior occipital gyrus 1188 −40.5 −79.5 35.5 
Superior medial frontal gyrus 1134 −52.5 22.5 23.5 
Right medial frontal gyrus 864 1.5 43.5 41.5 
Right middle temporal gyrus 810 64.5 −13.5 −6.5 
 
Regions Showing a Quadratic Interaction of Face Valence and Power 
Right inferior frontal gyrus 9855 52.5 19.5 −0.5 
Right supramarginal gyrus 2187 58.5 −46.5 26.5 

Regions with linear effects all had negative linear coefficients, and regions with quadratic effects all had positive quadratic coefficients. All cluster sizes are significant at p < .05, corrected for multiple comparisons.

Figure 6. 

Responses to face valence in Experiment 2 for regions showing a significant quadratic effect. (A) Axial, coronal, and sagittal slices at x = −24, y = −4, z = −13, showing brain regions with a quadratic response to face valence, thresholded at voxelwise p < .001. Red color coding indicates that brain responses in these areas was highest for the extremes of valence. (B) Response in the bilateral fusiform gyri to valence, overlayed with a best-fit (least squares) second-order polynomial. ROI was functionally defined by its quadratic response to valence at voxelwise p < .001 and intersected with an anatomical mask of the fusiform gyrus extracted from the AFNI Talairach atlas. (C) Response in the bilateral amygdala. This region was also functionally defined by its quadratic response and intersected with an anatomical mask of the amygdala from the AFNI Talairach atlas.

Figure 6. 

Responses to face valence in Experiment 2 for regions showing a significant quadratic effect. (A) Axial, coronal, and sagittal slices at x = −24, y = −4, z = −13, showing brain regions with a quadratic response to face valence, thresholded at voxelwise p < .001. Red color coding indicates that brain responses in these areas was highest for the extremes of valence. (B) Response in the bilateral fusiform gyri to valence, overlayed with a best-fit (least squares) second-order polynomial. ROI was functionally defined by its quadratic response to valence at voxelwise p < .001 and intersected with an anatomical mask of the fusiform gyrus extracted from the AFNI Talairach atlas. (C) Response in the bilateral amygdala. This region was also functionally defined by its quadratic response and intersected with an anatomical mask of the amygdala from the AFNI Talairach atlas.

As shown in Table 2, five regions—left superior occipital gyrus, left fusiform gyrus, left anterior middle temporal gyrus, left middle frontal gyrus, and superior medial frontal gyrus—showed a significant linear interaction of valence and power. The pattern of responses for all regions showing the linear interaction was similar. For all levels of face power except for extremely low-power faces (−3 SD), the response to face valence was negative with stronger responses to low-valence faces. This trend was reversed for extremely low-power faces. For the other levels of power, the magnitude of the negative trend appeared to increase with higher levels of power, resembling the pattern of the avoidance decisions in Experiment 1. Two additional regions—right supramarginal gyrus and right inferior frontal gyrus—showed a significant quadratic interaction (Table 2), but there was no clear interpretation of this interaction.

Conjunction Analysis across Both Experiments

In both experiments, regions in the bilateral middle occipital, fusiform, and inferior frontal gyri showed a negative linear response to valence with a stronger response to low- than high-valence faces (Table 3). The regions showing quadratic responses to valence were in the bilateral fusiform gyri and bilateral dorsal amygdalae.

Table 3. 

Locations of Clusters Showing Linear and Quadratic Effects of Face Valence in Both Experiments


Volume (mm3)
x
y
z
Regions Showing Linear Effects 
Right fusiform gyrus, right middle occipital gyrus 9774 34.5 −40.5 −18.5 
Left middle occipital gyrus 3132 −37.5 −82.5 −6.5 
Left fusiform gyrus 1323 −34.5 −55.5 −15.5 
Right inferior frontal gyrus 810 52.5 28.5 17.5 
Left inferior frontal gyrus 594 −43.5 37.5 11.5 
Left cerebellum 243 −13.5 −64.5 −27.5 
 
Regions Showing Quadratic Effects 
Left fusiform gyrus, left middle occipital gyrus 7965 −40.5 −37.5 −18.5 
Right fusiform gyrus, right middle occipital gyrus 5292 43.5 −61.5 −9.5 
Right anterior fusiform gyrus 2538 40.5 −37.5 −21.5 
Right amygdala extending into lentiform nucleus 486 25.5 1.5 −9.5 
Left amygdala extending into lentiform nucleus 243 −19.5 1.5 −9.5 

Volume (mm3)
x
y
z
Regions Showing Linear Effects 
Right fusiform gyrus, right middle occipital gyrus 9774 34.5 −40.5 −18.5 
Left middle occipital gyrus 3132 −37.5 −82.5 −6.5 
Left fusiform gyrus 1323 −34.5 −55.5 −15.5 
Right inferior frontal gyrus 810 52.5 28.5 17.5 
Left inferior frontal gyrus 594 −43.5 37.5 11.5 
Left cerebellum 243 −13.5 −64.5 −27.5 
 
Regions Showing Quadratic Effects 
Left fusiform gyrus, left middle occipital gyrus 7965 −40.5 −37.5 −18.5 
Right fusiform gyrus, right middle occipital gyrus 5292 43.5 −61.5 −9.5 
Right anterior fusiform gyrus 2538 40.5 −37.5 −21.5 
Right amygdala extending into lentiform nucleus 486 25.5 1.5 −9.5 
Left amygdala extending into lentiform nucleus 243 −19.5 1.5 −9.5 

Voxels were thresholded at p < .001 in both experiments, and clusters were identified that consisted of voxels surviving the threshold in both experiments with a conjoint probability of .000001. No voxels survived this conjunction analysis when it was applied to face dominance or interaction effects.

Relationship between Brain and Behavioral Responses

We further explored the relationship between the brain responses in these conjunction ROIs and the behavioral responses from Experiments 1 and 2. Although variations of the faces on the valence and power dimensions predicted both behavioral and brain responses, it is not clear whether these responses are related. We were specifically interested in exploring the relationship between avoidance decisions and brain responses, because these decisions are highly related to face evaluation. To test whether the behavioral responses predicted the activation in the ROIs, for each participant we computed the correlation between their avoidance decisions for each of the 16 faces and the parameter estimates for the faces in the ROIs. Then, we tested whether the average correlations were significantly different from zero. The tests were performed on the Fisher transformed correlation coefficients. We performed similar analysis for the performance on the one-back task. Finally, we compared the magnitudes of the brain correlations with the avoidance and recognition decisions. When the average correlations had different signs (e.g., negative correlation for avoidance but positive for recognition), we first recoded one of the correlations (e.g., multiplying by −1 the avoidance correlations) and then compared the Fisher transformed correlations with an independent t test.

For all fROIs showing linear or quadratic effects (Table 3), we correlated the avoidance decisions with the parameter estimates for the fROIs in Experiment 1 and the correct recognition decisions (hit rate) with the parameter estimates for the fROIs in Experiment 2. Given the shape of the behavioral responses in the two experiments, one might predict that regions showing a linear response would predict avoidance decisions and that regions showing a quadratic response would predict recognition decisions. However, as shown in Table 4, in both cases, the correlations were stronger for the avoidance decisions. In all regions, stronger brain responses corresponded to faces for which participants were more likely to make avoidance decisions. For several of the regions—bilateral amygdala, occipital and fusiform gyri—there was also a positive relationship with the recognition decisions. Stronger brain responses corresponded to faces for which participants were more likely to make correct recognition decisions. However, with the exception of the amygdala, the magnitude of the correlations was stronger for avoidance decisions than for recognition decisions (Table 4).

Table 4. 

Mean Individual Correlations (Standard Deviations and t Values of Tests of Correlations against Zero) between Parameter Estimates in Functionally Defined Regions of Interests from a Conjunction Analysis of Experiments 1 and 2 (See Table 3) and Behavioral Responses in Experiment 1 and Experiment 2


Experiment 1
Experiment 2
Experiment 1 vs. Experiment 2
Avoidance Decisions
Correct Recognition
t and p Values
fROIs Showing Linear Effects 
Bilateral inferior frontal gyri .30 (.21), 6.60** −.01 (.28), 0.12 4.12, p < .001 
Bilateral fusiform gyri .36 (.19), 8.46** .15 (.22), 3.32* 3.49, p < .001 
Bilateral occipital gyri .41 (.17), 10.04** .16 (.21), 3.59* 4.37, p < .001 
 
fROIs Showing Quadratic Effects 
Bilateral fusiform gyri .32 (.23), 6.25** .18 (.24), 3.53* 2.07, p < .044 
Bilateral occipital gyri .40 (.17), 9.65** .17 (.21), 3.71* 4.03, p < .001 
Bilateral amygdala .19 (.23), 3.80** .18 (.14), 5.94** 0.36, p = .72 

Experiment 1
Experiment 2
Experiment 1 vs. Experiment 2
Avoidance Decisions
Correct Recognition
t and p Values
fROIs Showing Linear Effects 
Bilateral inferior frontal gyri .30 (.21), 6.60** −.01 (.28), 0.12 4.12, p < .001 
Bilateral fusiform gyri .36 (.19), 8.46** .15 (.22), 3.32* 3.49, p < .001 
Bilateral occipital gyri .41 (.17), 10.04** .16 (.21), 3.59* 4.37, p < .001 
 
fROIs Showing Quadratic Effects 
Bilateral fusiform gyri .32 (.23), 6.25** .18 (.24), 3.53* 2.07, p < .044 
Bilateral occipital gyri .40 (.17), 9.65** .17 (.21), 3.71* 4.03, p < .001 
Bilateral amygdala .19 (.23), 3.80** .18 (.14), 5.94** 0.36, p = .72 

The last column compares the magnitude of the correlations in the two experiments.

*p < .01.

**p < .001.

We also conducted correlation analyses at the level of the faces averaging across the participants' avoidance decisions from Experiment 1 (Figure 7). This averaging procedure allowed us to conduct the same analyses for Experiment 2. That is, although participants in the latter experiment did not make avoidance decisions, we could correlate the average brain responses to the faces in this experiment with the average avoidance face scores that were derived from Experiment 1. As shown in Table 5, for all regions, the correlations were highly stable across the two experiments.

Figure 7. 

Scatter plots of avoidance decisions and parameter estimates in functionally defined ROIs from a conjunction analysis of Experiments 1 and 2 (see Table 3). Avoidance decisions and parameter estimates in (A) bilateral occipital gyri (r = .64, p < .008), (B) bilateral fusiform gyri (r = .61, p < .012), (C) bilateral inferior frontal gyri (r = .61, p < .012), and (D) bilateral amygdala (r = .44, p < .088). Each point represents 1 of the 16 faces from the 4 (Valence) × 4 (Power) matrix, and the line represents the best linear fit. For these scatter plots, behavioral and brain responses were averaged across participants.

Figure 7. 

Scatter plots of avoidance decisions and parameter estimates in functionally defined ROIs from a conjunction analysis of Experiments 1 and 2 (see Table 3). Avoidance decisions and parameter estimates in (A) bilateral occipital gyri (r = .64, p < .008), (B) bilateral fusiform gyri (r = .61, p < .012), (C) bilateral inferior frontal gyri (r = .61, p < .012), and (D) bilateral amygdala (r = .44, p < .088). Each point represents 1 of the 16 faces from the 4 (Valence) × 4 (Power) matrix, and the line represents the best linear fit. For these scatter plots, behavioral and brain responses were averaged across participants.

Table 5. 

Correlations between Brain Responses to Faces in Functionally Defined Regions of Interests from a Conjunction Analysis of Experiments 1 and 2 (See Table 3) and Avoidance Decisions


Experiment 1
Experiment 2
fROIs Showing Linear Effects 
Bilateral inferior frontal gyri −.61* −.69* 
Bilateral fusiform gyri −.61* −.58* 
Bilateral occipital gyri −.64* −.58* 
 
fROIs Showing Quadratic Effects 
Bilateral fusiform gyri −.56* −.51* 
Bilateral occipital gyri −.63* −.54* 
Bilateral amygdala −.44 −.52* 

Experiment 1
Experiment 2
fROIs Showing Linear Effects 
Bilateral inferior frontal gyri −.61* −.69* 
Bilateral fusiform gyri −.61* −.58* 
Bilateral occipital gyri −.64* −.58* 
 
fROIs Showing Quadratic Effects 
Bilateral fusiform gyri −.56* −.51* 
Bilateral occipital gyri −.63* −.54* 
Bilateral amygdala −.44 −.52* 

Both brain responses and avoidance decisions were averaged across participants to obtain scores for each of the 16 faces from the 4 (Valence) × 4 (Power) matrix. The avoidance decisions were correlated with the brain responses in Experiment 1 and the brain responses in Experiment 2.

*p < .05.

GENERAL DISCUSSION

We measured the hemodynamic response to the social value of faces while participants performed two different tasks. We used faces that varied on two dimensions—valence and power—empirically derived from multiple social judgments of faces (Oosterhof & Todorov, 2008). In Experiment 1, participants made avoidance decisions that require explicit evaluation of the faces and are closely related to the dimensions of face evaluation. Not surprisingly, the probability of avoidance decisions increased with face negativity. This trend was augmented for high-power, dominant faces. In Experiment 2, participants made one-back recognition decisions that do not require explicit evaluation of faces.

Despite the large task differences in the two experiments, the pattern of brain responses was highly consistent. We showed that extensive areas of the brain are sensitive to face valence. The fusiform gyrus, lateral occipital lobe, lateral pFC, and portions of the parietal lobe all showed negative linear responses to face valence, with greater responses to low- than high-valence faces. We did not find any regions that showed the opposite relationship. The findings of stronger brain responses to negative than positive faces largely replicate prior studies (Todorov & Engell, 2008; Winston et al., 2002). These findings are also consistent with behavioral studies suggesting that negative face information is weighed more heavily in social decisions than positive information (Todorov, Loehr, & Oosterhof, 2010).

In addition to the linear responses, there were extensive quadratic responses to face valence in the occipital gyri, fusiform gyri, and dorsal amygdalae, with greater responses to the extremes of face valence than to faces in the middle of the continuum. Interestingly, even for the regions showing quadratic responses to valence, there was a positive correlation with the avoidance decisions (Figure 7D), as was the case for regions showing linear responses (Table 4). In all these regions, brain responses were stronger for faces that were likely to be avoided than for faces that were likely to be approached. The correlation analyses at the level of the face stimuli were highly consistent across both experiments (Table 5).

Functional Networks for Face Evaluation

Many of the areas we identified, such as the lateral frontal cortex, the superior parietal lobule, and the inferior parietal lobule, are thought to be integral nodes of the brain's attention network (Posner & Rothbart, 2007; Corbetta & Shulman, 2002; Kastner & Ungerleider, 2000; Duncan, 1998). Attention to faces is known to increase activation in the fusiform area (Wojciulik, Kanwisher, & Driver, 1998), and attention to visual stimulation, in general, is known to increase activation in occipital areas (Kastner & Ungerleider, 2000). Therefore, a simple explanation for many of our observed findings is that the extremes of valence, particularly the negative extreme, engage the attention network more than faces in the middle of the continuum. When confronted with extreme faces, attention source areas in the frontal and parietal lobes trigger top–down control of fusiform and occipital areas, enhancing neural responses in all involved regions. Similarly, there is evidence that emotional expressions elicit activation in frontal-parietal areas as well as in posterior areas such as the fusiform gyrus and occipital cortex (Vuilleumier & Pourtois, 2007; Ishai, Bikle, & Ungerleider, 2006; Ishai, Pessoa, Bikle, & Ungerleider, 2004; Pessoa, McKenna, Gutierrez, & Ungerleider, 2002; Vuilleumier, Armony, Driver, & Dolan, 2001). To the extent that the valence of neutral faces is, in part, driven by structural resemblance to emotional expressions (Oosterhof & Todorov, 2008, 2009; Said, Sebe, & Todorov, 2009; Zebrowitz, Fellous, Mignault, & Andreoletti, 2003), the results of the present study are consistent with past work.

In addition to the frontal-parietal attention network, it is likely that feedback connections from the amygdala are responsible for much of the fusiform activation. The fusiform area is known to respond to fearful faces (Surguladze et al., 2003; Vuilleumier et al., 2001; Morris et al., 1996), but this response is reduced or abolished in patients with amygdala damage relative to controls (Vuilleumier, Richardson, Armony, Driver, & Dolan, 2004). A direct test of this hypothesis would compare responses to face valence in the fusiform area of patients with amygdala damage to those of controls. In any case, we did not observe significant linear responses to face valence in the amygdala unlike in the fusiform gyri.

It should be noted that it is unlikely that the observed responses in the amygdala are face specific. Single-cell recording studies show that the amygdala responds to a variety of different stimuli (Gothard, Battaglia, Erickson, Spitler, & Amaral, 2007) in contrast to face selective regions in inferior temporal cortex that appear to show a high degree of category selectivity (Tsao, Freiwald, Tootell, & Livingstone, 2006). In terms of the temporal stages of face evaluation, it is plausible to assume that the faces are processed in the fusiform face-selective regions before processing in the amygdala (Said, Haxby, & Todorov, in press) and that, subsequently, the amygdala modulates activity in face-selective regions.

The Amygdala Response to Face Valence

There have been inconsistent results in previous studies investigating the amygdala response to face valence. Some studies have reported quadratic, nonmonotonic response functions to trustworthiness and attractiveness (Said, Baron, et al., 2009; Winston, O'Doherty, Kilner, Perrett, & Dolan, 2007), whereas other studies have shown mostly linear responses to trustworthiness (Engell et al., 2007). Still others have shown mixed results: Those that are mixed include a study showing a linear trend in the right amygdala and a quadratic trend in the left (Todorov, Baron, et al., 2008), a study showing a linear trend for implicit and explicit tasks, except for implicit tasks in the right amygdala (Winston et al., 2002), and another study showing that the shape of the response function during evaluative judgments of famous names depended systematically on task goal (Cunningham, Van Bavel, & Johnsen, 2008). The highly heterogeneous nature of these findings makes it difficult to identify a single factor that accounts for the discrepant results. However, there does not seem to be a task-dependent relationship because studies using explicit or implicit evaluative judgments have both yielded linear and quadratic results. In our study, we found quadratic responses in the amygdala regardless of whether participants were required to perform a one-back task or an approach–avoidance task requiring explicit evaluative judgments. These findings are consistent with proposals that stimulus facial features could automatically trigger the amygdala's response (Whalen et al., 1998, 2004).

One hypothesis that could account for linear and nonlinear responses in the amygdala is that the distribution of face stimuli is critical for the observed response. For example, if the face stimuli predominantly cover the negative part of the valence dimension, the observed response should be linear with stronger responses to more negative faces. If the face stimuli cover both the negative and positive parts of the valence dimension, as in the current study, the response should be quadratic. However, this has yet to be tested.

An alternative proposal that is not necessarily inconsistent with a stimulus distribution explanation is that the nature of the amygdala's response depends on the typicality of faces used in the study (Said et al., 2010). Specifically, Said and colleagues showed that face typicality—defined as the distance of the face from the average face—rather than face valence drives this response. They further showed that, when typicality is linearly correlated with valence, the amygdala's response is linear with stronger response to atypical faces (e.g., Engell et al., 2007). In contrast, when typicality and valence are nonlinearly related—with faces at the extremes perceived as less typical—the response is nonlinear with stronger response to atypical faces at both extremes. This proposal provides a more general explanation of previous findings related to face evaluation. The proposal is also broadly consistent with studies reporting stronger amygdala responses to both positive and negative stimuli than neutral stimuli (Paton, Belova, Morrison, & Salzman, 2006; Baxter & Murray, 2002; Hamann, Ely, Grafton, & Kilts, 1999) and studies reporting that the amygdala responds to the intensity rather than the valence of stimuli (Sabatinelli, Bradley, Fitzsimmons, & Lang, 2005; Anderson et al., 2003; Small et al., 2003).

An alternative explanation for the nonmonotonic functions observed not only in the amygdala but also in the fusiform and occipital cortices is that, under our stimulus sequence (and indeed, most random sequences), the change in valence between sequentially presented faces was, on average, larger for extreme than for moderate valence faces. Thus, faces at the extremes of the continuum evoked more release from fMRI adaptation (and, thus, greater responses) than faces in the middle of the continuum (Davidenko, Remus, Glover, & Grill-Spector, 2007). However, this explanation is tempered by the fact that the response to face power was much weaker and does not explain the linear component of the response function.

Another alternative interpretation for the quadratic responses is that these responses are driven by a task difficulty rather than by specific face properties, per se. Faces in the middle of the continuum were more difficult for both the approach–avoidance task and the one-back recognition task. However, this account would most likely predict a negative quadratic effect rather than the positive quadratic effect that we found, because greater attentional resources would need to be devoted to the faces in the middle of the continuum (e.g., Bokde et al., 2005; Druzgal & D'Esposito, 2001, 2003). This interpretation does not explain the linear component of the response function either.

Conclusions

In summary, across two tasks differing in their demands of evaluation, many regions tracked the valence of faces, showing stronger responses to negative than positive faces. In addition, a network of regions including the amygdala and inferior temporal cortex responded to face valence in a nonlinear fashion with stronger responses to faces at the extreme than at the middle of the continuum. Our findings show that both implicit and explicit face evaluations engage brain regions involved in attention, affect, and decision making.

Acknowledgments

This research was supported by National Science Foundation grant 0823749 and the Russell Sage Foundation.

Reprint requests should be sent to Alexander Todorov, Department of Psychology, Princeton University, Princeton, NJ 08540, or via e-mail: atodorov@princeton.edu.

REFERENCES

REFERENCES
Adolphs
,
R.
,
Tranel
,
D.
, &
Damasio
,
A. R.
(
1998
).
The human amygdala in social judgment.
Nature
,
393
,
470
474
.
Aguirre
,
G. K.
(
2007
).
Continuous carry-over designs for fMRI.
Neuroimage
,
35
,
1480
1494
.
Anderson
,
A. K.
,
Christoff
,
K.
,
Stappen
,
I.
,
Panitz
,
D.
,
Ghahremani
,
D. G.
,
Glover
,
G.
,
et al
(
2003
).
Dissociated neural representations of intensity and valence in human olfaction.
Nature Neuroscience
,
6
,
196
202
.
Ballew
,
C. C.
, &
Todorov
,
A.
(
2007
).
Predicting political elections from rapid and unreflective face judgments.
Proceedings of the National Academy of Sciences, U.S.A.
,
104
,
17948
17953
.
Bar
,
M.
,
Neta
,
M.
, &
Linz
,
H.
(
2006
).
Very first impressions.
Emotions
,
6
,
269
278
.
Baxter
,
M.
, &
Murray
,
E.
(
2002
).
The amygdala and reward.
Nature Reviews Neuroscience
,
3
,
563
573
.
Blair
,
I. V.
,
Judd
,
C. M.
, &
Chapleau
,
K. M.
(
2004
).
The influence of afrocentric facial features in criminal sentencing.
Psychological Science
,
15
,
674
679
.
Bokde
,
A. L. W.
,
Dong
,
W.
,
Born
,
C.
,
Leinsinger
,
G.
,
Meindl
,
T.
,
Teipel
,
S. J.
,
et al
(
2005
).
Task difficulty in a simultaneous face matching task modulates activity in face fusiform area.
Cognitive Brain Research
,
25
,
701
710
.
Corbetta
,
M.
, &
Shulman
,
G. L.
(
2002
).
Control of goal-directed and stimulus-driven attention in the brain.
Nature Reviews Neuroscience
,
3
,
201
215
.
Cox
,
R. W.
(
1996
).
AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages.
Computers in Biomedical Research
,
29
,
162
173
.
Cunningham
,
W. A.
,
Van Bavel
,
J. J.
, &
Johnsen
,
I. R.
(
2008
).
Affective flexibility: Evaluative processing goals shape amygdala activity.
Psychological Science
,
19
,
152
160
.
Davidenko
,
N.
,
Remus
,
D.
,
Glover
,
G.
, &
Grill-Spector
,
K.
(
2007
).
Sensitivity to image format and distinctiveness in face-selective cortex.
Paper presented at the Society for Neuroscience.
Druzgal
,
T. J.
, &
D'Esposito
,
M.
(
2001
).
Activity in fusiform face area modulated as a function of working memory load.
Cognitive Brain Research
,
10
,
355
364
.
Druzgal
,
T. J.
, &
D'Esposito
,
M.
(
2003
).
Dissecting contributions of prefrontal cortex and fusiform face area to face working memory.
Journal of Cognitive Neuroscience
,
15
,
771
784
.
Duncan
,
J.
(
1998
).
Converging levels of analysis in the cognitive neuroscience of visual attention.
Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences
,
353
,
1307
1317
.
Eberhardt
,
J. L.
,
Davies
,
P. G.
,
Purdie-Vaughns
,
V.
, &
Johnson
,
S. L.
(
2006
).
Looking deathworthy: Perceived stereotypicality of Black defendants predicts capital sentencing outcomes.
Psychological Science
,
17
,
383
388
.
Engell
,
A. D.
,
Haxby
,
J. V.
, &
Todorov
,
A.
(
2007
).
Implicit trustworthiness decisions: Automatic coding of face properties in the human amygdala.
Journal of Cognitive Neuroscience
,
19
,
1508
1519
.
Finney
,
D. J.
, &
Outhwaite
,
A. D.
(
1956
).
Serially balanced sequences in bioassay.
Proceedings of the Royal Society of London, Series B, Biological Sciences
,
145
,
493
507
.
Gothard
,
K. M.
,
Battaglia
,
F. P.
,
Erickson
,
C. A.
,
Spitler
,
K. M.
, &
Amaral
,
D. G.
(
2007
).
Neural responses to facial expression and face identity in the monkey amygdala.
Journal of Neurophysiology
,
97
,
1671
1683
.
Hamann
,
S.
,
Ely
,
T. D.
,
Grafton
,
S. T.
, &
Kilts
,
C. D.
(
1999
).
Amygdala activity related to enhanced memory for pleasant and aversive stimuli.
Nature Neuroscience
,
2
,
289
293
.
Ishai
,
A.
,
Bikle
,
P. C.
, &
Ungerleider
,
L. G.
(
2006
).
Temporal dynamics of face repetition suppression.
Brain Research Bulletin
,
70
,
289
295
.
Ishai
,
A.
,
Pessoa
,
L.
,
Bikle
,
P. C.
, &
Ungerleider
,
L. G.
(
2004
).
Repetition suppression of faces is modulated by emotion.
Proceedings of the National Academy of Sciences, U.S.A.
,
101
,
9827
9832
.
Kastner
,
S.
, &
Ungerleider
,
L. G.
(
2000
).
Mechanisms of visual attention in the human cortex.
Annual Review of Neuroscience
,
23
,
315
341
.
Kennedy
,
D. P.
,
Glascher
,
J.
,
Tyszka
,
J. M.
, &
Adolphs
,
R.
(
2009
).
Personal space regulation by the human amygdala.
Nature Neuroscience
,
12
,
1226
1227
.
Kwong
,
K. K.
,
Belliveau
,
J. W.
,
Chesler
,
D. A.
,
Goldberg
,
I. E.
,
Weisskoff
,
R. M.
,
Poncelet
,
B. P.
,
et al
(
1992
).
Dynamic magnetic resonance imaging of human brain activity during primary sensory stimulation.
Proceedings of the National Academy of Sciences, U.S.A.
,
89
,
5675
5679
.
Mazur
,
A.
,
Mazur
,
J.
, &
Keating
,
C.
(
1984
).
Military rank attainment of a West Point class: Effects of cadets' physical features.
American Journal of Sociology
,
90
,
125
150
.
Montepare
,
J. M.
, &
Zebrowitz
,
L. A.
(
1998
).
Person perception comes of age: The salience and significance of age in social judgments.
Advances in Experimental Social Psychology
,
30
,
93
163
.
Morris
,
J. S.
,
Frith
,
C. D.
,
Perrett
,
D. I.
,
Rowland
,
D.
,
Young
,
A. W.
,
Calder
,
A. J.
,
et al
(
1996
).
A differential neural response in the human amygdala to fearful and happy facial expressions.
Nature
,
383
,
812
815
.
Mueller
,
U.
, &
Mazur
,
A.
(
1996
).
Facial dominance of West Point cadets as a predictor of later military rank.
Social Forces
,
74
,
823
850
.
Ogawa
,
S.
,
Lee
,
T. M.
,
Nayak
,
A. S.
, &
Glynn
,
P.
(
1990
).
Oxygenation-sensitive contrast in magnetic resonance image of rodent brain at high magnetic fields.
Magnetic Resonance in Medicine
,
14
,
68
78
.
Olivola
,
C. Y.
, &
Todorov
,
A.
(
2010
).
Elected in 100 milliseconds: Appearance-based trait inferences and voting.
Journal of Nonverbal Behavior
,
34
,
83
110
.
Oosterhof
,
N. N.
, &
Todorov
,
A.
(
2008
).
The functional basis of face evaluation.
Proceedings of the National Academy of Sciences, U.S.A.
,
105
,
11087
11092
.
Oosterhof
,
N. N.
, &
Todorov
,
A.
(
2009
).
Shared perceptual basis of emotional expressions and trustworthiness impressions from faces.
Emotion
,
9
,
128
133
.
Paton
,
J. J.
,
Belova
,
M. A.
,
Morrison
,
S. E.
, &
Salzman
,
C. D.
(
2006
).
The primate amygdala represents the positive and negative value of visual stimuli during learning.
Nature
,
439
,
865
870
.
Pessoa
,
L.
,
McKenna
,
M.
,
Gutierrez
,
E.
, &
Ungerleider
,
L. G.
(
2002
).
Neural processing of emotional faces requires attention.
Proceedings of the National Academy of Sciences, U.S.A.
,
99
,
11458
11463
.
Posner
,
M. I.
, &
Rothbart
,
M. K.
(
2007
).
Research on attention networks as a model for the integration of psychological science.
Annual Review of Psychology
,
58
,
1
23
.
Sabatinelli
,
D.
,
Bradley
,
M. M.
,
Fitzsimmons
,
J. R.
, &
Lang
,
P. J.
(
2005
).
Parallel amygdala and inferotemporal activation reflect emotional intensity and fear relevance.
Neuroimage
,
24
,
1265
1270
.
Said
,
C. P.
,
Baron
,
S. G.
, &
Todorov
,
A.
(
2009
).
Nonlinear amygdala response to face trustworthiness: Contributions of high and low spatial frequency information.
Journal of Cognitive Neuroscience
,
21
,
519
528
.
Said
,
C. P.
,
Dotsch
,
R.
, &
Todorov
,
A.
(
2010
).
The amygdala and FFA track both social and nonsocial face dimensions.
Neuropsychologia
,
48
,
3596
3605
.
Said
,
C. P.
,
Haxby
,
J. V.
, &
Todorov
,
A.
(
in press
).
Brain systems for the assessment of the affective value of faces.
Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences
.
Said
,
C. P.
,
Sebe
,
N.
, &
Todorov
,
A.
(
2009
).
Structural resemblance to emotional expressions predicts evaluation of emotionally neutral faces.
Emotion
,
9
,
260
264
.
Small
,
D. M.
,
Gregory
,
M. D.
,
Mak
,
Y. E.
,
Gitelman
,
D.
,
Mesulam
,
M. M.
, &
Parrish
,
T.
(
2003
).
Dissociation of neural representation of intensity and affective valuation in human gestation.
Neuron
,
39
,
701
711
.
Surguladze
,
S. A.
,
Brammer
,
M. J.
,
Young
,
A. W.
,
Andrew
,
C.
,
Travis
,
M. J.
,
Williams
,
S. C.
,
et al
(
2003
).
A preferential increase in the extrastriate response to signals of danger.
Neuroimage
,
19
,
1317
1328
.
Todorov
,
A.
(
2008
).
Evaluating faces on trustworthiness: An extension of systems for recognition of emotions signaling approach/avoidance behaviors.
In A. Kingstone & M. Miller (Eds.), The Year in Cognitive Neuroscience 2008,
Annals of the New York Academy of Sciences
,
1124
,
208
224
.
Todorov
,
A.
,
Baron
,
S. G.
, &
Oosterhof
,
N. N.
(
2008
).
Evaluating face trustworthiness: A model based approach.
Social Cognitive and Affective Neuroscience
,
3
,
119
127
.
Todorov
,
A.
, &
Engell
,
A.
(
2008
).
The role of the amygdala in implicit evaluation of emotionally neutral faces.
Social, Cognitive, and Affective Neuroscience
,
3
,
303
312
.
Todorov
,
A.
,
Loehr
,
V.
, &
Oosterhof
,
N. N.
(
2010
).
The obligatory nature of holistic processing of faces in social judgments.
Perception
,
39
,
514
532
.
Todorov
,
A.
,
Mandisodza
,
A. N.
,
Goren
,
A.
, &
Hall
,
C. C.
(
2005
).
Inferences of competence from faces predict election outcomes.
Science
,
308
,
1623
1626
.
Todorov
,
A.
,
Pakrashi
,
M.
, &
Oosterhof
,
N. N.
(
2009
).
Evaluating faces on trustworthiness after minimal time exposure.
Social Cognition
,
27
,
813
833
.
Todorov
,
A.
,
Said
,
C. P.
,
Engell
,
A. D.
, &
Oosterhof
,
N. N.
(
2008
).
Understanding evaluation of faces on social dimensions.
Trends in Cognitive Sciences
,
12
,
455
460
.
Todorov
,
A.
,
Said
,
C. P.
, &
Verosky
,
S. C.
(
2011
).
Personality impressions from facial appearance.
In A. Calder, J. V. Haxby, M. Johnson, & G. Rhodes (Eds.),
Handbook of face perception
(pp.
631
652
).
Oxford University Press
.
Tsao
,
D. Y.
,
Freiwald
,
W. A.
,
Tootell
,
R. B. H.
, &
Livingstone
,
M. S.
(
2006
).
A cortical region consisting entirely of face-selective cells.
Science
,
311
,
670
674
.
Vuilleumier
,
P.
(
2005
).
How brains beware: Neural mechanisms of emotional attention.
Trends in Cognitive Sciences
,
9
,
585
594
.
Vuilleumier
,
P.
,
Armony
,
J. L.
,
Driver
,
J.
, &
Dolan
,
R. J.
(
2001
).
Effects of attention and emotion on face processing in the human brain: An event-related fMRI study.
Neuron
,
30
,
829
841
.
Vuilleumier
,
P.
, &
Pourtois
,
G.
(
2007
).
Distributed and interactive brain mechanisms during emotion face perception: Evidence from functional neuroimaging.
Neuropsychologia
,
45
,
174
194
.
Vuilleumier
,
P.
,
Richardson
,
M. P.
,
Armony
,
J. L.
,
Driver
,
J.
, &
Dolan
,
R. J.
(
2004
).
Distant influences of amygdala lesion on visual cortical activation during emotional face processing.
Nature Neuroscience
,
7
,
1271
1278
.
Whalen
,
P. J.
,
Kagan
,
J.
,
Cook
,
R. G.
,
Davis
,
F. C.
,
Kim
,
H.
,
Polis
,
S.
,
et al
(
2004
).
Human amygdala responsivity to masked fearful eye whites.
Science
,
306
,
2061
.
Whalen
,
P. J.
,
Rauch
,
S. L.
,
Etcoff
,
N. L.
,
McInerney
,
S. C.
,
Lee
,
M. B.
, &
Jenike
,
M. A.
(
1998
).
Masked presentations of emotional facial expressions modulate amygdala activity without explicit knowledge.
Journal of Neuroscience
,
18
,
411
418
.
Willis
,
J.
, &
Todorov
,
A.
(
2006
).
First impressions: Making up your mind after 100 ms exposure to a face.
Psychological Science
,
17
,
592
598
.
Winston
,
J. S.
,
O'Doherty
,
J.
,
Kilner
,
J. M.
,
Perrett
,
D. I.
, &
Dolan
,
R. J.
(
2007
).
Brain systems for assessing facial attractiveness.
Neuropsychologia
,
45
,
195
206
.
Winston
,
J. S.
,
Strange
,
B. A.
,
O'Doherty
,
J.
, &
Dolan
,
R. J.
(
2002
).
Automatic and intentional brain responses during evaluation of trustworthiness of faces.
Nature Neuroscience
,
5
,
277
283
.
Wojciulik
,
E.
,
Kanwisher
,
N.
, &
Driver
,
J.
(
1998
).
Covert visual attention modulates face-specific activity in the human fusiform gyrus: fMRI study.
Journal of Neurophysiology
,
79
,
1574
1578
.
Zebrowitz
,
L. A.
,
Fellous
,
J. M.
,
Mignault
,
A.
, &
Andreoletti
,
C.
(
2003
).
Trait impressions as overgeneralized responses to adaptively significant facial qualities: Evidence from connectionist modeling.
Personality and Social Psychology Review
,
7
,
194
215
.
Zebrowitz
,
L. A.
, &
McDonald
,
S.
(
1991
).
The impact of litigants' babyfacedness and attractiveness on adjudications in small claims courts.
Law and Human Behavior
,
15
,
603
623
.