Faces expressing fear may attract attention in an automatic bottom–up fashion. Here we address this issue with magneto-encephalographic (MEG) recordings in subjects performing a demanding visual search combined with the presentation of irrelevant neutral or fearful faces. The impact of the faces' emotional expression on attentional selection was assessed by analyzing the N2pc component—a modulation of the event-related magnetic field response known to reflect attentional focusing in visual search. We observed that lateralized fearful faces elicited an N2pc approximately between 240 and 400 msec in ventral extrastriate cortex that was independent of the N2pc reflecting target selection in visual search. Despite their clear influence on neural processing, fearful faces did not significantly influence behavioral performance. To clarify this discrepancy, we further performed an MEG experiment in which the demands of the search task were reduced. Under those conditions, lateralized fearful faces elicited an N2pc response that was again independent of the N2pc response to the search target. Behavioral performance was, however, influenced in a significant manner, suggesting that for behavioral effects to appear, sufficient attentional resources need to be left unoccupied by the search task—a notion put forward by the perceptual load theory. Our observations are taken to indicate that irrelevant fearful faces influence attentional processing in extrastriate visual cortex in an automatic fashion and independent of other task-relevant attentional operations. However, this may not necessarily be echoed at the behavioral level as long as task-relevant selection operations exhaust attentional resources.
Given the general ecological importance of facial gestures for human social interactions, it is not surprising that faces expressing emotional states are suspected to capture attention in an automatic way (Dolan & Vuilleumier, 2003; Ohman, Lundqvist, & Esteves, 2001; Vuilleumier & Schwartz, 2001; White, 1995). Although face features per se may not necessarily give rise to attentional capture (Nothdurft, 1993), the emotional valence of faces is suggested to be mandatorily registered outside the focus of attention (Lavie, Ro, & Russell, 2003; Vuilleumier, 2002; Morris, Ohman, & Dolan, 1999). In particular, negative emotional valence was observed to speed face identification in visual search significantly (Eastwood, Smilek, & Merikle, 2001, 2003; Fox et al., 2000; Hansen & Hansen, 1988), which was taken to indicate that negative valence may act like a pop-out feature in guiding attentional focusing automatically (e.g., anger superiority effect). However, the results of Hansen and Hansen (1988) obtained with natural face images (Ekman & Friesen, 1978) were challenged by attesting low-level stimulus confounds (Purcell, Stewart, & Skov, 1996) or configurational shortcomings (Cave & Batty, 2006; Horstmann & Bauland, 2006). Purcell et al. (1996), for example, demonstrated that conspicuous black spots were more frequent in threatening than in happy faces, which presumably gave rise to the capture effect. Horstmann and Bauland (2006) suggested that the “anger superiority effect” demonstrated with angry faces embedded in friendly face distractors may rather reflect the fact that observers were more efficient in rejecting happy distractors than in detecting angry target faces.
To avoid low-level stimulus confounds, a number of studies used schematic face drawings (Eastwood et al., 2001; Ohman et al., 2001; Fox et al., 2000; White, 1995) to show that negative emotional valence undergoes automatic priority decoding in visual search. These studies, however, provided rather heterogeneous and partially conflicting results and were overall not entirely conclusive. Many of the observations may simply be attributable to lower level configurational confounds and not to the emotional valence of the face drawings (Cave & Batty, 2006; Horstmann & Bauland, 2006).
Regarding the automatic capture hypothesis, there are further issues to be considered. In simple search tasks, attentional capture by salient singletons has been shown to be contingent on top–down expectations about the target (Yantis & Egeth, 1999; Folk, Remington, & Wright, 1994; Yantis, 1993; Folk, Remington, & Johnston, 1992). In other words, attentional capture has been observed when the capturing distractor's feature was at least partially task relevant, but not when the searched-for feature set was completely nonoverlapping with the distractor. Hence, to verify true automatic capture, it is mandatory to demonstrate that capture appears for completely task-irrelevant features (Yantis, 1996). As face information has typically been task relevant in most of the abovementioned search studies, it remains unclear whether true automatic capture was involved.
Furthermore, for clarifying this issue, it is important to define the notion of automaticity thoroughly. Automaticity turns out to be a broad concept that has been defined from many different perspectives. A recent comprehensive analysis (Moors & de Houwer, 2006) reviewed definitions on the basis of notions as diverse as controllability, intentionality, goal dependence, consciousness, autonomy, efficiency, speed of processing, and more. Hence, it is critical to clarify what particular notion of automaticity we follow here. In terms of the classification of Moors and de Houwer (2006), the present study refers to automaticity in the sense of demonstrating the presence of neural signatures of processing face expression despite the fact that there is neither the goal nor the intention to do so. Automaticity in the present study does not refer to whether the processing of fear features is efficient or fast or becomes conscious in any form.
Recently, a few ERP studies used task-irrelevant face stimuli to address the automaticity issue in the sense just sketched (Eimer & Kiss, 2007; Pourtois, Grandjean, Sander, & Vuilleumier, 2004). In these studies, evidence could be obtained compatible with the possibility that negative emotional valence, indeed, captures attention automatically. Finally, recent evidence from fMRI and ERP indicates that emotional face recognition competes with other attentional operations for processing resources (Silvert et al., 2007; Eimer, Holmes, & McGlone, 2003; Holmes, Vuilleumier, & Eimer, 2003; Pessoa, McKenna, Gutierrez, & Ungerleider, 2002). According to these observations, emotional face expression may not capture attention automatically (Anderson, Christoff, Panitz, De Rosa, & Gabrieli, 2003) when resources are exhausted by some other demanding attentional operation (Lavie, 2001; Lavie & Tsal, 1994).
Thus, the issue whether the emotional valence of faces grants attentional priority in a purely automatic fashion (Yantis & Johnson, 1990) is far from being settled. Here we address this issue with high-temporal resolution MEG recordings in human observers. In two experiments, we combined conventional search tasks requiring subjects to find and discriminate a colored oriented bar among distractor bars with a simultaneous presentation of irrelevant face photographs expressing neutral or negative emotional valence. Each search frame contained two photographs of the same person, one presented in the left and one in the right visual field (VF), with the emotional expression systematically varied between VFs. That is, a face with a negative expression could appear in the left VF whereas a neutral expression could appear in right VF, and vice versa. In addition, faces in both VFs could display a negative or a neutral expression. To avoid low-level stimulus confounds (Hansen & Hansen, 1988), we matched the face photographs in luminance and contrast and band-pass filtered it with a spatial Fourier filter (see Methods), which effectively eliminated conspicuous black “mouth spots” that may potentially capture attention independent of emotional valence (Purcell et al., 1996). As a critical experimental manipulation, the location of negative valence was varied independently of the location of the search target, which permitted to analyze the time course of activity modulations because of the lateralized valence with respect to the time course of activity reflecting attentional focusing onto the search target.
Neural activity reflecting attentional focusing was assessed by analyzing the magnetic analog of the so-called N2pc component of the ERP response. The N2pc—a relative negativity over the occipital scalp contralateral to the target item between 180 and 350 msec after search frame onset—has been first described in the ERP response by Luck and Hillyard (1994a, 1994b) and was subsequently characterized in the neuromagnetic response (Hopf, Boelmans, Schoenfeld, Luck, & Heinze, 2004; Hopf, Boelmans, Schoenfeld, Heinze, & Luck, 2002; Hopf et al., 2000). The N2pc reflects the operation of focusing attention onto the target in visual search and was particularly shown to permit tracking of multiple and rapid shifts of attention in real time (Woodman & Luck, 1999, 2003b).
Twelve healthy students (9 women, age = 18–27 years) of the Otto-von-Guericke University, Magdeburg, took part in Experiment 1. Eight of the students participating in Experiment 1 and four additional students took part in Experiment 2. All subjects were right-handed, had normal color vision, and had normal or corrected-to-normal visual acuity. Subjects gave written informed consent and were paid for participation. The study was approved by the Ethics Committee of the Otto-von-Guericke University, Magdeburg.
The stimulus setup is illustrated in Figure 1A–D. Each search frame contained two orthogonally oriented double-colored bars in each VF superimposed onto two face photographs of the same person. Color was randomly assigned to the VFs, with the bars in one VF both being either red-green or blue-yellow. The same color combination never appeared in both VF. The bars in each VF were presented within a 1° × 1.5° (visual angle) region 8.5° away from the vertical and 3.5° below the horizontal meridian. At the start of each trial block, one color combination was designated as being the target (e.g., red/green), and subjects searched for the vertical bar drawn in that combination. The adjacent horizontal bar was irrelevant and served as a search distractor. The relative distribution of color within the bars was random, and subjects had to indicate with an alternative button press whether one color of the target color combination (e.g., red) appeared in the upper or lower half of the vertical bar. Target color was fixed within but varied across different trial blocks, resulting in two possible target definitions (vertical/green-red and vertical/blue-yellow). The presentation of a target color combination in one VF was always accompanied by a different color combination in the opposite VF. This permitted to direct the focus of attention to the left or right VF by presenting identical search frames but simply instructing subjects to attend alternative color combinations. The important advantage of this approach is that low-level stimulus confounds due to color lateralization are eliminated when comparing the event-related magnetic field (ERMF) response elicited by left and right VF targets, thereby leaving the purely attention-related response.
In addition to the search items, two face photographs were presented, one in the left and one in the right VF. Face photographs were taken from a subset of a hundred fearful and neutral faces of the Ekman and Friesen (1978) and Gur et al. (2002) collection. They subtended 8.6°/10.9° (width/height) of visual angle and were presented in a region centered 7.6° away from the vertical and 3.5° below the horizontal meridian. Although the photographs always showed the same person's face in both VFs, the emotional expression was varied systematically between VFs relative to the VF where the target appeared. That is, in 25% of the trials, the face would have a negative expression in the target VF (right VF in Figure 1B) but a neutral one in the opposite nontarget VF. In another 25% of the trials, a neutral expression in the target VF would be combined with a negative expression in opposite VF (Figure 1C). In yet another 25%, a negative emotional expression would appear in both the target and the nontarget VF (Figure 1D), and in the remaining 25% of the trials, a neutral expression would appear bilaterally (Figure 1A). This permitted to assess the effect of a lateralized emotional face expression onto the ERMF response relative to the ERMF response reflecting attentional focusing onto the search target (target-related N2pc). Although the bar stimuli were superimposed onto the task-irrelevant faces, they always appeared in the buccal region without covering critical features of the faces. To eliminate low-level stimulus confounds, we converted all face photographs to grayscale, matched them in size, luminance, and contrast and low-pass filtered them with a two-dimensional fast Fourier filter (second-order Butterworth window, six cycles per image). This transformation particularly eliminates the conspicuousness of mouth spots in fearful faces, which was a critical confound in other studies (as outlined by Purcell et al., 1996). Note that because the fearful expression of faces is mainly contained in the low spatial frequency content of face photographs (Vuilleumier, Armony, Driver, & Dolan, 2003), low-pass filtering of faces preserved this information. All stimuli were presented on a gray background.
The general stimulus setup is illustrated in Figure 1E. The setup was identical to Experiment 1, except for changing the coloration of the search items. Search frames now contained only uniformly colored bars, with one red and one green bar presented in opposite VFs together with two orthogonally oriented blue distractor bars (one in each VF). The number and the size of the colored bars were identical to Experiment 1. The orientation of the bars changed randomly from trial to trial. The subjects' task was to discriminate the orientation of the bar drawn in the target color, which was designated at the start of each trial block. The spatial distribution of fearful and neutral faces was identical to Experiment 1.
At the beginning of each trial block, an instruction frame informed participants about the target color. Each trial block required subjects to fixate the center of the screen (white fixation dot) and to perform a sequence of 64 search frames, with each frame presented for 400 msec in a sequence with randomized ISIs between 1300 and 1500 msec. Participants indicated via button press with the right hand whether the target color appeared in the upper (index finger) or lower part (middle finger) of the vertical bars. Subjects performed on eight trial blocks, such that two possible combinations of the target color (red and blue), were searched for four times. This resulted in a total number of 128 trials per condition (2 VFs × 4 distributions of emotional valence).
The experimental procedure was identical to Experiment 1, except the discrimination task. Subjects were required to report the orientation (horizontal/vertical) of the bar drawn in the target color again with a two-alternative button press of the right index/middle finger. Experiment 1 required focusing onto subparts of the target bar and determining the color distribution, the experimental task now required a simple orientation discrimination of the whole target bar that made the discrimination task perceptually easier. In terms of perceptual load theory (PLT; Lavie, Hirst, de Fockert, & Viding, 2004), perceptual load was reduced by making the perceptual task less demanding but without changing the number of distractors. Importantly, this change of the discrimination task did not change top–down executive demands. Target color was randomly assigned from trial to trial. Trial structure was identical to Experiment 1. As in Experiment 1, subjects performed eight trial blocks amounting to a total of 128 trials per condition.
Data Recording and Analysis
The MEG and the EOG were registered simultaneously using a 148-channel Bti Magnes 2500 whole-head magnetometer (Biomagnetic Technologies Inc., San Diego, CA) and a Synamps amplifier (NeuroScan Inc., Herndon, VA). The signals were digitized at a rate of 254 Hz and band-pass filtered from 0.1 to 50 Hz. The horizontal and the vertical EOG were recorded bipolarly, using two electrodes behind the lateral orbital angles for the horizontal EOG whereas the vertical EOG was recorded from an electrode below the right orbital ridge and one above the right eye. Impedances were kept less than 5 kΩ, and an electrode placed at FPZ served as ground. MEG signals were submitted to on-line and off-line noise reduction (Robinson, 1989). Epochs containing artifacts were rejected when exceeding a peak-to-peak threshold of 3 pT for the MEG and 100 μV for the EOG signal.
For further data analysis, ERMF waveforms were averaged for each experimental condition and target VF, were time locked to search frame onset, and averaged relative to an immediately preceding baseline interval of 100 msec. For statistical validation, we quantified data as the mean magnetic response in selected time windows and then subjected to repeated measures ANOVAs; nonsphericity was corrected with the Greenhouse–Geisser epsilon when necessary. To determine the onset latency of ERMF waveform differences, we computed t tests on the basis of mean amplitude measures in a short time window (30 msec) that was moved sample by sample between 150 and 450 msec after stimulus onset. The first significant t test (p < .05) in a sequence of three or more significant consecutive t tests was taken to mark the onset of the waveform difference (Guthrie & Buchwald, 1991).
Current Source Localization
Current source localization was constrained by realistic anatomical models of volume conductor and source compartment. To this end, anatomical and functional data had to be in precise register, which was achieved in the following way: Anatomical landmarks (left and right preauricular points, nasion) were digitized using a Polhemus 3Space Fastrak system (Polhemus Inc., Colchester, VT) and then referenced to the sensor coordinate system. In addition, five spatially distributed coils attached to the head were digitized, which generated magnetic field markers permitting to coregister the functional data. Current source localization was based on minimum norm least squares (MNLS) estimates as implemented in the multimodal neuroimaging software Curry 5.08 (Neuroscan Inc.). MNLS represents a distributed source model (Fuchs, Wagner, Kohler, & Wischmann, 1999; Hamalainen & Ilmoniemi, 1994) that yields the distribution of currents over the cortical surface. Realistic anatomical models of the source compartment and the volume conductor were obtained by three-dimensional segmentations of the head, the cerebrospinal fluid space, and the cortical surface of the brain from the Montreal Neurological Institute (average of 152 T1-weighted volumes from the ICBM project, see www.bic.mni.mcgill.ca/cgi/icbm_view/) using the boundary element method (Hämäläinen & Sarvas, 1989).
The effect of the spatial distribution of fearful face expression on response time and response accuracy was analyzed with two-way repeated measures ANOVAs with the factors Condition (bilateral neutral, fearful face at target side, fearful face at nontarget side, and bilateral fearful) and Target VF (left and right). This analysis yielded no significant main effect of condition, neither regarding the observers' average response time, F(3, 33) = 1.39, p > .25, nor regarding their response accuracy, F(3, 33) = 1.08, p > .35. Subjects showed almost identical response times and accuracy measures in the left VF (RT[msec]/correct[%]: bilateral neutral = 647 msec/93%; bilateral fearful = 643 msec/96%; fearful face at target side = 645 msec/94%; fearful face at nontarget side = 646 msec/95%). Subjects were somewhat faster in the right VF when fearful faces were absent (697 msec) or in the nontarget VF (698 msec) as compared with when fearful faces were presented at the target side (707 msec) or bilaterally (708 msec). However, the Condition × Target VF interaction did not reach significance, F(3, 33) = 2.47, p = .10. Accuracy was almost identical in the right VF (bilateral neutral = 88%, bilateral fearful = 87%, fearful face at target side = 87%, and fearful face at nontarget side = 88%), and the Condition × Target VF interaction was not significant, F(3, 33) = 2.47, p = .10. There was, however, a significant main effect of target VF on response time, F(1, 11) = 16.79, p < .01, and accuracy, F(1, 11) = 19.65, p < .01, reflecting the fact that responses were generally faster and more accurate for targets presented in the left VF.
Effect of target lateralization
To assess neural activity reflecting attentional focusing onto the target, we analyzed the N2pc—a component that is derived as the difference between ERMF responses elicited by left and right VF targets (target-related N2pc). We first focused on search frames with the same emotional face expression in both VFs (bilateral neutral and bilateral fearful) because this provides a clean measure of the target-related N2pc, which is unbiased by potential-lateralized responses because of presentation of a fearful face in only one VF. Figure 2A shows waveforms elicited by search targets in the left (solid lines) and right (dashed lines) VFs together with a field distribution map of the left-minus-right ERMF difference (at 270 msec) of the bilateral neutral condition. Waveforms are shown from selected sensors over the left and right occipito-temporal cortex (circles), which reveal a clear N2pc effect. The N2pc is visible as the waveform difference between traces (gray area) approximately from 220 to 380 msec after search frame onset. It should be noted that in contrast to the N2pc of the ERP, the magnetic analog of the N2pc does not appear as activity modulation contralateral to the target item. Instead, it appears as similar polarity modulation at occipito-temporal sensors in the left and right hemispheres when responses elicited by left and right VF targets are compared. Specifically, a left VF target elicits a negativity (downward deflection in Figure 2A) at both left and right occipito-temporal sensors relative to a right VF target. For simplicity, we will keep the term N2pc to refer to the magnetic analog of this component. In the field distribution map, the N2pc appears as efflux–influx configuration over left and right inferior occipito-temporal regions (encircled by ellipses), with red lines reflecting the efflux part and blue lines the influx part. As detailed previously (Hopf, Vogel, Woodman, Heinze, & Luck, 2002; Hopf et al., 2000), the current origin underlying such field configuration is predicted to arise from underneath the transition zone between the efflux and the influx components (asterisks). Current source density estimates of the left-minus-right ERMF difference confirm this prediction in showing current maxima in the left and right inferior occipito-temporal cortex (upper left insert). Figure 2B illustrates the results for the bilateral fearful condition. As in the bilateral neutral condition, a clear N2pc effect is visible. Importantly, the waveforms as well as the field distribution of the left-minus-right ERMF difference show an N2pc effect that is very similar to the neutral condition. Source density estimates of the left-minus-right ERMF difference reveal current source maxima in left and right ventral occipito-temporal cortex.
For statistical validation, we computed a repeated measures ANOVA with the factors Target VF (left/right), Sensor Hemisphere (left/right), and Condition (bilateral neutral/bilateral fearful) on the basis of mean amplitudes between 250 and 350 msec. This revealed a significant effect of Target VF, F(1, 11) = 21.7, p < .001, which validates the presence of the N2pc. There was no main effect of Condition, F(1, 11) = 0.18, p > .65, and no Target VF × Condition interaction, F(1, 11) = 0.16, p > .7, indicating that the N2pc amplitude did not significantly differ between conditions. Waveforms generally differed between left and right hemisphere sensors, which were reflected by a significant effect of Sensor Hemisphere, F(1, 11) = 10.7, p < .01.
In sum, the field distribution and the source localization results indicate that visual search in the control and the bilateral fearful condition produced a typical N2pc effect approximately between 220 and 380 msec that is perfectly in line with our previous characterization of the N2pc in the magnetic response (Hopf, Vogel, et al., 2002; Hopf et al., 2000). Notably, there was no difference when comparing the N2pc responses elicited by the control and the bilateral fearful condition.
The effect of lateralized fearful faces
To analyze the effect of irrelevant fearful faces on attentional focusing, we considered search frames with a fearful face expression only in the left or right VF. Specifically, the magnetic response elicited by a fearful face in the left VF (Figure 1C) was compared with the response elicited by a fearful face in the right VF (Figure 1B), when both times the target was presented in the right VF. Analogously, the response to a fearful face in the left and right VFs was compared when the search target appeared in the left VF. Ideally, those comparisons equal attention effects related to focusing onto the search target but leave effects related to the lateralized fearful faces. Figure 3 summarizes the results separately for targets in the left (Figure 3A) and right (Figure 3B) VFs. For both target VFs, we obtained a clear modulation of the ERMF response that starts around 240 msec. Although smaller in amplitude, this modulation is similar in time course, polarity, and field distribution (left-minus-right difference) to the N2pc seen when comparing left versus right VF targets (cf. Figure 2). Furthermore, source density estimates of the left-minus-right ERMF difference reveal current maxima in ventral occipito-temporal cortex regions typical for the N2pc. Taken together, those results indicate that fearful faces, in fact, elicit an N2pc effect on their own, which is independent of the neural processes mediating the focusing of attention onto the search target.
To validate the N2pc to the lateralized fearful faces statistically, we computed an overall repeated measures ANOVA with the factors Face Lateralization (fearful face left and fearful face right), Sensor Hemisphere (left and right), and Target VF (left and right) on the mean ERMF response between 250 and 350 msec. This analysis revealed a significant main effect of Face Lateralization, F(1, 11) = 16.9, p < .005, indicating the presence of an N2pc due to the fearful faces. There was also a significant effect of Target VF, F(1, 11) = 6.2, p < .05, reflecting the presence of an N2pc to the lateralized targets. There was, however, no interaction between Face Lateralization × Target VF, F(1, 11) = 1.9, p > .2, which indicates that the size of the N2pc effect to fearful faces was independent of the localization of the search target.
Given that irrelevant fearful faces elicited an N2pc response that is independent of the N2pc response to the search targets, it would be informative to determine the relative time course of both N2pc responses. To this end, we computed sample-wise sliding t tests (see Methods) comparing ERMF waveforms elicited by left versus right VF targets and by left versus right VF fearful faces between 150 and 450 msec. Figure 3C summarizes respective results. Time ranges of significant waveform differences (p < .05) reflecting the N2pc are highlighted by horizontal bars. The analysis was based on waveforms collapsed across left and right hemisphere sensors. Apparently, the N2pc to fearful faces appears in the same time range as the target-related N2pc. Importantly, it does not arise before the target-related N2pc. In fact, it onsets with a slight delay of approximately 20 msec (see arrows in Figure 3C, which mark the average onset of the target- and fearful face-related N2pc).
The distribution of fearful faces had no significant influence on response time and accuracy in Experiment 1. This is unexpected given that lateralized fearful faces elicited a significant N2pc component in the neuromagnetic response. To evaluate this dissociation, it may be helpful considering notions put forward by the PLT of visual attention (Lavie et al., 2004; Lavie & de Fockert, 2003; Lavie & Tsal, 1994). PLT predicts that irrelevant distractors influence performance only under conditions of sufficiently low perceptual load imposed by the primary search task, that is, when attentional resources are spare and assignable to the processing of distractor information (face valence). In other words, if the search task in Experiment 1 represented a high load condition in terms of PLT, significant effects on performance may not have appeared because not enough resources were left to “leak” to the processing of face valence.
It should be noted that Lavie et al. (2003) have provided behavioral evidence suggesting that the processing of faces per se may not be susceptible to manipulations of perceptual load; however, it is possible that the processing of the emotional expression of faces depends on load. Experiment 2 was performed to address the effect of reducing perceptual load (see Methods) on the processing of irrelevant fearful faces.
A comparison of response time and accuracy data between Experiments 1 and 2 reveals that subjects were generally faster (587 vs. 673 msec in Experiment 1) and more accurate in Experiment 2 (97% vs. 91% correct in Experiment 1). This could be confirmed by independent samples t tests, which yielded significant differences of RT, t(22) = 3.32, p < .005, as well as of accuracy, t(22) = 3.96, p < .005. Hence, the performance differences between experiments validate our experimental manipulation in aiming at lowering the perceptual demands of the discrimination task. Importantly, a repeated measures ANOVA with the factors Condition (bilateral neutral, fearful face at target side, fearful face at nontarget side, and bilateral fearful) and Target VF (left and right) revealed that the distribution of fearful faces significantly influenced response time, F(3, 33) = 5.62, p < .05, in Experiment 2. The Condition × Target VF interaction was not significant, F(3, 33) = 0.84, p > .35. A subsequent post hoc analysis revealed that the effect on response time was mainly due to the fact that subjects were significantly slower when fearful faces appeared in the target VF (594 msec) as opposed to the nontarget VF (580 msec) (t = 3.23, p < .01). Response times for the bilateral fearful and bilateral neutral conditions were both 587 msec. Response time was generally faster in the right VF than in the left VF, which is reflected by a significant main effect of Target VF, F(1, 11) = 9.14, p < .01. In contrast to response time, accuracy was completely uninfluenced by the distribution of fearful faces (bilateral neutral = 97.0%, bilateral fearful = 97.2%, fearful face target side = 97.3%, fearful face nontarget side = 96.9%). Neither the factor Condition, F(3, 9) = 0.37, p > .65, nor the Condition × Target VF interaction, F(3, 33) = 2.43, p > .15, was significant. There was also no main effect of VF, F(1, 11) = 0.08, p > .75. What is notable is that although subjects were generally faster in responding to left VF targets in Experiment 1, they were now somewhat faster in the right VF. This difference is unexpected but may reflect qualitative differences in perceptual task. Experiment 2 required a simple orientation discrimination, whereas Experiment 1 required to discriminate the relative position of color within a bar. The latter task is equivalent to discriminating spatial phase, and spatial phase discrimination has been shown to be better in the left than in the right VF (Fiorentini & Berardi, 1984). It is possible that the change of VF asymmetries between experiments relates to this difference of perceptual tasks.
Taken together, Experiment 2 revealed that when the perceptual demands of target discrimination are reduced in comparison to Experiment 1, significant performance effects on response time appear. This is consistent with the possibility that the task requirements of Experiment 2 leave more attentional resources unoccupied that could spill to the processing of irrelevant fearful faces.
Figure 4 summarizes the MEG results. Figure 4A shows the target-related N2pc effect, that is, waveforms elicited by targets in the left and right VFs of the bilateral neutral (left) and bilateral fearful (right) conditions. The N2pc effect is visible as waveform difference (gray area between traces) approximately between 240 and 370 msec. A comparison with corresponding waveforms of Experiment 1 (Figure 2A and B) reveals that the effects are rather similar both in terms of the size of the N2pc as well as its time course. Moreover, current source density estimates of the N2pc revealed similar current maxima in the left and right inferior occipito-temporal cortex. As in Experiment 1, statistical validation of the N2pc effect was performed on the basis of mean amplitude measures between 250 and 350 msec. A repeated measures ANOVA with the factors Target VF (left/right), Sensor Hemisphere (left/right), and Condition (bilateral neutral/bilateral fearful) revealed a significant effect of Target VF, F(1, 11) = 11.2, p < .01, which validates the presence of the N2pc. There was no main effect of Condition, F(1, 11) = 0.21, p > .65, and no Target VF × Condition interaction, F(1, 11) = 1.77, p > .20, indicating that the N2pc amplitude did not significantly differ between conditions. Waveforms generally differed between left and right hemisphere sensors, which were reflected by a significant effect of Sensor Hemisphere, F(1, 11) = 5.75, p < .05.
The response to lateralized fearful faces is shown in Figure 4B. As in Experiment 1, the lateralized presentation of fearful faces produces an N2pc response that is independent of the location of the search target. Again, the effect is generally smaller than the N2pc to the search targets, but it appears consistently and in roughly the same time range. Current source density estimates reveal current maxima in ventral occipito-temporal cortex, typical for the N2pc and corresponding with the maxima observed in Experiment 1. For statistical validation, we computed an overall repeated measures ANOVA with the factors Face Lateralization (fearful face left and fearful face right), Sensor Hemisphere (left and right), and Target VF (left and right) on the mean ERMF response between 250 and 350 msec. This analysis revealed a significant main effect of Face Lateralization, F(1, 11) = 7.86, p < .02, confirming the presence of an N2pc due to the fearful faces. The factor Target VF was also significant, F(1, 11) = 6.47, p < .05, reflecting the presence of an N2pc to the lateralized targets. As in Experiment 1, no interaction between face lateralization and target VF was observed, F(1, 11) = 0.06, p > .8, indicating that the N2pc to fearful faces was independent of the location of the search target.
We finally determined the time course of the N2pc responses to the target and to the lateralized fearful faces on the basis of sample-wise sliding t tests (see Methods) analogous to Experiment 1 (Figure 1D). Time ranges of significant waveform differences reflecting the N2pc are highlighted by horizontal bars in Figure 4C. The N2pc to fearful faces appears roughly in the same time range as the target-related N2pc, and as in Experiment 1, the onset of the N2pc to fearful faces is delayed by 25 msec, that is, by an amount that is comparable to Experiment 1 (20 msec). Also, the onset of the target-related N2pc of the bilateral fearful condition arises with a small delay (18 msec) relative to the bilateral neutral condition—a delay not observed in Experiment 1 (Figure 3C). To summarize, irrelevant fearful faces elicited an N2pc response that is completely independent of the target-related N2pc response. Despite the reduced perceptual demands on target discrimination in Experiment 2, size, time course, and current origin of the N2pc response to the fearful faces are very similar in both experiments.
The presented data from both experiments clearly demonstrate that a lateralized presentation of an irrelevant fearful face elicits a small but significant N2pc effect in ERMF response approximately between 240 and 400 msec that shows a typical current origin in ventral extrastriate cortex (Hopf et al., 2000). The results of Experiment 2 indicate that this effect is not altered when changing the perceptual demands of the task-relevant search. The N2pc has been shown to represent a reliable signature of rapid shifts of the attentional focus in visual search (Woodman & Luck, 1999, 2003b), even under conditions where target information does not reach the stage of conscious representation (Woodman & Luck, 2003a). Given that face valence in the present experiments was completely task irrelevant and nonpredictive with respect to the location of the search target, the observation of an N2pc appears to attest true automatic capture (Yantis, 1996), at least with respect to neuromagnetic correlates of attentional focusing. In other words, faces with negative emotional valence—even when being completely task irrelevant—do influence attentional processing in extrastriate visual areas to a certain degree in a mandatory manner. As detailed in the introduction, previous demonstrations of automatic attentional capture by fearful faces were challenged (Purcell et al., 1996) on the basis of the possibility that low-level featural differences between fearful and neutral faces were accounting for the behavioral effects. The present experiments extend those studies in showing that fearful faces capture attention when respective low-level confounds are eliminated. Experiment 2 reveals that when the perceptual demands of the search task are low, the presence of an irrelevant fearful face in the target VF reduces search speed, consistent with the notion that fearful faces impose a distracting influence on focusing onto the search target.
The present findings are generally in line with recent observations by Eimer and Kiss (2007) who reported that irrelevant fearful faces presented in the visual periphery elicited an N2pc in the ERP response while subjects performed a task requiring the detection of occasional luminance changes at fixation. A similar conclusion was drawn by Pourtois et al. (2004). They observed that when presenting an irrelevant fearful face versus a neutral face at the location of a subsequent target item, the ERP response to the target was enhanced around 130 msec (P1 attention effect), indicating that fearful faces led to automatic location precuing. It should be noted that although the present data demonstrate automatic attentional capture by fearful in comparison to neutral faces, the capturing effect may not be specific to the fear feature of faces. It is likely that other emotional expressions like sadness or happiness produce similar capture. In fact, a recent ERP study by Brosch, Sander, Pourtois, and Scherer (2008) demonstrates that irrelevant face stimuli eliciting positive emotions (nurturance-inducing baby faces) produce attentional capture as revealed by a P1 attention effect analogous to the one seen in Pourtois et al. (2004).
The present MEG studies combine the presentation of emotional face features with the simultaneous requirement to perform a demanding visual search task. This adds to these previous studies the possibility to evaluate the time course of operations underlying the automatic capture by fearful face features relative to the time course of neural processes subserving attentional focusing in visual search. The present data show that the N2pc to irrelevant fearful faces arises with an onset not earlier than the N2pc reflecting target selection. In fact, the time course is delayed relative to task-relevant processes of attentional focusing onto the bars by a comparable amount (20 and 25 msec) in both experiments, suggesting that perceptual load does not influence the relative time course of neural processing devoted to the target versus the fearful faces. Hence, fearful face expression modulates neural processing in extrastriate cortex automatically, but not with temporal priority. In Experiment 2, a small delay of the target-related N2pc is seen for the bilateral fearful relative to the bilateral neutral condition (Figure 4C). Response times to the bilateral neutral and fearful conditions, however, are not different, indicating that this delay may not relate to the response time slowing to unilateral fearful faces presented in the target versus the nontarget VF.
Evidence for mandatory and automatic processing of fear features in extrastriate cortex has also been provided with fMRI recordings. Vuilleumier, Armony, Driver, and Dolan (2001), for example, observed that fearful faces enhanced the BOLD response not only in the amygdala but also in the fusiform cortex even when faces appeared at irrelevant spatial locations. On the basis of converging evidence from fMRI lesion studies (Vuilleumier, Richardson, Armony, Driver, & Dolan, 2004), the authors speculate that feedback projections from the amygdala to extrastriate visual cortex entail modulatory influences on visual processing that are fast and independent of modulations arising from typical attentional control structures in parietal and frontal cortex (Vuilleumier & Driver, 2007). Analyses of fMRI-based functional connectivity have further corroborated that notion (Noesselt, Driver, Heinze, & Dolan, 2005). Given the independence of the N2pc responses to fearful faces and to the search target, the present data are clearly in line with the possibility that the N2pc to fearful faces reflects top–down modulatory influences from the amygdala. However, the onset latency of approximately 240 msec suggests a considerable delay relative to the initial feed-forward sweep of processing in visual cortex (Foxe & Simpson, 2002). The N2pc to fearful faces is delayed even with respect to the typical face-related response in the electro-magnetic signal (N170 and N200) (Allison, Puce, Spencer, & McCarthy, 1999; Bentin, McCarthy, Perez, Puce, & Allison, 1996) and is therefore less consistent with face valence information being transferred by a fast route that bypasses the canonical visual pathway (Pessoa et al., 2002). A delayed onset of this order, however, appears to be in line with the time course of modulatory effects observed with ERPs (Krolak-Salmon, Fischer, Vighetto, & Mauguiere, 2001) or recently with intracranial recordings (Krolak-Salmon, Henaff, Vighetto, Bertrand, & Mauguiere, 2004). Krolak-Salmon et al. (2004), for example, report that faces expressing emotion (fearful or happy) produced a marked enhancement of neural activity in the amygdala and fusiform cortex but approximately 180 msec onward after face presentation.
Dissociation between Neuromagnetic and Behavioral Data
Despite the presence of N2pc effects to irrelevant fearful faces in both experiments, behavior was not significantly influenced by the distribution of fearful faces in Experiment 1. In contrast, reducing the perceptual demands of target discrimination in Experiment 2 produced a significant effect on response time—an observation that fits with notions of the PLT of visual attention (Lavie et al., 2004; Lavie & de Fockert, 2003; Lavie & Tsal, 1994). PLT suggests that under high perceptual load, distractors may not have any effect on performance because perceptual capacity is exhausted, but when load is sufficiently reduced, idle resources may spill over to process distractor information. We may, therefore, conclude that irrelevant fearful faces modulate neural operations underlying attentional focusing and target discrimination in extrastriate cortex mandatorily, but this modulation may not relate to effects on the behavioral level unless the perceptual demands of the attention task are sufficiently low.
It should be noted that this conclusion partially conflicts with fMRI studies that have investigated this issue (Bishop, Jenkins, & Lawrence, 2007; Pessoa, Padmala, & Morland, 2005; Pessoa et al., 2002). Those studies observed that BOLD effects to irrelevant emotional faces in the amygdala and extrastriate cortex are eliminated when attentional resources are exhausted by a demanding discrimination task. Although the present data suggest such load dependency at the behavioral level in line with Pessoa et al. (2005), we find neural correlates in the neuromagnetic response under high load conditions of Experiment 1. Of course, it is possible that the search task in Experiment 1 did not exhaust attentional resources as much as the orientation discrimination task in the Pessoa et al. (2002) study. Also, there are critical differences in experimental design and methodology. In the studies of Bishop et al. (2007) and Pessoa et al., (2002, 2005), faces were presented at fixation, whereas in the present study faces appeared at peripheral VF locations. That is, the way faces influenced attentional selection presumably differed between studies. Furthermore, studies performed with fMRI may not be as sensitive as MEG recordings in assessing brief and transient activity differences. Further research is clearly needed to clarify the discrepancies.
Nevertheless, the apparent dissociation between neurophysiological measures of attention and performance has a notable implication. Under high perceptual load performance, effects are negligible while neural correlates reflecting attentional selection in extrastriate visual areas are clearly observed. Reducing perceptual load leads to significant behavioral effects but does not change the neural correlates of attention in extrastriate cortex. This suggests that the ultimate locus where resources trade for controlling behavior (in the sense of PLT) may not be the locus of sensory/attentional selection reflected by the N2pc. This is notable as the N2pc is known to reflect the attenuation of distractor interference in visual search (Hopf, Boelmans, et al., 2002; Luck, Girelli, McDermott, & Ford, 1997), and PLT in fact suggests that load effects pertain to the reduction of distractor interference during stimulus encoding. Although this appears to be puzzling, it is important to acknowledge that the perceptual encoding of stimuli is likely mediated by multiple selection processes and the N2pc does not reflect all of them (Hopf et al., 2000). It is possible that behavioral effects under low load result from a spill of resources from perceptual processes that attentional selection reflected by the N2pc does not draw on.
In sum, despite variable effects on the behavioral level, the presented MEG data indicate that completely irrelevant fearful faces influence attentional processing in extrastriate cortex in an automatic way. This automatic influence turns out to be unaltered by a change of the perceptual demands of the task-relevant search. Automaticity, however, does not imply temporal priority. Fearful face associated processing in extrastriate cortex is found to arise with a slight delay relative to task-relevant attentional processing. Our observations add to a growing body of neurophysiological research suggesting privileged processing of emotional stimuli (Brosch et al., 2008; Eimer & Kiss, 2007; Anderson, 2005; Noesselt et al., 2005; Vuilleumier, 2002).
Although it is widely suggested that fearful faces capture attention in an automatic fashion, this notion has been questioned after a critical evaluation of the available behavioral evidence (Cave & Batty, 2006; Horstmann & Bauland, 2006; Purcell et al., 1996). Here, we add to this debate by providing neurophysiological evidence that irrelevant fearful faces, indeed, “capture” sensory processing subserving attentional selection in extrastriate visual cortex in an automatic way, even when effects on behavior are nonsignificant.
This research was made possible with the support provided by the DFG-Grant, KFO 163, TP4.
Reprint requests should be sent to Jens-Max Hopf, Department of Neurology, Otto-von-Guericke University, Leipziger Strasse 44, 39120 Magdeburg, Germany, or via e-mail: email@example.com.