The ability to attend to an input selectively while ignoring distracting sensations is thought to depend on the coordination of two processes: enhancement of target signals and attenuation of distractor signals. This implies that attending and ignoring may be dissociable neural processes and that they make separable contributions to behavioral outcomes of attention. In this study, we tested these hypotheses in the context of sustained attention by measuring neurophysiological responses to attended and ignored stimuli in a noncued, continuous, audiovisual selective attention task. We compared these against responses during a passive control to quantify effects of attending and ignoring separately. In both sensory modalities, responses to ignored stimuli were attenuated relative to a passive control, whereas responses to attended stimuli were enhanced. The scalp topographies and brain activations of these modulatory effects were consistent with the sensory regions that process each modality. They also included parietal and prefrontal activations that suggest these effects arise from interactions between top–down and sensory cortices. Most importantly, we found that both attending and ignoring processes contributed to task accuracy and that these effects were not correlated—suggesting unique neural trajectories. This conclusion was supported by the novel observation that attending and ignoring differed in timing and in active cortical regions. The data provide direct evidence for the separable contributions of attending and ignoring to behavioral outcomes of attention control during sustained intersensory attention.
Attention is thought to guide our behavior by directing neurocognitive resources toward sensations that are of interest and away from those that are not currently relevant (Desimone & Duncan, 1995). Support for this idea stems from copious evidence showing that neural activity is enhanced for attended targets relative to ignored distractors during selective attention (Kastner & Ungerleider, 2001; Desimone & Duncan, 1995; Mangun, 1995; Hillyard, 1985). Moreover, evidence is mounting to suggest that ignoring distractors, like attending to targets, is an independent control process driven by top–down input from associative control structures including the prefrontal and parietal cortices (Chadick & Gazzaley, 2011; Ruff & Driver, 2006; Serences, Yantis, Culberson, & Awh, 2004), and its effect is to down-regulate neural responses to distractors (Gazzaley, 2011; Hopf et al., 2006; Kelly, Lalor, Reilly, & Foxe, 2006; Ruff & Driver, 2006; Moran & Desimone, 1985). This implies that attending to targets and ignoring of distractors should make separable contributions to behavioral outcomes of attention.
This hypothesis is important not only to elucidate the mechanisms of attention control but also to expose the sources of attention impairments. Recent studies that measure target and distractor handling as separable processes have shown that our ability to ignore distractions can be impaired independently from our ability to attend targets, an idea supported by reports of impaired distractor suppression with increased task load (Rissman, Gazzaley, & D'Esposito, 2009), cognitive aging (Gazzaley, Cooney, Rissman, & D'Esposito, 2005), and sleep deprivation (Kong, Soon, & Chee, 2012), without a decrement in target enhancement. Further questions remain regarding whether the inverse situation, a decrement in target enhancement without a deficit in distractor suppression or a deficit in the processing of either stimulus stream, is also possible, whether these scenarios are related to a particular disease state of attention deficit, whether they arise from separable cortical pathways, and how these fluctuate across time.
To address these questions, an initial step is to demonstrate that both attending to targets and ignoring of distractors contribute to attention in a nondisease state of attention control. Pioneering studies examining anticipatory attention have shown that valid cues decrease RT to subsequent stimuli relative to neutral cues, whereas the opposite is observed for invalid cues (Posner, 1980; Posner, Nissen, & Ogden, 1978), and cued enhancement and suppression have also been reported in neural responses (Luck et al., 1994). The imperative manipulation in these studies was the use of a neutral cue, which allowed for the interpretation of signal modulations as either enhancement or suppression, an inference not possible given only the contrast of attended versus ignored conditions. A similar approach was employed effectively in the context of attention–memory interactions (reviewed in Gazzaley, 2011; also Kong et al., 2012; Johnson & Zatorre, 2005, 2006) where sensory activity during encoding showed enhancement and suppression for relevant and irrelevant stimuli, relative to passive viewing, and in some cases suppression was predictive of participants' subsequent recall (e.g., Zanto & Gazzaley, 2009; Gazzaley, Cooney, Rissman, et al., 2005).
In this study, we were interested in testing if the effects of attending on target processing and ignoring on distractor processing would also be present and impact performance in sustained attention. This condition differs from anticipatory attention, as it relies on the continuous deployment of attention control across time and is therefore indicative of ongoing control processes (Braver, Gray, & Burgess, 2007; Braver, Reynolds, & Donaldson, 2003). Sustained attention is of particular interest because its impairments are increasingly relevant to the understanding of attention deficits in disease states, such as attention-deficit/hyperactivity disorder (Castellanos & Proal, 2012; Huang-Pollock, Nigg, & Halperin, 2006; Biederman & Spencer, 1999). Being able to separately measure effects of attending and effects of ignoring in this context, along with an assessment of their relationship to behavior, will facilitate future studies of their mechanisms, interactions, and conditions for dysfunction. Two fMRI studies have demonstrated the presence of both an attending effect (relative to passive) and an ignoring effect (relative to passive) in the context of sustaining attention (Daffner et al., 2012; Johnson & Zatorre, 2005), although the relationship of these effects to performance could not be ascertained. Also using fMRI, Weissman, Warner, and Woldorff (2009) showed a significant negative correlation between magnitude of activation in sensory regions for target processing and a positive correlation between magnitude of activation in sensory regions for distractor processing and RT. These findings suggest that an analysis of the neural and temporal dynamics of attend and ignore effects (changes relative to a passive control) and of their relationship to behavior during sustained attention are warranted.
In the current study, we therefore sought to expand on the fMRI findings. First, we asked if both attending-related processes and ignoring-related processes are involved in the control of intersensory (audiovisual) attention and whether these processes comprise changes in sensory activity relative to a passive control. We used EEG methodology to evaluate the timing of these processes, thus expanding on the fMRI results. We chose audiovisual attention, as this is a domain in which electrophysiological attention effects—measured as differences between attended and ignored signals—have been characterized in ERPs (Alho, Woods, Algazi, & Näätänen, 1992; e.g., Foxe & Simpson, 2005; Foxe, Simpson, Ahlfors, & Saron, 2005; Teder-Salejarvi, Di Russo, McDonald, & Hillyard, 2005; Talsma & Kok, 2001; Woods, Alho, & Algazi, 1992; Hillyard, 1985; Hillyard, Simpson, Woods, Van Voorhis, & Munte, 1984). In addition, we asked whether these attend and ignore effects predict both accuracy and RT between participants. Our results suggest that attending can operate at different stages of processing than ignoring and that both attending and ignoring make separable contributions to ongoing performance during sustained attention. Our results are consistent with the conclusion that attending and ignoring processes follow different neural trajectories.
We tested 35 healthy right-handed individuals (22 women, age: = 21.0 years, σ = 5.4), recruited through the Psychology department subject pool at the University of California, Los Angeles. All participants provided written informed consent. This study was approved by the local investigational review board.
Experimental Design and Procedures
Task and Stimuli
An overview of the experimental protocol is portrayed in Figure 1. We presented participants with two streams of stimuli, one auditory and one visual, and instructed them to direct their attention according to one of three instruction cues. In the attend visual condition, their task was to attend and respond to visual stimuli while ignoring the auditory stream. In contrast, in the attend auditory condition, their task was to attend and respond to auditory stimuli, ignoring the visual stream. These two conditions were designed to elicit attention control by requiring participants to attend actively to one stream and to suppress the irrelevant stream. Attending and ignoring were compared with a passive control condition, in which participants received both streams of stimuli but were instructed to neither attend nor respond to any stimuli while maintaining gaze fixation at the center of the screen.
We calculated the effect of attending by contrasting the EEG neural response for one domain's stimuli in the corresponding attend condition (e.g., visual stimuli in attend visual) against that in the passive control. Similarly, we calculated the effect of ignoring by comparing the EEG neural response for same-domain stimuli while attending to the other domain (e.g., visual stimuli in attend auditory) against the passive control.
In each of the attend conditions, participants made binary decisions for each attended-modality stimulus and made a forced-choice response to each. In the attend visual condition, participants had to decide if the visual stimuli, circular Gabor patches, were oriented vertically (standard orientation) or if they were tilted diagonally (off-vertical). These sinusoidal gratings were 5.7 in. in diameter, with spatial frequency of 1.36 cycles/deg alternating between gray and white, and were presented centrally on a gray background of a 20-in. PC monitor (1680 × 1050 resolution, refresh rate 60 Hz, Dell, Round Rock, TX). In the attend auditory condition, participants had to decide if the auditory stimuli, binaurally presented tones, were at a standard frequency of 700 Hz or if they were higher/lower.
We customized the nonstandard stimuli—the off-vertical visual and non-700-Hz auditory stimuli—to each participant using a 3-up/1-down staircase protocol (Garcia-Perez, 1998; k = 0.3, delta ratio = .7393) to ensure that the participants were able to discriminate the stimuli (e.g., vertical vs. off-vertical left/right and 700 Hz vs. higher/lower) with at least 83% accuracy. This produced an average off-vertical rotation of 12° counterclockwise (SE = 3.9°) and 11° clockwise (SE = 3.7°) relative to vertical and 598 Hz (SE = 23 Hz) and 803 Hz (SE = 25 Hz) relative to 700 Hz. We included two nonstandard stimuli in each domain to curb effects of neural habituation and boredom. In any given miniblock (see Design) only one of the two nonstandard stimuli, randomly selected, was presented along with the standard stimuli (i.e., vertical and 700 Hz). Standard and nonstandard stimuli occurred with equal probability within a miniblock.
The participants viewed and listened to the two streams simultaneously, but the stimulus onsets were independent across streams. In each stream separately, we sampled the ISIs randomly from a uniform distribution ranging from 700 to 2000 msec. To prevent multisensory effects on perception (Spence & Squire, 2003), we adjusted the ISIs subsequently to ensure that no two stimuli occurred within 350 msec of one another. As a result, the ISI range for stimuli within the same sensory modality was 850–2300 msec, and the ISI range for any two stimuli was 350–800 msec. We adopted this scheme to minimize expectation-based strategies and thus maximized the interfering effect of the ignored stream on the attended stream. All stimuli had durations of 100 msec.
Participants responded using index and middle fingers of the right hand and the “<” and “>” keys of a QWERTY keyboard. Finger assignments to responses were varied randomly across participants. We constructed auditory pure tones (sampled at 22,050 Hz, 10 msec ramp up and down) and visual gratings in Matlab (Mathworks, Natick, MA; v7.10). The experiment was programmed in PsychToolbox (Kleiner, Brainard, & Pelli, 2007; Brainard, 1997), running on an Apple MacBook Pro computer (Apple Computer, Cupertino, CA; OS 10.6.8).
The participants performed each task in 40-sec “miniblocks,” each of which comprised 25 auditory and 25 visual stimuli. A single-word audiovisual instruction (presented both on screen as a written word and spoken aloud through the speakers) was presented before the miniblock. Instruction stimuli were presented for 1 sec, followed by a 3-sec gap, and then by the 40-sec miniblock. This was followed by a 17-sec break period. Each block comprised two miniblocks of each of the three task conditions (attend auditory, attend visual, passive), ordered randomly across blocks and participants. Participants performed a total of four blocks of the miniblock design, resulting in eight miniblocks or 200 trials per condition.
EEG Data Acquisition
During the task, we acquired EEG data using a 256-electrode HydroCel Geodesic Sensor Net (EGI, Eugene, OR), digitized using a Net Amps 300 amplifier (10,000 Hz anti-aliasing filter; common-mode rejection 90 dB; input impedance 200 MΩ) and sampled at 250 Hz. We kept electrode impedances below 50 kΩ. During the recording, electrodes were referenced to the vertex electrode but were re-referenced offline to a common average. We used NetStation software (v.4.4; EGI) to control the acquisition.
Behavioral Data Analysis
We assessed performance separately for auditory and visual stimuli within the attend condition. For each stimulus modality, we calculated the median RT and accuracy and assessed modality differences by a paired t test (two-tailed), p < .05.
We preprocessed and analyzed the data using the EEGLAB toolbox (Delorme & Makeig, 2004; v. 10.2.5.5). We first applied a high-pass filter (>1 Hz) and inspected the data visually for noisy channels which, when present, were replaced using spherical interpolation. On average, this process led to removal of 27 electrodes (SE = 3) out of the 256 (10%); >90% of these electrodes were located over face and ears and at the edge of the high-density net where contact between electrodes and skin is easily compromised by head motion. To isolate artifacts, we submitted the data to extended INFOMAX temporal ICA (Lee, Girolami, & Sejnowski, 1999). We identified independent components visually (ICs) that accounted for eye movements, muscle, and high-frequency noise, which we removed from the raw data. We then extracted data epochs from 100 msec before to 500 msec after stimulus onset. We inspected these visually for the presence of any remaining high-amplitude noise; if present, we removed these epochs from further analysis. Across participants, this procedure resulted in 185 epochs (SD = 9.5) remaining in each condition. We defined the baseline as the 100-msec prestimulus period and subtracted the mean of this period from the poststimulus interval. For visualization, we applied a 30-Hz filter to ERPs—we used unfiltered data for computing statistics.
In our first set of analyses, we tested if attending and ignoring modulated neural responses to stimuli in reference to a passive control, offering evidence that both contribute to selective attention. We defined an attend effect as the difference in response amplitude when attending to stimuli relative to the passive condition. We defined the ignore effect as the difference in potential when ignoring stimuli relative to the passive condition. Note that modulation of sensory cortex, enhancement of neural activity by attending and suppression of neural activity by ignoring, may be expected to have opposite effects on the ERP amplitude relative to its value during the passive control.
We were particularly interested in testing these effects for ERP components that have known correspondence with sensory processing and attention (Hillyard & Anllo-Vento, 1998; Hansen & Hillyard, 1988). For visual stimuli, these were P1 and N1 (positive and negative peaks, respectively, occurring around 100 and 150 msec after stimulus onset and with an occipital scalp distribution) and N2 (negative peak occurring around 200 msec after stimulus onset, with an occipital scalp distribution). For auditory stimuli, these were P1/N1 and Nd (a “negative difference,” corresponding to a sustained negativity beginning approximately 140–180 msec after stimulus onset and lasting about 200 msec). The distinctions between the sequential components within each modality are with respect to stage of processing. The P1/N1 in the visual domain and the P1/N1 in the auditory domain reflect sensory processing before stimulus identification. The subsequent visual N2 and auditory Nd—referred to as “selection” and “processing” negativities, respectively—have been linked with additional sensory and attention processing, such as matching of a stimulus against an internal template (van der Stelt, Kok, Smulders, Snel, & Boudewijn Gunning, 1998; Näätänen, 1982). We were interested in distinguishing at which stages the effects of attending and ignoring would be observed. We wanted to use these ERP components as ballpark measures for different stages of attention effects but recognize that the timing of ignoring and attending effects relative to a passive condition may not correspond with these components or the attention effects associated with them. In addition, the findings in brain space are likely to reveal further differentiation from what is seen at the scalp.
For each of these potential components, we first isolated the latency of peak activity by inspecting the scalp topography of the mean potential across all conditions. At this latency, we then converted the mean value at each electrode to a t statistic and used a scree plot to select, for further analysis, those electrodes that had the largest effect size. This approach allowed us to isolate electrodes that captured the spatial extent of the scalp response of components of interest, which we felt was a more appropriate strategy than picking single electrodes. Attend and ignore effects were tested on the mean potential across these electrodes, using a paired t test (two-tailed), at all time points between 50 and 400 msec after stimulus onset to account for peak variability and to explore attention effects across the entire stimulus response. We applied a false discovery rate (FDR) correction, p < .05, to correct for multiple comparisons across multiple time points.
Analysis of Brain–Behavior Relationships
In our second set of analyses, we were interested in testing the hypothesis that when attention operates on two competing parallel streams (attended and ignored) the outcome in each stream ought to influence performance.
To test this idea, we analyzed the attend affects for the attended modality (e.g., visual in attend visual) as well as the ignore effects for the ignored stimulus modality (e.g., auditory in attend visual), as a function of accuracy and median RT. We split our participants into two groups based on performance. For each modality and each performance measure, we identified the top and bottom 30% of individuals ranked on that performance measure (n = 10 in each group). We then conducted an independent samples t test to test if individuals with higher performance showed greater attend effects to the target stimuli than individuals with lower performance. We conducted a similar analysis to test if individuals with higher performance show greater ignore effects for the distractor stimuli than individuals with lower performance.
To assess the relationship of attend effects with performance within participants, we performed a split-group analysis on single trials. For each participant and modality, within each condition, we ranked the single trials by the trial RT. The top and bottom 40% of trials were then averaged. The effects of attending on ERPs (see Activation Analysis) were then calculated across participants with a paired t test, comparing top and bottom RT percentile trials. In this manner, we tested whether faster response trials are associated with stronger effects of attending to the target stream. Note that because our design required that distractor stimuli occur independent of target stimuli, distractor stimuli could not be related to target responses in a one-to-one manner; this analysis was performed therefore for attended but not for ignored stimuli. All behavior analyses were conducted on the mean of the five electrodes showing the strongest group attend and ignore effects and at corresponding peak latencies.
In a final activation analysis, we asked if attending and ignoring modulated the activity of sensory cortex. To address this question, we performed a source imaging analysis for the auditory and visual ERPs. We employed Brainstorm software (Tadel, Baillet, Mosher, Pantazis, & Leahy, 2011) and defined the lead field matrix using the software's default anatomy; FreeSurfer's (Dale, Fischl, & Sereno, 1999; Fischl, Sereno, & Dale, 1999) fsaverage brain, which represents the average surface anatomy of 40 participants. We parcellated the anatomy to 15,000 vertices and constructed the lead field using a boundary element model (OpenMEEG; Gramfort, Papadopoulo, Olivi, & Clerc, 2010; Kybic et al., 2005). The noise covariance matrix was estimated from prestimulus baselines. To compute sources, we used the whitened and depth-weighted linear L2-minimum norm estimates algorithm (Baillet, Mosher, & Leahy, 2001; Tikhonov λ = 10%), with the constraint that source orientations are normal to the cortex. Before statistical testing, to minimize between participant variability, each participant's source results were smoothed spatially using a 10-mm (FWHM) Gaussian filter. As in the activation analysis, we first performed an omnibus test, using a one-sample t test, to evaluate the reliability of mean source activations (collapsing across conditions) at time points of interest. We applied an FDR correction, p < .001, correcting for multiple comparisons in both time and across space (vertices of lead field). Within significantly activated sources, we then evaluated effects of attending (attend vs. passive) and ignoring (ignore vs. passive) using a paired t test, p < .01 (uncorrected).
Our participants were overall faster when attending to auditory stimuli ( = 511 msec, SE = 9 msec) than when attending to visual stimuli ( = 562 msec, SE = 6 msec), t(34) = 6.31, p < .001 (two-tailed). Accuracy on the attend auditory task ( = .94, SE = .01) was not significantly different than that on the attend visual task ( = .93, SE = .01), t(34) < 1.
Attend and Ignore Effects and ERPs
The effect of attending to visual stimuli was significant in the N2 cluster of occipital electrodes (Figure 2B) from 188 to 267 msec (Figure 3A and B, red) after stimulus onset. The N2 negativity was enhanced in the attend visual task relative to passive (Figure 2A and B). No effects were significant in other electrode clusters or time points.
The effect of ignoring auditory stimuli during this visual task was significant in the Nd cluster of frontocentral electrodes (Figure 2E) from 56 to 152 msec after stimulus onset (Figure 3C and D, red). The potential at these sites was more positive for the ignored auditory stimuli relative to passive (Figure 2E and F). We refer to this early positivity as an auditory “P1.” Notably, during the N1 (144 msec), the scalp topography appeared to shift anteriorly for the ignore condition relative to passive or attend conditions (Figure 2F vs. Figure 2D, E), suggesting a change in source activity.
Attend and Ignore Effects and Performance
To evaluate if the N2 visual attend and P1 auditory ignore effects contribute to visual task performance, we first identified the five electrodes and the time point at which attend and ignore effects were greatest within the significant interval of interest (indicated in Figure 4). Our percentile split analysis of these data based on attend visual accuracy ( = .90, SE = .007; = .97, SE = .004) revealed that the effect of attending (attend–passive) to visual targets on the N2 (224 msec) was significantly stronger, t(18) = 2.00, p < .05, for the high-accuracy group (−1.72 μV) than for the low-accuracy group (−1.15 μV; Figure 4, left). In addition, the effect of ignoring (ignore–passive) of auditory distractors on the auditory P1 (92 msec) was also stronger, t(18) = 2.14, p < .05, in the high-accuracy group (.91 μV) than in the low-accuracy group (.34 μV; Figure 4, right). These findings support the conclusion that both the attend effect for visual targets and also the ignore effect for auditory distractors—when attending to visual stimuli—contribute to accuracy.
The same analysis based on RT did not show significant differences between fast and slow responders for either attend or ignore effects, t(18) < 1, possibly because participants were instructed to respond as quickly as possible; the fast pace of the task limited variability among the groups ( = 533 msec, SE = 13 msec; = 563 msec, SE = 12 msec). The N2 attend effect for visual targets was, however, significantly different across trials in the within-subject analysis. As shown in Figure 6 (left), the N2 attend effect was significantly stronger, t(18) = −3.52, p < .001, for fast response trials (−1.81 μV) than for slow response trials (−1.03 μV).
Attend and Ignore Effects and ERPs
The effect of attending to auditory stimuli was significant in the Nd cluster of frontocentral electrodes (Figure 2E) from 136 to 284 msec (Figure 3C and D, blue) after stimulus onset. The negative difference wave during this period (Nd) was greater in the attend auditory task relative to passive (Figure 2D and E). No effects were significant in other electrode clusters or time points.
The effect of ignoring visual stimuli during this auditory task was not significant in any of the occipital electrode clusters (Figure 2B, C and Figure 3A, B, blue). The auditory task was therefore associated only with an attend effect; visual stimulus scalp responses did not appear to show an ignore effect. However, see Brain Sources of Audiovisual Attention Effects section below, in which an early effect of ignoring is found in the brain source results.
Attend and Ignore Effects and Performance
As for the visual task, we tested the relationship between attending and ignoring and performance at peak electrodes and time points. Our percentile split analysis of these data based on auditory attend accuracy ( = .88, SE = .014; = .98, SE = .003) revealed that the effect of attending (attend–passive) to auditory targets on the Nd (204 msec) was significantly stronger, t(18) = 2.36, p < .03, for the high-accuracy group (−1.54 μV) than for the low-accuracy group (−.68 μV; Figure 5, right). Because no ignore effect was present for the visual distractor, we used the peak and latency parameters from the effect of attending to visual stimuli instead (indicated in Figure 5). We reasoned that perhaps an effect of ignoring would be revealed in this time window (N2) for individuals with highest performance. However, the effect of ignoring (ignore–passive) of visual distractors on the visual N2 (224 msec) did not differ, t(18) < .05, between the low-accuracy (−1.0 μV) and high-accuracy participants (−1.25 μV; Figure 5, left). These data indicate that the attend effect for auditory targets, as in the visual task for visual targets, contributes to task accuracy. Consistent with the group result, the visual distractors did not show an effect of ignoring relative to passive.
Analogous to the visual task, the split percentile analysis based on RT did not show significant differences between fast and slow responders for either attend or ignore effects, t(18) < 1. Again, a plausible cause was lack of RT variability among participants ( = 504 msec, SE = 18 msec; = 516 msec, SE = 18 msec). As before, the attend effect was significantly different across trials in the within-subject analysis. As shown in Figure 6 (right), the Nd attend effect for attended auditory targets was significantly stronger, t(18) = −2.97, p < .006, for fast response trials (−1.33 μV) than for slow response trials (−.91 μV).
Brain Sources of Audiovisual Attention Effects
The activation and behavior analyses illustrated the presence of both attending and ignoring effects in audiovisual attention and a relationship between these brain processes and performance. In an additional analysis, we decomposed the ERPs within each modality (Figure 2) into their cortical sources—additionally testing the hypothesis that attending and ignoring effects reflect modulation of sensory cortex activity.
Sources of Visual Evoked Potentials
In the visual sensory domain, we tested if the attention effects observed in the ERPs corresponded to modulation of occipital cortex. The source analysis for visual ERPs (Figure 7) indicated that, during the P1 (116 msec), the mean across-condition response to visual stimuli was associated with activation within lateral occipital (latOcc) cortex as well as inferior and superior parietal cortex (iPL, sPL; Figure 7A, left). The attend effect during the P1 (attend–passive, Figure 7B, left) was associated with activity increases in latOcc cortex, iPL, FEF, and middle frontal gyrus (mFG). The ignore effect during the P1 (ignore–passive; Figure 7C, left) had weaker effects (p < .05) that included a decrease of activity within lateral and inferior occipital cortices along with activity increases in mFG, much of which overlaps with the mFG activity in the attend effect (Figure 7B, left). This region and the iPL activations have been associated with visuospatial attention control (Corbetta & Shulman, 2002), potentially indicating here associative sensory cortex interactions. We note that the P1 ignore effects were absent in the scalp ERPs, possibly because of signal mixing at the scalp or because of the electrodes that were selected for testing. Consistent with this notion and the source results, the scalp topography of the P1 (Figure 2A–C) showed a P1 that qualitatively decreased across the most inferior occipital electrodes from attended to passive and (to a lesser extent) to the ignore condition. However, our selected electrodes extended dorsally, where this trend was qualitatively weaker (or absent), and possibly blurring out the ignore effect in the across-electrode mean ERP.
During the N1 (144 msec), both the mean across-condition source strength and the attend effect (Figure 7A and B, middle) were associated with increased activity within latOcc and iPL cortices. During the processing negativity, N2 (228 msec), additional associative regions were active, spanning the junction of angular gyrus, superior and middle temporal cortex, and prefrontal cortex (Figure 7A and B, right). The ignore effects during the N1 and N2 were associated with weak localized increases and decreases in activity within prefrontal, superior temporal, and occipital cortices.
Sources of Auditory Evoked Potentials
In the auditory modality, we tested if attend and ignore effects were associated with modulation of auditory cortex. The source analysis confirmed that the early P1 mean activation (Figure 8A, left) included temporal and prefrontal activations. As in the scalp results, no attend effect was detected during the P1 (Figure 8B, left). The ignore effect (Figure 8C, left) was associated with decreased activity in the superior temporal plane (sTP, planum temporale), temporal–parietal junction, and increased activation within anterior superior temporal gyrus (sTG), as well as both inferior frontal gyrus (iFG) and mFG, extending into insula. This finding is suggestive of active suppression involving interactions between associative regions and secondary auditory cortex and is reminiscent of the weak ignore effect in the visual task (Figure 7B and C, left) that also comprised a decrease of activity in sensory (visual) cortex and an increase of activity in mFG.
During the N1 (144 msec), the mean cortical activation included sTP as well as parietal, frontal, and temporal associative cortices (Figure 8A, middle). The attend effect showed an increase in activity in sTP, whereas the ignore effect showed a decrease (Figure 8B and C, middle). This change in network configuration between attending and ignoring may have contributed to the change in the scalp topography during the N1 (Figure 2F). The presence of both attend and ignore effects is generally consistent with the scalp results, which showed a temporal overlap in attend and ignore effects for auditory stimuli (Figure 2D) between 136 and 152 msec. Finally, during the processing negativity (Nd, 228 msec), additional prefrontal activation was present in both attending and ignoring effects (Figure 8B and C, right).
In this study, we demonstrated modulation of ERPs with both attending to and ignoring of stimuli, relative to a passive control, in sustained attention. The temporal stages of processing affected by attending differed from the stages affected by ignoring. The brain regions involved in attending and ignoring were similar in some stages and differed in others. The magnitudes of attend (in both tasks) and ignore effects (in the visual task) were predictive of participants' task accuracy, and the magnitude of the attending effect was additionally predictive of RT variability across trials. The results of our study support the hypothesis that attending and ignoring are separable processes, and both contribute to ongoing goal-directed behavior.
Contributions of Attending and Ignoring Processes to Sustained Attention
A prominent finding in this study is that of a relationship between performance and the size of the effects of both attending and ignoring relative to a passive control. It is striking that the magnitude of the effect of ignoring of a task-irrelevant stream of stimuli in one modality affected participants' accuracy on the attended stream in the other modality, a finding that to our knowledge has never been reported. Prior EEG studies of cross-modal attention have shown a modulation of the ERP to stimuli when they were attended versus when they were unattended (Green & McDonald, 2006; Talsma & Kok, 2002; Eimer & Driver, 2001; Teder-Salejarvi, Hillyard, Roder, & Neville, 1999); however, without a neutral reference such modulations do not separate the contributions of attending from those of ignoring. To address this, in a series of fMRI studies, Johnson and Zatorre (2005, 2006) measured enhancement and suppression of auditory and visual sensory cortex relative to a set of passive control conditions. These studies provided an in-depth characterization of sensory cortex enhancement and suppression effects. Although their work examined these effects during sustained attention, it did not assess their relationship to continuous performance. To this end, Weissman et al. (2009) reported that greater activity in target-related sensory cortex and decreased activity in distractor-related cortex predict RT in selective attention, which resonates with our findings. This is an important result that also demonstrates the relationship between target processing and distractor processing and performance. Unlike our approach, that study did not employ a neutral condition to directly assess the contributions of attending and ignoring per se as independent processes. Our results therefore bring the observations in the literature together by showing modulation of target and distractor activity relative to a neutral reference, their relationships to performance, and that they occur within a sustained attention context.
The ability to separate attention control into attending and ignoring processes makes a strong statement about the mechanisms of attention control. Namely, it is possible that effects that appear in distractor processing are entirely a byproduct of target attending. For instance, one alternative to two separable mechanisms is a yoked system in which the active process is a top–down initiated increase in activation for targets and the decreased activation of distractors is because of inhibitory connections arising from target processing regions. As target signals are enhanced, any competing distractors signals are suppressed. A further alternative is that suppression of distractor signals is illusory, arising from the removal of enhancement to the ignored stream when attentional resources shift to the target processing regions. For instance, if attention in our passive condition were randomly and evenly alternated between streams, then even if the only tool in the attention toolbox were attending, we would observe an ignore effect—activation in the distractor stream would appear to decrease during ignoring because attention was removed, not because these signals were suppressed. Indeed, Johnson and Zatorre (2005) have shown that bimodal, passively perceived stimuli elevate the neural response in each modality above the case in which no stimuli are presented, and it is unclear whether such activation occurs because of passive sensory activation or because of randomly distributed attention. A similar criticism applies to the use of neutral cues as a referent, which creates a divided attention condition in which attention is “broadly focused” across stimuli (Luck et al., 1994; Corbetta, Miezin, Dobmeyer, Shulman, & Petersen, 1990). Johnson and Zatorre (2006) demonstrated that in fact passive viewing and divided attention can have similar effects on activation in sensory cortex.
Indirect evidence that speaks against a yoked mechanism or allocation of attention alone without active ignoring of distractors is the finding that distractor processing can be manipulated independently of target processing (Chadick & Gazzaley, 2011; e.g., Payne & Allen, 2011; Mevorach, Hodsoll, Allen, Shalev, & Humphreys, 2010; Sylvester, Jack, Corbetta, & Shulman, 2008; Ruff & Driver, 2006; Serences et al., 2004). In visual search, also, attention has been recently shown to modulate attended targets separately from ignored distractors—eliciting contralateral negativities for attended targets and contralateral positivities for ignored distractors (Hickey, Di Lollo, & McDonald, 2009). Our findings provide additional evidence; in particular, we found that within the auditory modality ignoring and attending had separable time courses. Moreover, the brain source results indicated that attending and ignoring can involve different brain regions (Figure 7B, P1), meaning that they can follow independent trajectories in the brain. Within a yoked system, we would expect attend and ignore effects to be correlated perfectly as attention either directs the limited processing resources away from the ignored stimuli and toward the attended stimuli or passively inhibits activity within competing sensory cortices. We tested this assertion directly, considering that even spatially and temporally separable mechanisms may be correlated in magnitude (assuming a linear relationship), but did not find the correlation to be significant in peak effects either in the auditory task (r < .01, p > .9) or in the visual task (r < .08, p > .7). As such, our findings are more parsimoniously interpreted as showing functionally separable processes of ignoring and attending, each of which contributes to performance.
An additional important finding in this report in support of active ignoring is the relationship between ignoring and performance. A relationship between the magnitude of suppression in sensory cortex, relative to a passive baseline, and performance has also been documented by Gazzaley and colleagues (reviewed in Gazzaley, 2011), who demonstrated that the degree of suppression of ignored stimuli predicts subsequent memory recall (Clapp & Gazzaley, 2012; Rutman, Clapp, Chadick, & Gazzaley, 2010; Berry, Zanto, Rutman, Clapp, & Gazzaley, 2009; Zanto & Gazzaley, 2009; Gazzaley, Cooney, Rissman, et al., 2005). Similarly, better performance in video-gamers during sustained attention was associated with smaller neural responses to distractors compared with a control group; however, this between-group comparison of distractor processing did not measure suppression against a neutral control (Mishra, Zinni, Bavelier, & Hillyard, 2011). Interestingly, these studies did not isolate a relationship between attending and performance. In contrast, we have shown that the magnitude of attention processes in intersensory sustained attention contributes to speed of RT as well as accuracy. This discrepancy, whether attending or ignoring or both contribute to performance, suggests that the relative contribution of attending and ignoring processes to behavior may be sensitive to contextual factors such as relative task demands. In the memory task reported by Gazzaley and colleagues, suppression of distractors may have been more important than target enhancement in determining the contents of memory. For instance, if participants were at ceiling in encoding the targets, then only the degree of interruption by the distractor would influence recall. In our sustained attention task, the control of both streams of information was necessary, perhaps because both streams could influence the behavioral outcome (response).
In our study, task demands also appeared to influence results. Overall, the effect of ignoring was more robust in the auditory modality than in the visual modality. The precise reason for this observation is unclear; one possibility is that auditory stimuli were easier to ignore than visual stimuli, perhaps because our visual stimuli were foveally presented, perhaps because there are inherent differences in control of inputs across modalities. Some authors have suggested that auditory stimuli have privileged access to attention because they provide an “early warning system” (Dalton & Lavie, 2004), which would also imply that ignoring of auditory stimuli may require action earlier in the processing stream than ignoring of visual stimuli or attending to auditory stimuli. In our data, this occurred promptly after stimulus onset (56 msec) and involved both auditory and prefrontal activations, consistent with the latter interpretation.
Dynamics of Top–Down Influences in Selective Attention
What are the neural mechanisms by which enhancement and suppression arise? A prominent hypothesis is that top–down inputs from associative control structures such as the prefrontal and parietal cortices to sensory cortices modulate sensory activity (Miller & Cohen, 2001; Desimone & Duncan, 1995). Our observations are consistent with this view. The results of our brain source analyses indicate coactivation in both parietal (at junction with associative occipital and temporal cortices) and prefrontal structures along with sensory structures during attend and ignore effects. These associate regions resemble the supramodal network that has been reported to guide attention to operate across attended modalities (Green, Doesburg, Ward, & McDonald, 2011; Green & McDonald, 2008; Driver, Eimer, Macaluso, & Van Velzen, 2003; Eimer, van Velzen, Forster, & Driver, 2003).
The timing of these attending and ignore effects varied. We observed modulations in earlier stages (P1/N1) and later stages (N2/Nd) of processing. The effects of attending were most pronounced in the later stages (N2/Nd), associated with stimulus identification (Hillyard & Anllo-Vento, 1998; Hansen & Hillyard, 1980, 1988; Näätänen, 1982). Similar latencies were reported previously by Talsma and Kok (2001) in a study of intermodal audiovisual attention (also de Ruiter, Kok, & van der Schoot, 1998). It is notable that Woods et al. (1992) observed a positivity around 200 msec following onset of nontarget stimuli in their detection paradigm. They interpreted this as a rejection of these stimuli from further processing. It is an open question whether this process is compatible with ignoring in the current study, which we observed at an earlier time stage. The timing of the N2/Nd effects is consistent with that of a frontal negativity with latency around 200 msec, observed by us (D'Ardenne et al., 2012; Lenartowicz, Escobedo-Quiroz, & Cohen, 2010) and others (Potts, 2004), and that has been linked with interactions between sensory and top–down regions, which also contributed to the brain source activities during this time in our results. This is a compatible interpretation for the process of modulating sensory activity during attending.
A novel finding in this study is that ignore effects occurred considerably earlier than the strongest attend effects, beginning around 56 msec for auditory stimuli (associated with sTP and prefrontal activity) and, in the source results, during the P1 for ignored (and also attended) visual stimuli. Although the effect for visual stimuli was considerably weaker than for auditory stimuli, in both modalities we saw a decrease in activity within the respective sensory modality but an increase in prefrontal cortex, consistent with an active suppression of sensory processing through the interaction of these cortices. Early ignore effects were also described by Luck et al. (1994), who reported a dissociation of spatial attention cueing effects on P1 (modulation observed for ignored locations, 80–130 msec) and N1 (modulation observed for attended locations, 130–180 msec). It is notable that here the attention modulation occurred much earlier here than in our own work, a finding also reported by Gazzaley et al. (2008) and Gazzaley, Cooney, McEvoy, Knight, & D'Esposito (2005) for house/face processing. The timing of attending and ignoring may well vary with task characteristics such as content domain and the nature of the decision process. For instance spatial attention shows earlier attention effects than feature discrimination (Hillyard & Anllo-Vento, 1998). Attention timing may also depend on whether attention was sustained, as in the current study, or whether it was triggered by a cue preceding each trial (as in the studies by Gazzaley et al., 2008; Gazzaley, Cooney, McEvoy, et al., 2005)—a feature that would be expected to impact attention strategy.
A prominent difference between our findings and prior reports is that, whereas prior intermodal attention effects were based on the difference between attended and ignored stimuli, we report attend and ignore effects relative to a passive control. This is an important difference that may help to reconcile our findings with prior reports. For instance, Alho, Woods, and colleagues have reported attention effects (attend-ignore) during the P1/N1/P2 sequence of potentials (Woods, Alho, & Algazi, 1993; Alho et al., 1992; Woods et al., 1992), whereas in the current study we did not find attend or ignore effects at these earlier stages in the scalp recordings but did find both attend and ignore effects in the brain source results. The absence of effects at the scalp suggests that these were weak effects, which is not surprising when one considers that our analysis split in two the attend–ignore difference reported in the other studies. The benefit of the passive condition however is in interpretation. This is exemplified in the early auditory positivity, which we could interpret as ignoring because it occurred without an attend effect. Without the passive condition, the effect of attending versus ignoring would still be significant, but it might be interpreted as a negative potential associated with attending (e.g., Karns & Knight, 2009).
Employing an intersensory continuous attention paradigm with a passive reference condition, this study provides evidence for the existence of attending targets and ignoring distractors as two processes that contribute to performance in the context of sustained attention control. We also present a novel finding that these processes are uncorrelated across individuals and vary across space, time, and task, evidence that they may be independent processes.
We would like to thank Jyoti Mishra for useful discussions regarding the paradigm. This research was supported by NIH grants R33DA026109 and R21MH096329 to M. S. C.
Reprint requests should be sent to Agatha Lenartowicz, University of California, Los Angeles, 760 Westwood Plaza Suite 17-369, Los Angeles, CA 90095, or via e-mail: firstname.lastname@example.org.