Biased competition theory proposes that representations in working memory drive visual attention to select similar inputs. However, behavioral tests of this hypothesis have led to mixed results. These inconsistent findings could be due to the inability of behavioral measures to reliably detect the early, automatic effects on attentional deployment that the memory representations exert. Alternatively, executive mechanisms may govern how working memory representations influence attention based on higher-level goals. In the present study, we tested these hypotheses using the N2pc component of participants' event-related potentials to directly measure the early deployments of covert attention. Participants searched for a target in an array that sometimes contained a memory-matching distractor. In Experiments 1 to 3, we manipulated the difficulty of the target discrimination and the proximity of distractors, but consistently observed that covert attention was deployed to the search targets and not the memory-matching distractors. In Experiment 4, we showed that when participants' goal involved attending to memory-matching items, these items elicited a large and early N2pc. Our findings demonstrate that working memory representations alone are not sufficient to guide early deployments of visual attention to matching inputs and that goal-dependent executive control mediates the interactions between working memory representations and visual attention.
In daily life, we regularly perform visual search for specific items that we need for our current task in cluttered scenes. It is of significant theoretical importance to determine how we guide our limited-capacity processing mechanisms to these task-relevant objects. Several theories of attention propose that our top–down control over search is dependent on holding a template representation of the object we wish to find in working memory (Bundesen, Habekost, & Kyllingsbaek, 2005; Duncan, 1996; Desimone & Duncan, 1995; Bundesen, 1990; Duncan & Humphreys, 1989). The theory of biased competition makes the clearest statement about how working memory templates influence visual attention (Desimone & Duncan, 1995). It proposes that the act of holding a representation in working memory leads to increased activity of the cells in the visual system that code for the features of the template. This increased activity creates a bias for neurons to represent template-matching items during the pervasive competition for access to the limited-capacity processing mechanisms of the brain (e.g., the receptive field of a neuron, representation in working memory, response selection). This elegant mechanism of top–down attentional control is dependent only on the template representation being in working memory. In the present study, we directly tested the hypothesis that maintaining a representation in visual working memory is sufficient for covert attention to be directed toward similar items in complex scenes.
Although the attentional template proposal from biased competition (Desimone & Duncan, 1995) is intriguing for its simplicity, previous studies of the influence of working memory representations on visual attention that measured overt behavior yielded mixed results. All of the studies examining the influence of working memory on attention have used a similar paradigm where participants are required to hold a representation in visual working memory while concurrently performing another visual task, typically a visual search. A number of studies have reported RT and eye movement findings that appear to support the hypothesis that attention is automatically captured by items that match a working memory representation early in the course of visual processing (Han & Kim, 2009; Olivers, 2009; Soto & Humphreys, 2008; Huang & Pashler, 2007; Soto & Humphreys, 2007; Olivers, Meijer, & Theeuwes, 2006; Soto & Humphreys, 2006; Soto, Humphreys, & Heinke, 2006a, 2006b; Soto, Heinke, Humphreys, & Blanco, 2005; Downing, 2000). A recent review by Soto, Hodsoll, Rotshtein, and Humphreys (2008) concluded from this evidence that, “guidance of selection from working memory occurs automatically, even when it is detrimental to performance” (p. 342). Thus, the successful maintenance of a working memory representation is sufficient to involuntarily direct attention to memory-matching items. However, other studies have provided convincing evidence that holding a representation in visual working memory is not sufficient for attention to be captured by memory-matching items and that the focus of attention might actively avoid such items when it is adaptive to do so (i.e., consistent with task demands; Woodman & Luck, 2007; Houtkamp & Roelfsema, 2006; Downing & Dodds, 2004).
A simple explanation for the variability of the findings based on behavioral evidence may be due to the nature of the evidence itself. It is possible that working memory representations consistently produce an attentional bias, but behavioral measures are not sensitive enough to detect early deployments of attention to memory-matching items. Overt behavior inherently measures the ultimate output of the entirety of cognitive processing. Although theories propose that attentional templates in working memory will influence mechanisms of perceptual attention, it is possible that the variability of processing time consumed during stages other than perception (e.g., response selection) results in behavioral measures that are not reliable in showing the early and automatic effects that working memory representations exert upon attentional selection. Consistent with this hypothesis, Han and Kim (2009) have proposed that attention is directed to memory-matching items early in perceptual processing of multi-element arrays, but that a relatively late onset of cognitive control can wash out this involuntary effect during search tasks with sufficiently long RTs. Similarly, measures of overt attentional selection (i.e., eye movements) may not be a consistent measure of the deployment of covert attention. Although shifts of covert attention appear to precede overt eye movements (Kowler, Andersen, Dosher, & Blaser, 1995), covert attentional orienting need not be followed by an overt shift of gaze (Posner, 1988). Therefore, it is critical to directly measure deployments of covert attention if we wish to know how working memory representations influence the focusing of perceptual attention. Fortunately, ERPs can provide such measures, and thus, provide critical information about the nature of the interactions between working memory representations and attention.
An alternative to the hypothesis that behavioral measures are simply imprecise is that additional executive control mechanisms mediate the influence of representations in working memory on mechanisms of perceptual attention. This hypothesis is supported by theoretical proposals (Bundesen et al., 2005) and by evidence from a number of empirical studies (Peters, Goebel, & Roelfsema, 2008; Woodman & Luck, 2007; Downing & Dodds, 2004). However, it is contrary to the proposal that the representation of an item in working memory is sufficient to create an involuntary bias to similar environmental inputs (Soto et al., 2008). One finding from Woodman and Luck (2007) suggests that a goal to attend to memory-matching items may be necessary for perceptual attention to be influenced by working memory. When participants knew that memory-matching items in the search array would never be the target of search, there was no significant influence of the presence of memory-matching items on RTs (Experiment 5A). However, when participants knew that the memory-matching items would be the search target on some trials, attention was then directed to the memory-matching items (Experiment 5B). These findings suggest that a goal of attending to memory-matching items might mediate the relationship between working memory and perceptual attention.
If executive control is necessary for a working memory representation to direct attention to a matching perceptual input, then we expect different patterns of findings across studies with different task demands. These different patterns should be present in studies that directly measure early shifts of covert attention. Specifically, ERP studies that measure the deployment of covert attention should show us that attention is deployed either toward, or away from, memory-matching items, depending upon the nature of the task at hand. Findings from two ERP studies seem to show preliminary support for this proposal. Kumar, Soto, and Humphreys (2009) reported that memory-matching distractors elicited a contralateral negativity (i.e., the N2pc component) during an efficiently performed visual search task. This supports the proposal that covert attention is deployed to distractors that match a working memory representation because the N2pc is believed to index the deployment of covert perceptual attention (Woodman & Luck, 2003b; Luck, Girelli, McDermott, & Ford, 1997; Luck & Hillyard, 1994b). In contrast, Peters, Goebel, and Roelfsema (2008) had observers detect targets in serially presented streams of objects. They found that distractors that matched a memory representation elicited a P3b that was indistinguishable from that elicited by the other distractors, but was different from the P3b elicited by targets. This latter study contradicts the former in suggesting that memory-matching items are not selected by perceptual attention mechanisms like task-relevant targets are. These opposing patterns of findings could be explained by the hypothesis that different cognitive control settings were operative in these different task contexts. However, many methodological differences exist between the two studies. One of our present goals was to determine whether working memory representations influence ERP measures of early covert attention in different ways depending on task demands while controlling other experimental variables.
In the present study, we used the N2pc component to directly measure where attention was covertly deployed during visual search while an object representation was maintained in working memory for a concurrent change-detection task. We used the N2pc component because it provides an index of selection by a covert perceptual attention mechanism that operates prior to awareness and working memory encoding (i.e., early in the course of visual processing, see Woodman & Luck, 2003a). Due to the lateralized nature of the N2pc, it can be used to measure where attention is focused throughout an extended process of visual search as attention is shifted between multiple objects, including nontargets (Woodman & Luck, 1999, 2003b). If working memory representations consistently and involuntarily bias perceptual attention to select matching items early in the course of processing complex scenes (i.e., automatic attentional capture by memory-matching items), then we should find that memory-matching items elicit an early N2pc regardless of whether attending to memory-matching items is consistent with the task demands. We tested this prediction in Experiments 1 to 3 in which the memory-matching item was never the search target. In contrast, the executive control hypothesis predicts that we should only find evidence of memory-matching items eliciting an N2pc when attending to these items is consistent with the goals of the task. We tested this prediction in Experiment 4, where we examined whether the influence of working memory representations on attention occurs when the participant's goal is to report a feature of the memory-matching input. Contrasting these two predictions will help clarify our understanding of how working memory influences attention, what properties an attentional control mechanism should possess, and which existing theories of attention can account for the findings.
In Experiment 1, we recorded ERPs from participants while they performed a visual search task during the retention interval of a change-detection task. Figure 1 shows an example of the Landolt-square stimuli and procedure for these two concurrently performed tasks. To test the potency of the guidance of attention by working memory representations, the memory item on each trial never shared features with the search target and participants were made aware of this during the instructions (e.g., Woodman & Luck, 2007; Downing & Dodds, 2004; Folk, Remington, & Johnston, 1992). Our design provided three trial types based on the items in the search array. On memory-match absent trials, no memory-matching distractor was in the search array. On memory-match opposite trials, a memory-matching distractor was in the opposite visual hemifield as the search target. On memory-match same trials, a memory-matching distractor was in the same visual hemifield as the search target.
During search tasks performed while an item is maintained in working memory for another task (e.g., change detection), the search target template and the other working memory representation may compete to influence attention (Downing & Dodds, 2004). To create a situation in which we were most likely to see an effect of the memory-matching item during visual search, we used a shape-defined target and a color-defined memory-matching distractor. This was done for two reasons. First, most of the previous studies of the N2pc component using visual search tasks have defined the task-relevant item based on color and the use of such stimuli results in a large and early N2pc to the color-defined target (e.g., a ∼1.0-μV effect beginning about 175 msec poststimulus; Luck, in press). Second, previous behavioral research has shown that it is easier to orient attention to items based on their color (using highly distinguishable colors) than even moderately complex shapes (e.g., Olivers & Humphreys, 2002; Wolfe, 1998a; Duncan & Humphreys, 1989). If the search target and memory representation are competing for control of attention, then our stimuli and experimental design should help the color maintained in working memory for the change-detection task win the competition for attentional guidance over the actual search target defined by the more diminutive feature of gap location on the Landolt-square stimuli. Thus, we stacked the deck in favor of observing an influence of the memory-matching items on covert perceptual attention.
Our paradigm allows for two complementary measures of early covert attention during search using the N2pc. First, we can measure an N2pc to the memory-matching item. Second, we can determine how the presence of the memory-matching item influences the N2pc to the search target by comparing memory-match absent trials with trials in which the memory match was in the same or opposite hemifield. We predicted that if memory-matching items automatically guide covert attention, and previous failures to see evidence of guidance were due to imprecise measures of early covert attention, then the N2pc would index attentional deployments to memory-matching distractors. The strong prediction based on the proposals that memory-matching items automatically capture attention (Han & Kim, 2009; Soto et al., 2005, 2008) is that when the memory-matching item is present in the search array, it will elicit the first N2pc activity following array onset. Figure 2 illustrates the pattern of N2pc activity predicted if memory-matching items capture attention (see also Kumar et al., 2009). The hemisphere contralateral to the memory-matching item should become more negative than the ipsilateral hemisphere at approximately 175 msec poststimulus regardless of the location of the search target in the array. When the memory-matching distractor is in the same hemifield as the search target, we expect to see the N2pcs to these items summate because attention will be immediately and consistently summoned to this hemifield, unlike trials when the memory match is absent and attention is first inadvertently shifted into the hemifield that does not contain the target on a subset of trials. A more moderate automatic capture prediction would be that the memory-matching item captures attention on a subset of trials. This would result in the N2pc to the search target being reduced in amplitude on memory-match opposite hemifield trials and increased on memory-match same hemifield trials relative to when the memory match is absent from the search array.
The competing hypothesis is that task goals determine whether a given memory representation will be used to guide attention (Woodman & Luck, 2007; Bundesen et al., 2005; Downing & Dodds, 2004; Logan & Gordon, 2001). If this hypothesis is correct, then memory-matching distractors should not elicit an N2pc during the search task in Experiment 1 because participants know that memory-matching items are not the target and attending to them is inconsistent with the goals of the search task. This would mean that the N2pc elicited by the search target would be unaffected by the presence or location of the memory-matching item.
Ten participants were paid $10 per hour of participation. All reported normal or corrected-to-normal vision, and provided informed consent before participation. The Vanderbilt University Institutional Review Board approved all procedures. Three participants were replaced due to excessive artifacts (i.e., artifacts on more than 30% of the trials with correct search and memory performance).
Stimuli and Apparatus
Participants were seated approximately 114 cm from a computer screen and responded using a gamepad (Logitech Precision, Switzerland). All stimuli were presented on a light gray background (33.1 cd/m2). A black fixation cross (0.13° line length, 0.03° line thickness, 0.84 cd/m2) was presented at the center of the screen throughout each trial. Memory and search stimuli were drawn from seven highly distinguishable colors (red, x = 0.616 y = 0.337, 13.9 cd/m2; green, x = 0.284 y = 0.959, 44.1 cd/m2; blue, x = 0.146 y = 0.720, 6.42 cd/m2; yellow, x = 0.407 y = 0.504, 54.3 cd/m2; magenta, x = 0.295 y = 0.153, 19.3 cd/m2; white, x = 0.290 y = 0.313, 57.2 cd/m2; sienna, x = 0.432 y = 0.405, 16.1 cd/m2). Memory stimuli were colored Landolt squares (0.33° × 0.33°, 0.07° line thickness, gap size 0.20°) centered 0.49° above the fixation cross. Six possible search targets were colored Landolt squares (0.33° × 0.33°, 0.07° line thickness, with a gap size of 0.20°) presented equidistant on an imaginary circle 2.61° from fixation. Two rings of black Landolt squares (0.84 cd/m2) were presented as distractors and scaled according to the cortical magnification factor with eccentricity (inner ring stimuli: 0.20° × 0.20°, 0.03° line width, 0.11° gap width, 1.57° from fixation; outer ring stimuli: 0.46° × 0.46°, 0.10 line width, 0.28° gap width, 3.65° from fixation; see Woodman & Luck, 2003b). Articulatory suppression stimuli were presented on an instructional screen at the beginning of each set of 30 trials (each digit or letter was approximately 0.40° × 0.40° and was drawn in black, 0.84 cd/m2).
The timing and stimulus sequence are illustrated in Figure 1. Each trial began with the presentation of the fixation point for 800–1200 msec (randomly jittered with a square distribution). Next, the memory item was displayed for 500 msec followed by a 500-msec period where only the fixation point was visible. Next, the search array was presented for 3000 msec. After the search array offset, a fixation point was presented for 500 msec followed by the memory test which was presented for 2000 msec. After the memory test, a blank screen was presented for 1400–1800 msec (randomly jittered with a square distribution) before the next trial began. It is important to note that these timing parameters have been shown to elicit consistent RT effects of 30 msec or larger due to the presence of memory-matching distractors (Dombrowe, Olivers, & Donk, 2010).
The search target was the one Landolt square in each array with a gap on the top or bottom and distractors had a gap on the right or the left. Half of the search arrays contained no memory-matching distractor. The other half of the trials contained a memory-matching distractor presented at a random location in relation to the target. This memory-matching distractor matched the memory item on color.1 This led to three search array types: memory-match absent, memory-match same hemifield, and memory-match opposite hemifield. The memory test item could be the same as the memory item (half of all trials) or the gap could move to the opposite side (quarter of all trials) or the color could change (quarter of all trials). All of these types of trials were randomly interleaved within each block of trials during the experiment.
Participants made a speeded response to report whether the target had a gap on the top or the bottom. They used the thumb on their left hand to press either the up or down directional button on the gamepad indicating the target gap location. Participants responded whether the memory test was the same or different as the memory item by pressing one of two buttons with the thumb on their right hand in an unspeeded manner.
Each participant was instructed to maintain fixation during each trial and blink during the intertrial interval. Participants were informed that a memory-matching item might appear in the search array, but that it would never be the search target. Participants performed eight practice trials before the experimental trials began and completed 960 experimental trials in four blocks with 15-sec breaks every 30 trials. During breaks, a countdown was shown on the screen along with an instructional sentence and new articulatory suppression stimuli. Articulatory suppression stimuli were to be repeated silently to prevent the muscle noise due to articulation from interfering with the ERP recordings but the prevent the recoding of the visual stimuli into verbal working memory (Baddeley, 1986). The experimenter asked participants to report the articulatory suppression stimuli after the practice block, and during a subset of the breaks to ensure compliance. After each block, participants were allowed to take a self-paced break.
ERP Recording and Analysis
We recorded the EEG from tin electrodes embedded in an elastic cap (Electro-Cap International, Eaton, OH). The caps contained a subset of the International 10–20 System sites (Fz, Cz, Pz, F3, F4, C3, C4, P3, P4, PO3, PO4, T3, T4, T5, T6, O1, and O2) in addition to two nonstandard sites (OL, placed halfway between O1 and T5; OR, placed halfway between O2 and T6). These electrodes were referenced on-line to the right mastoid, and re-referenced off-line to the average of the right and left mastoids (Nunez, 1981). We recorded the horizontal EOG from two electrodes placed approximately 1 cm from the external canthus of each eye. An electrode placed approximately 3 cm below the left eye, referenced to the right mastoid, measured vertical EOG. Signals were amplified using a SA Instrumentation amplifier with a gain of 20,000 and a bandpass of 0.01–100 Hz. The amplified signals were digitized by a PC-compatible computer at a rate of 250 Hz and averaged off-line. All trials with incorrect search or memory responses were excluded from the averages. We rejected individual trials with eye movements, blinks, muscle noise, or channel blocking prior to averaging the signal (an average of 14% of trials per subject). In addition, participants who had more than 30% of correct-performance trials rejected due to these artifacts were replaced.
The N2pc was measured across the posterior electrode sites (O1/2, OL/R, and T5/6) contralateral versus ipsilateral to the target location for each type of visual search array (memory-match absent, memory-match opposite, and memory-match same hemifield). Specifically, we measured mean amplitude from 200 to 400 msec post-search-array onset and entered these ERP data into an ANOVA with the within-subjects factors of array type (memory-match absent, memory-match opposite, vs. memory-match same), contralaterality (ipsilateral vs. contralateral relative to the target), hemisphere (left vs. right), and electrode (O1/2, OL/R, vs. T5/6). The mean target elicited N2pc voltages that are reported are calculated by subtracting the ipsilateral from the contralateral activity measured during the 200–400 msec window at electrodes OL/R (Woodman & Luck, 2003b). To further test for effects due to the presence of the memory-matching item, we performed a separate analysis of the waveforms averaged relative to the location of the memory-matching item in the search arrays, collapsed across target location. We entered the waveforms relative to the memory match into an ANOVA with the within-subjects factors of hemifield (left vs. right), contralaterality (ipsilateral vs. contralateral to the memory-matching item), and electrode (O1/2, OL/R, vs. T5/6). For these analyses, we used a time window of 200–300 msec based on previous studies of the N2pc (Luck, in press; Luck & Hillyard, 1994a, 1994b) and what would be expected if the first shift of attention were to the memory-matching item (Woodman & Luck, 1999, 2003b). In addition to this 200–300 msec window used to measure the predicted N2pc, we also used a measurement time window from 300 to 400 msec post-search-array onset due to the presence of a lateralized positivity relative to these memory-matching items during this period. The mean voltages reported for this positivity are measured across the three posterior pairs of electrodes by subtracting the ipsilateral from the contralateral waveforms with respect to the memory match. All p values from the ANOVAs were corrected as necessary with the Greenhouse–Geisser correction for violations of sphericity (Jennings & Wood, 1976).
Visual search accuracy was near ceiling whether the memory-matching item was present or not [99.5% correct and 99.6% correct, respectively; t(9) = 1.59; p > .14]. Participants were slower to respond when the memory-matching item was present (M = 948 msec) than when there was no memory-matching item in the search array [M = 910 msec; t(9) = 3.57, p < .01] on trials with correct search and memory performance. Memory accuracy was similar whether the memory item was present (M = 90.8% correct) or absent from the search array [M = 91.7% correct; t(9) = 1.19, p > .26]. It is interesting to note that this pattern of RTs is identical to that previously interpreted as evidence for the capture of attention by memory-matching items (i.e., the early, involuntary orienting of attention to memory-matching inputs).
Figure 3A shows the N2pc elicited by the search target. Across the three types of trials (i.e., memory-match absent, memory-match opposite, memory-match same hemifield), the N2pc related to focusing covert attention on the search target was essentially unchanged (contralateral minus ipsilateral to the target using electrodes OL/R from 200 to 400 msec post-search-array onset; memory-match absent, M = −0.36 μV; memory-match opposite, M = −0.38 μV; memory-match same, M = −0.33 μV). Our statistical analyses supported these observations. We found a significant N2pc to the search target evidenced by a main effect of contralaterality from 200 to 400 msec poststimulus [F(1, 9) = 11.9, MSE = 0.84, p < .01] and an interaction of contralaterality and electrode [F(2, 18) = 4.3, MSE = 0.08, p < .05] due to the N2pc being larger at T5/6 and OL/R than at O1/2 (M = −0.43 μV, M = −0.36 μV, and M = −0.22 μV, respectively). An automatic influence of memory match on the deployment of covert attention would be expressed by an interaction of Array type × Contralaterality, however, this interaction was not significant [F(2, 18) = 0.19, MSE = 0.45, p = .83]. No other effects or interactions were significant.
In Figure 3B, we show that no early negativity was observable contralateral to the memory-matching item. Indeed, the waveforms actually show a trend in the opposite direction later in the time window. Specifically, the presence of the memory-matching distractor led to a positivity, most evident approximately 375 post-search-array onset, contralateral to the visual hemifield containing this critical distractor. Consistent with this observation, the effect of contralaterality relative to the memory-matching item from 200 to 300 msec poststimulus was not significant [F(1, 9) = 0.39, MSE = 0.25, p = .55] nor were there higher-level interactions involving this factor. Moreover, the analysis of the positivity contralateral to the memory match using the 300–400 msec window yielded a significant effect of contralaterality [F(1, 9) = 5.29, MSE = 0.33, p < .05]. We also found a significant interaction of Contralaterality × Hemifield [F(1, 9) = 5.12, MSE = .43, p < .05] due to the N2pc being larger when the memory-matching item was in the left hemifield. No other effects or interactions were significant.
We found that the search targets elicited a robust N2pc that was unmodulated by the presence or location of the memory-matching distractor. The only lateralized effect elicited by the memory-matching distractors was a relatively late contralateral positivity. This observation is consistent with two possible mechanistic explanations. First, it is possible that the focus of attention actively avoids the memory-matching items, increasing the amplitude of the target-elicited N2pc into the opposite hemifield. Alternatively, this pattern of lateralized activity might be due to the active suppression of the memory-matching items in the search arrays. Recent work suggests that active distractor suppression elicits a distactor positivity, or Pd (Hickey, Di Lollo, & McDonald, 2009). These findings and both of the mechanistic explanations are contrary to the predictions of the hypothesis that memory-matching items attract attention to themselves early in the process of visual search (e.g., compare Figures 2 and 3). If this hypothesis had been supported, then we should have first observed an N2pc elicited by the memory-matching item and only later an N2pc to the target. It is striking that the ERP results show that attentional capture by memory-matching items did not occur, although the simultaneously measured RTs were slower on trials with memory-matching items than trials without. This finding confirms an assumption that we made in motivating the present study. Specifically, behavioral RTs alone do not provide a measure precise enough to infer that an RT effect is due to the earliest deployments of attention and not to subsequent processes.
As described above, our search stimuli were designed so that the color-defined memory-matching item would be easy to orient attention toward, while the search target (defined by the location of the gap on the Landolt squares) would require scrutiny within the focus of attention (Woodman & Luck, 1999, 2003b; Wolfe, 1998b). Our observation of a smaller amplitude and longer latency N2pc to the search target compared to previous N2pc studies (Luck, in press; Luck & Hillyard, 1990, 1994b) is consistent with this logic and supports the idea that the point in time when attention was focused on the search target varied from trial-to-trial. However, this shift of attention to the target was essentially uninfluenced by the presence of the memory-matching items.
In Experiment 1, we found that the search targets, but not the memory-matching items, elicited an N2pc. If the search target and memory item were competing to control attention, then it is possible that our N2pc findings may have been due to the search target being too easy to discriminate in Experiment 1. In Experiment 2, we tested this hypothesis by reducing the discriminability of the search target. This entailed making the gap size of the Landolt squares smaller than those used in Experiment 1, a manipulation known to decrease the efficiency of visual search (e.g., Woodman, Vogel, & Luck, 2001). If we failed to find that attention was captured by memory-matching distractors in Experiment 1 due to attentional guidance of the search target overpowering the capture of attention by the memory-matching item, then in Experiment 2 we should see a modulation of the N2pc elicited by the difficult to discriminate targets based on the location of the conspicuous memory-matching items in the search array.
All methods were the same as Experiment 1, except as follows.
A new group of 10 participants was recruited and two participants were replaced due to artifacts on more than 30% of trials with correct search and memory performance.
As shown in Figure 4A, the gap size of the Landolt-square stimuli was reduced to one third of the size of the gaps in Experiment 1. Stimuli at the six potential target locations were identical to Experiment 1 except that the gaps were 0.07° wide. The gaps in the two rings of black Landolt-c distractors were similarly scaled (inner ring stimuli: 0.20° × 0.20°, 0.03° line width, 0.03° gap width; outer ring stimuli: 0.46° × 0.46°, 0.10° line width, 0.10° gap width).
Across participants, an average of 15% of trials with correct search and memory responses were removed due to ocular or other EEG artifacts.
Mean accuracy of visual search responses was 96.2% correct when the memory-matching item was present and 96.6% correct when absent [t(9) = 2.46, p < .05]. On trials with correct search and memory performance, participants were slower to respond when the memory-matching distractor was in the array (M = 1124 msec) than when the memory-matching item was absent [M = 1093 msec; t(9) = 2.70, p < .05]. These search RTs were slower than those from Experiment 1 [t(18) = 2.22, p < .05], demonstrating the effectiveness of our difficulty manipulation between Experiments 1 and 2. Memory task performance did not significantly differ based on whether a memory-matching item was present (M = 82.9%) versus absent from the array [M = 84.9%; t(9) = 2.06; p = .07].
Figure 4B shows the ERP waveforms relative to the search targets in Experiment 2. Just as in Experiment 1, we reliably measured an N2pc contralateral to the hemifield containing the search target that was first apparent just after 200 msec poststimulus and continued until approximately 400 msec poststimulus onset. Most importantly, the presence of the memory-matching item in the array had little influence on this N2pc to the search target (memory-match absent, M = −0.31 μV; memory-match same, M = −0.25 μV; memory-match opposite, M = −0.71 μV). Note that the N2pc elicited by the targets tended to be larger when the target was opposite the memory match compared to when it was in the same hemifield. This is the reverse of what we expect from memory-matching items that are capturing attention. The statistical analyses supported these observations. We found a significant N2pc to the search target evidenced by a significant effect of contralaterality [F(1, 9) = 14.7, MSE = 0.83, p < .01] due to more negative waveforms contralateral to the search target. We also found an interaction of Contralaterality × Electrode [F(2, 18) = 6.9, MSE = 0.10, p < .01] due to the N2pc being larger at T5/6 and OL/R than at O1/2 (M = −0.49 μV, M = −0.43 μV, and M = −0.19 μV, respectively). Contralaterality did not interact with array type [F(2, 18) = 2.6, MSE = 0.62, p = .10] because the N2pc elicited by the search target was not significantly influence by the presence or location of the memory match. We did find a higher order interaction of Array type × Hemifield × Electrode [F(4, 36) = 2.9, MSE = 0.04, p = .05]. No other effects or interactions were significant.
Figure 4C shows the lateralized effects relative to the location of the memory-matching item. These waveforms show that the presence of the memory-matching item in a hemifield did not elicit an early N2pc, but instead had the opposite effect later in the processing of the search array (i.e., M = 0.61 μV contralateral minus ipsilateral relative to the memory match from 300 to 400 msec poststimulus). As in Experiment 1, the absence of an early N2pc elicited by the memory-matching item resulted in the effect of contralaterality not being significant in the ANOVA using the ERP data from 200 to 300 msec relative to the memory-matching distractor [F(1, 9) = 0.10, MSE = 0.24, p = .75]. We also found an interaction of hemifield and electrode [F(2, 18) = 4.19, MSE = 2.22, p < .05]. However, the positivity in the 300–400 msec time window created a significant effect of contralaterality [F(1, 9) = 8.57, MSE = 1.01, p < .05]. We also found an interaction of Contralaterality × Electrode [F(2, 18) = 4.61, MSE = 0.06, p < .05] due to the positivity being larger at T5/6 and OL/R than at O1/2 (M = 0.66 μV, M = 0.61 μV, and M = 0.34 μV, respectively).
In Experiment 2, we replicated the finding that the memory-matching items did not capture covert attention during visual search as indexed by the N2pc. Contrary to such predictions, we again found that the memory-matching items elicited a contralateral positivity from 300 to 400 msec poststimulus. This again suggests that the focus of attention actively avoids the hemifield containing the memory-matching item or that this item is actively suppressed, eliciting a positivity. RTs were longer in Experiment 2 than in Experiment 1, indicating that our manipulation of decreasing the size of the gap did make the search task more difficult. However, this increase in the difficulty of target discrimination did not reveal evidence for early and automatic shifts of attention to the memory-matching distractors.
In Experiments 1 and 2, we observed a significant N2pc elicited by the search targets but did not find that the working memory representations resulted in the deployment of covert attention to memory-matching items. Previous research has shown that the amplitude of the N2pc increases when distractors appear near attended items (Luck et al., 1997) and adding additional items to a search array will decrease the distance between an attended item and the nearest distractor by increasing spatial crowding (Cohen & Ivry, 1991). It is possible that in Experiments 1 and 2, attention was directed to the memory-matching item, but no significant N2pc was generated during the shifts of attention to the memory-matching item because we presented search arrays with too few distractors. That is, the absence of nearby distractors in Experiments 1 and 2 may have minimized the amplitude of any N2pc that was present but not of sufficient amplitude to be detectable (e.g., see Woodman & Luck, 2003b). Thus, in Experiment 3, we increased the number of distractors by 450% to provide an aggressive test of the explanation that the memory-matching items in Experiments 1 and 2 were eliciting an undetectably small N2pc.
All methods were the same as Experiment 1, except as follows.
A new group of 10 participants was recruited from the same pool used in the previous experiments. Two participants were replaced due to excessive artifacts (i.e., more than 30% of trials with correct search and memory performance).
Figure 5A shows that we increased the number of black distractors in the search arrays of Experiment 3 to 54, compared to the 12 black distractors in Experiments 1 and 2 (compare Figures 1, 4A, and 5A). Specifically, we increased the number of distractors in the inner and outer ring from 6 to 12 and we added black distractors between the potential search targets in the middle ring. We also added two more rings of distractors, one between the inner and middle ring (0.26° × 0.26°, 0.05° line width, 0.16° gap width, 2.09° from fixation), and one between the middle and outer ring (0.39° × 0.39°, 0.08 line width, 0.23° gap width, 3.13° from fixation).
An average of 14% of trials with correct search and memory responses were excluded from the analyses due to eye movements, blinks, and other EEG artifacts.
Participants were slower to respond when the memory-matching distractor was in the array (M = 993 msec) than when it was absent [M = 958 msec; t(9) = −3.71, p < .01], although search accuracy did not differ between these trial types [98.9% vs. 99.1% correct, respectively; t(9) = 1.57, p > .15]. These RTs were not significantly slower than those of Experiment 1 [t(18) = 2.28, p < .05] and were faster than those in Experiment 2 using the difficult to discriminate target shapes [t(18) = 2.28, p < .05]. Memory task accuracy was 89.1% correct when a memory-matching item was in the array and 89.6% correct when absent [t(9) = 0.67, p = .51].
Figure 5B shows the ERP waveforms relative to the search targets in Experiment 3. Replicating a now familiar pattern, we observed that the search targets elicited an N2pc, but the memory-matching distractor did not modulate the target-elicited N2pc in the manner expected if such memory matches were capturing covert attention. Specifically, the amplitude of the target-elicited N2pc on memory-match absent (M = −0.23 μV), memory-match same hemifield (M = −0.17 μV), and memory-match opposite hemifield (M = −0.35 μV) trials were similar (using the 200–400 msec measurement window). Any potential modulation of the lateralized activity was again in the direction opposite to what would be expected if attention were being deployed to the memory-matching items (i.e., there was a late, increased positivity contralateral to the memory match). Consistent with these observations, the ANOVA of the ERP data related to the search target yielded a significant effect of contralaterality [F(1, 9) = 23.0, MSE = 0.21, p < .01], but not an interaction of Contralaterality × Array type [F(2, 18) = 0.91, MSE = 0.22, p = .42]. We also found an interaction of Array type × Electrode [F(4, 36) = 3.5, MSE = 0.06, p < .05]. No other effects or interactions were significant.
Figure 5C shows that the memory-matching item did not elicit an early N2pc. The analysis of the waveforms relative to the memory match from 200 to 300 msec did not result in a significant effect of contralaterality [F(1, 9) = 0.54, MSE = 0.15, p = .48] or interactions involving this factor. The contralateral positivity relative to the memory-matching item measured from 300 to 400 msec led to a significant effect of contralaterality in the ANOVA [F(1, 9) = 26.5, MSE = 0.10, p < .001]. The interaction of Contralaterality × Electrode [F(2, 18) = 5.46, MSE = 0.02, p < .05] was because this positivity was largest at OL/R (M = 0.36 μV) and T5/6 (M = 0.36 μV) followed by O1/2 (M = 0.18 μV).
In Experiment 3, we replicated the pattern of results from Experiments 1 and 2 using denser search arrays. These findings rule out one possible explanation for the findings of Experiments 1 and 2, which states that we used search arrays with too low a density to observe a memory-matching N2pc. The search arrays in Experiment 3 had 450% more distractors than Experiments 1 and 2 but the same pattern of results was found. Thus far, we have seen no evidence that covert attention, as measured by the N2pc, is directed to memory-matching items. On the contrary, the findings are consistent with observers avoiding shifting attention to these known nontarget items, perhaps due to active suppression of these items. The hypothesized early and automatic capture of attention by memory-matching items predicts that the N2pc should have indexed a shift of attention to the memory-matching item prior to orienting attention to the visual search target. However, we have not observed this pattern of results regardless of the difficulty of perceiving the target-defining feature or the number of distractors present in the visual search arrays.
Although the findings from Experiments 1 to 3 are inconsistent with the hypothesis that visual working memory representations cause matching perceptual inputs to capture attention, they could be accounted for by the idea that covert attention is deployed to items when doing so is consistent with the goals of the tasks at hand (Peters et al., 2008; Woodman & Luck, 2007; Downing & Dodds, 2004). In Experiments 1 to 3, it was consistent with the goal of the search task (i.e., find the search target as fast as possible) to avoid shifting attention to the memory-matching items that were known to not be the target and the ERPs we observed were consistent with such a goal. However, a simpler alternative explanation is that observers simply could not shift attention to memory-matching items early in the course of visual search given the stimuli we used. We next addressed these competing explanations.
In Experiments 1 to 3, if we would have observed early deployments of covert attention to the memory-matching items (i.e., an N2pc), then they would have been involuntary by definition, because attending to the memory-matching item was contrary to the goal of the search task (Jonides, 1981). The absence of an N2pc to memory-matching items in Experiments 1 to 3 is inconsistent with the proposal that “working memory exerts an involuntary influence on visual attention” (Soto et al., 2008; p. 344). In contrast, other researchers have proposed that it is necessary for the participant to have a goal of attend to the memory-matching item for working memory representations to guide attention (Peters et al., 2008; Woodman & Luck, 2007; Downing & Dodds, 2004). Consistent with this proposal from the empirical literature, the Neural Theory of Visual Attention (Bundesen et al., 2005) posits that templates are held in working memory, but an additional process must trigger a biasing signal before a working memory representation can guide attention. One could interpret this additional process as an executive function that determines which of multiple available templates in short- or long-term memory should be used to direct attention. To assess whether early deployments of covert attention to memory-matching items are determined by voluntary control, we altered the participants' search task while using the same stimuli and memory task applied in Experiment 3.
In Experiment 4, participants were required to report the gap location of the memory-matching item, thus, making the memory-matching item task relevant. If the influence of working memory on early deployments of covert attention is goal dependent, then we should find that early attentional deployments are directed to the memory-matching items in Experiment 4. This would be evidenced by an early N2pc to the memory-matching item. Alternatively, our previous failures to observe attentional deployments to memory-matching items could have been due to a specific aspect of our experimental design. Recently, Olivers (2009) has suggested that certain stimuli may not be capable of generating a guidance effect. Specifically, this article proposed that Landolt-squares may produce a stimulus energy that is too low for memory-matching items to capture attention. In other words, weak bottom–up signals from certain types of search items do not strongly engage the memory representation, and thus, it is difficult for attention to select such items. This explanation would also hold for stimuli other than Landolt-squares that have weak bottom–up signals that match memory. If our failures to find early deployments of attention to memory-matching items can be explained by the low stimulus energy of our particular stimuli, then we should once again find no early N2pc to the memory-matching item. Attention would only be directed to the task-relevant memory-matching item later in the search process. Thus, the design of Experiment 4 serves to distinguish between these competing explanations of our previous findings in this study.
All methods were the same as those of Experiment 3, except as follows.
A new group of 10 volunteers was recruited from the same pool. One participant was replaced due to ocular and EEG artifacts on more than 30% of trials with correct search and memory performance.
Stimuli and Apparatus
Figure 6A shows an example of the search arrays that were identical to those in Experiment 3, except that the memory-matching item was task relevant (i.e., see the dashed circle indicating the target). Correct search responses were defined by the location of the gap on the item that matched the color of the memory item. Responses were made using the directional buttons on the gamepad with the left arrow indicating a gap to the left and the right arrow indicating a gap to the right. Both buttons were pressed with the left-hand thumb of the participants.
Participants were instructed to attend to the item that matched the color of the item in memory and report the direction of the gap location on this search item. They were instructed that an object with a gap up or down would appear in the search array, but that this item would never be the search target. As in Experiments 1 to 3, participants had to remember both the color and gap direction of the memory item and to report if there was a change or there was no change when viewing the memory test with a right-hand button press at the end of each trial.
Across participants, an average of 11% of trials with correct search and memory responses were excluded from the analyses due to eye movements and other EEG artifacts.
Mean change-detection accuracy was 90.8% correct. The search responses for discriminating the gap location of the memory-matching item resulted in a mean RT of 652 msec and an accuracy of 98.9% correct.
The ERP waveforms recorded during the search task are shown in Figure 6B and C. The waveforms show that the N2pc elicited by the memory-matching items from 200 to 300 msec was large in amplitude (i.e., M = −1.41 μV at OL/R) and had an early onset (i.e., approximately 175 msec poststimulus). The polarity of this N2pc flipped when the hemispheres were defined relative to hemifield containing the search element with a gap up or down (see Figure 6B). This is the pattern that is expected if the element with the gap up or down had little or no influence on the early N2pc elicited by the memory-matching item (M = −0.08 μV; see also below).
To confirm our observations, we subjected the ERP data relative to the memory-matching item from 200 to 300 msec after search-array presentation to an ANOVA with the within-subjects factors of hemifield (left vs. right), contralaterality (ipsilateral or contralateral to memory-match), and electrode (O1/2, OL/R, vs. T5/6). In contrast to the findings from Experiments 1 to 3, when participants were instructed to report the gap location of the memory-matching color, the contralaterality factor relative to the memory-matching item was significant [F(1, 9) = 12.7, MSE = 2.88, p < .01], as was the main effect of electrode [F(2, 18) = 27.57, MSE = 4.28, p < .0001]. We also found significant interactions of Contralaterality × Electrode [F(2, 18) = 4.3, MSE = 0.20, p < .05] due to the N2pc being larger at OL/R and T5/6 than O1/2 (M = −1.41 μV, M = −1.09 μV, and M = −0.89 μV, respectively), and Hemifield × Contralaterality × Electrode [F(2, 18) = 3.7, MSE = 0.05, p < .05].
When the goal of the search task required that the memory-matching item be processed, we found clear evidence that covert attention was deployed to these items early in the course of analyzing the search arrays. This indicates that the specific stimuli we used throughout this study did not prevent attention from being directed to the memory-matching items early in the attention-demanding search process. This allows us to rule out a “low stimulus energy” account of the findings from Experiments 1 to 3 (Olivers, 2009). These findings support the competing hypothesis that the influence of working memory on attention is goal dependent (e.g., Woodman & Luck, 2007) as opposed to involuntary (Soto et al., 2008).
The findings from Experiments 1 to 3 consistently showed that the targets of the visual search tasks elicited an N2pc, a sensitive measure of the deployment of covert perceptual attention (Luck, in press; Woodman & Luck, 2003a, 2003b). However, we found no evidence of deployments of covert attention to memory-matching items, either early or late in the process of analyzing the search arrays. Instead, we found a consistent pattern in the waveforms suggesting that the focus of attention either actively avoided the memory-matching items or these items were actively suppressed from 300 to 400 msec after search onset. That is, the memory-matching items may have elicited a distractor positivity (i.e., Pd) due to their active suppression, or increased the amplitude of the N2pc due to attention mechanisms avoiding the memory-matching distractor. However, both of these interpretations are in direct opposition to the proposal that memory-matching items capture attention. When the goal of the search task involved attending to memory-matching items in Experiment 4, we found an early and large amplitude N2pc elicited by these objects. This latter finding supports proposals that attention is not deployed to memory-matching inputs unless it is part of a task goal (Woodman & Luck, 2007; Bundesen et al., 2005). These findings demonstrate that memory-matching items in our visual field do not consistently and involuntarily capture covert attention. Instead, our findings suggest that early covert attention is deployed to items that are relevant for the goal of the task at hand.
The crux of the debate regarding how working memory representations interact with attention is that working memory representations cause an early and involuntary orienting of attention to memory-matching items (Soto et al., 2005, 2008). How can we be certain that memory-matching items do not capture attention before the N2pc is elicited? This explanation is unlikely for the following reasons. First, previous studies showed that the N2pc measures the first shifts of attention during demanding search tasks using the same type of stimuli as that in the present study (Woodman & Luck, 1999, 2003b). Second, masking experiments have shown that the N2pc can measure the deployment of perceptual attention to targets that are presented so briefly that subjects are not aware that they were shown (Woodman & Luck, 2003a). Thus, it is extremely unlikely that shifts of attention were occurring that we were unable to measure.
In this study, we observed a striking contrast between the pattern of behavioral findings and the ERP effects. The RTs were significantly longer when a memory-matching item was present in the search display than when it was absent in Experiments 1, 2, and 3. These are the type of findings typically interpreted as evidence that memory-matching items automatically capture attention, slowing the deployment of attention to the actual search target. However, the N2pc measured in Experiments 1–3 showed that covert attention was not directed to the memory-matching distractors early (or at all) during the attention-demanding search process.
How do we reconcile these seemingly contradictory findings from the behavioral and ERP measures? Our ERP findings support the explanation that participants may have avoided attending to the memory-matching items in Experiments 1–3, perhaps via active suppression. If so, this may have slowed search RTs when the search target happened to be near the memory-matching item relative to when the memory match was at a distant location or absent from the array. To assess the likelihood of this explanation of the RT effects, we combined the search RTs from Experiments 1 to 3 based on the distance between the memory-matching item and the search target. RTs were slowest when the memory-matching distractor appeared next to the search target (M = 1034 msec), faster when the memory match and target were separated by one additional colored distractor (M = 1016 msec), and faster still when the memory-matching item was opposite the search target (M = 1008 msec). We entered these RTs into an ANOVA with the between-subjects factor experiment (Experiments 1, 2, and 3) and the within-subjects factor distance (1, 2, and 3 spatial locations removed from the target). We found a significant effect of distance [F(2, 54) = 4.913, MSE = 0.001, p < .05], a marginal effect of experiment [F(2, 27) = 3.34, MSE = 0.07, p =.05], and no interaction of these factors. This slowing of RT based on the proximity of the memory-matching item to the target, taken together with the ERPs indicating that memory-matching items were actively avoided or suppressed, supports the conclusion that the elevated RTs when the memory-matching distractors were present was due to the suppression of the memory-matching items spreading to the search target. Our converging evidence from the RTs and ERPs emphasizes the caution necessary in relying upon a behavioral effect to infer the operation of attentional mechanisms early in the course of processing.
The hypothesis drawn from the elegant proposal of the theory of biased competition (Desimone & Duncan, 1995), that the maintenance of an object representation in visual working memory is sufficient to bias perceptual attention to select similar objects, appears to be incomplete. The present findings show that attention is only driven to select items matching those in working memory if it is task relevant to do so. We propose that an additional process should be added to the theory of biased competition in which goal-dependent executive control is an intermediate step between working memory representations and the biasing of visual attention. The result of such an addition will draw biased competition closer to the Neural Theory of Visual Attention (Bundesen et al., 2005), a theoretical cousin that has the flexibility to account for the findings of this study.
The present results and the previous studies of the influence of working memory on attention (Han & Kim, 2009; Olivers, 2009; Soto & Humphreys, 2008; Huang & Pashler, 2007; Soto & Humphreys, 2007; Olivers, Meijer, & Theeuwes, 2006; Soto & Humphreys, 2006; Soto et al., 2006a, 2006b; Soto et al., 2005; Downing, 2000) cannot be explained by a simple mechanism creating an involuntary attentional bias based on just any working memory representation. An account that emphasizes a goal-dependent interaction between working memory and attention is necessary to explain the sum of the findings. If goals influence whether working memory representations guide attention toward or away from memory-matching inputs, then differences across studies may be seen as evidence of the flexibility of the control mechanism. We propose that the influence of working memory on attention may be described as a form of conditional, goal-dependent automaticity (Bargh, 1989), in which working memory representations guide attention only when one's goals are consistent with attending to memory-matching items.
We thank Steve Luck for valuable comments regarding this study. This research was supported by the National Eye Institute of the National Institutes of Health (RO1-EY019882) and the National Science Foundation (BCS 09-57072).
Reprint requests should be sent to Geoffrey F. Woodman, Department of Psychology, Vanderbilt University, PMB 407817, 2301 Vanderbilt Place, Nashville, TN 37240-7817, or via e-mail: firstname.lastname@example.org.
Analyses were performed to determine if the presence of distractors with an exact memory match (i.e., color and shape) led to effects that differed from those of color-only memory matches. In each experiment, no difference was found between exact and color-only memory matches in the N2pc analyses (all ps > .15). Thus, we collapsed across these trials in the analyses reported.