The encoding of information from one event into working memory can delay high-level, central decision-making processes for subsequent events [e.g., Jolicoeur, P., & Dell'Acqua, R. The demonstration of short-term consolidation. Cognitive Psychology, 36, 138–202, 1998, doi:10.1006/cogp.1998.0684]. Working memory, however, is also believed to interfere with the deployment of top–down attention [de Fockert, J. W., Rees, G., Frith, C. D., & Lavie, N. The role of working memory in visual selective attention. Science, 291, 1803–1806, 2001, doi:10.1126/science.1056496]. It is, therefore, possible that, in addition to delaying central processes, the engagement of working memory encoding (WME) also postpones perceptual processing as well. Here, we tested this hypothesis with time-resolved fMRI by assessing whether WME serially postpones the action of top–down attention on low-level sensory signals. In three experiments, participants viewed a skeletal rapid serial visual presentation sequence that contained two target items (T1 and T2) separated by either a short (550 msec) or long (1450 msec) SOA. During single-target runs, participants attended and responded only to T1, whereas in dual-target runs, participants attended and responded to both targets. To determine whether T1 processing delayed top–down attentional enhancement of T2, we examined T2 BOLD response in visual cortex by subtracting the single-task waveforms from the dual-task waveforms for each SOA. When the WME demands of T1 were high (Experiments 1 and 3), T2 BOLD response was delayed at the short SOA relative to the long SOA. This was not the case when T1 encoding demands were low (Experiment 2). We conclude that encoding of a stimulus into working memory delays the deployment of attention to subsequent target representations in visual cortex.
Despite the immense processing power of the human brain, it is severely capacity limited. This is dramatically illustrated when we attempt to perform even two simple sensorimotor tasks simultaneously; under these conditions, there is typically significant dual-task interference (Pashler, 1994). Such interference is generally thought to reflect an attentional bottleneck at a central/amodal decision-making stage that precludes the efficient processing of both tasks. Although a response is selected for an initial task-relevant stimulus, the response to a second stimulus is delayed in an all-or-none (e.g., Pashler, 1994) or graded (e.g., Tombu & Jolicoeur, 2003; Navon & Miller, 2002) manner (but see Meyer & Kieras, 1997). This attentional limitation has been shown to preclude not only concurrent response selection processes but also other relatively late stage operations such as response execution (Ulrich et al., 2006), mental rotation (Ruthruff, Miller, & Lachmann, 1995), memory retrieval (Carrier & Pashler, 1995), and working memory encoding (WME) (Brisson & Jolicoeur, 2007; Robitaille, Jolicoeur, Dell'Acqua, & Sessa, 2007; Jolicoeur & Dell'Acqua, 1998; Osman & Moore, 1993). Both electrophysiological and neuroimaging studies support a late stage origin for this bottleneck. For example, encoding an initial target into working memory can delay the encoding of a subsequent target (Dell'Acqua, Pascali, Jolicoeur, & Sessa, 2003; Giesbrecht, Bischof, & Kingstone, 2003; Giesbrecht & Di Lollo, 1998) and delay an electrophysiological measure of working memory updating of the second target that occurs well over 300 msec after stimulus presentation (P300; Vogel & Luck, 2002). In addition, fMRI studies have linked this central bottleneck of information processing to the prefrontal and parietal cortex, thereby providing neurobiological support for its occurrence at a relatively late, central stage of information processing (Dux et al., 2009; Sigman & Dehaene, 2008; Dux, Ivanoff, Asplund, & Marois, 2006).
Although the central bottleneck is most often associated with late stages of information processing, there is also evidence that it can impact early sensory analysis. It is now well established that the performance of a working memory task can disrupt attention-related processing in visual cortex (de Fockert, Rees, Frith, & Lavie, 2001). In particular, the engagement of WME by an initial target item has been shown to suppress the N2pc—an occipital ERP component associated with the allocation of visuospatial attention (see Woodman & Luck, 1999)—to subsequently presented targets (Sergent, Baillet, & Dehaene, 2005). Furthermore, the degree of N2pc suppression is dependent on the level of WME demand (Dell'Acqua, Sessa, Jolicoeur, & Robitaille, 2006). Thus, it appears that, although central processing is devoted to the encoding of a task-relevant item, processing of a subsequent item can be compromised either at early or late stages of information processing. Such results are consistent with behavioral evidence that the encoding (Ouimet & Jolicœur, 2007) or manipulation (Fougnie & Marois, 2007) of material in working memory may impair conscious access to subsequent material (e.g., Chun & Potter, 1995; see Dux & Marois, 2009, for a review).
To date, support for the notion that WME impairs attentional processing at sensory levels is restricted to modulations in the amplitude of BOLD and ERP signals. Serial postponement of processes, however, is the hallmark of the central bottleneck; in other words, if WME and attentional modulation of sensory signals both rely on the same central resources, attentional modulation of a second stimulus signal should be delayed until WME encoding of an initial stimulus is completed. To our knowledge, neither electrophysiological nor BOLD data have provided evidence of WME-related delays in the attentional modulations of sensory signals. This is a crucial prediction to examine if we are to fully understand the action of top–down processing on sensory analysis in the brain.
The goal of the present study was to use the relatively high spatial and temporal resolution of time-resolved fMRI (see Dux et al., 2009; Dux, Ivanoff, et al., 2006; Ecker, Brammer, David, & Williams, 2006; Formisano et al., 2002) to assess whether WME postpones the action of top–down attention on sensory processing in extrastriate cortex. Specifically, we examined whether the engagement of WME with an initial task (Target 1, T1) delayed the latency of the extrastriate signal evoked by attentional processing of a second task-relevant stimulus (Target 2, T2). We manipulated the SOA between T1 and T2 to vary the extent to which T1 WME would temporally overlap with the presentation of the second target and, therefore, interfere with T2's attentional processing. We predicted that top–down attentional enhancement of T2 activity in visual cortex should be delayed at the short SOA relative to the long SOA, because at the short SOA T2 would be presented during the period in which T1 WME would still engage the central processing resources, thereby preventing these resources from being concurrently used to enhance T2 sensory representations. Under such conditions, top–down enhancement of an unmasked T2 item should be delayed until these central processes are freed from the WME encoding requirements of T1. To bring these effects within the temporal resolution of fMRI, we used a T1 task that placed heavy demands on WME (see Ouimet & Jolicœur, 2007) by requiring participants to encode the identity and location of four distinct keyboard symbols (Figure 1). Ouimet and Jolicœur (2007) have shown that such a task requires at least 600 msec to complete. Task 2, by contrast, consisted of a simple on-line letter discrimination judgment (“S” or “H”) and was temporally separated from the onset of T1 by either 550 or 1450 msec. To ensure that we had the effective resolution to capture the effect of SOA on attentional modulation of T2 processing in visual cortex, we employed a fast sampling repetition time (TR) of 300 msec, which restricted coverage to an area of visual cortex that has been previously implicated by an electrophysiological study in WME-related suppression of top–down attention for letter stimuli (Sergent et al., 2005).
We collected data from 14 volunteers (eight men, 19–29 years) with normal or corrected-to-normal vision who participated in the experiment for financial compensation. All participants gave informed written consent. The study was approved by the Vanderbilt University Institutional Review Board.
Trials began with a 350-msec fixation cross, immediately followed by the T1 display, which was a row of four simultaneously presented keyboard symbols (#, %, &, and $) arranged in random order (Figure 1). T1 appeared for 400 msec and was subsequently masked for 100 msec by a row of four digits (8888). After either a 50-msec or 950-msec blank interval, T2 appeared for 100 msec. T2 was a row of either four Hs or four Ss and was followed by a 2-sec response interval. On the trials for which a T2 response was required (see below), participants responded during this interval as quickly and as accurately as possible via keypress (using their right index or middle finger) to the identity of the T2 item (H or S). Subsequent to the T2 response interval, a T1 probe appeared for 2 sec. It consisted of a single keyboard symbol randomly presented in one of the positions occupied by the four T1 symbols. In half of the trials, the probe matched the T1 stimulus at that location. In the remaining nonmatch trials, the probe identity corresponded to a T1 stimulus occupying one of the other positions. Participants made an unspeeded keypress (using their left index or middle finger) to indicate whether the T1 probe symbol occupied the same position it had in the T1 stimulus display. A 12-sec fixation intertrial interval (ITI) ensued before the onset of the next trial.
Two independent variables were manipulated in this experiment. First, we manipulated the SOA between T1 and T2. The T1–T2 SOA was 550 msec for short SOA trials and 1450 msec for long SOA trials. Fourteen trials of each SOA occurred in each fMRI run; four runs of each task condition would consequently produce 56 trials per condition. The second variable was the number of targets to which participants attended and responded. In single-target fMRI runs, participants were instructed to attend to T1 only, whereas in dual-target runs, they attended to both targets. The order of run presentation consisted of two single-target runs, followed by four dual-target runs and a final two single-target runs. We chose this run order to counterbalance the run positions of the single- and dual-task trials while minimizing the frequency with which participants' task instructions changed. Because of time limitations and/or technical difficulties, we were unable to acquire the final single target run in one subject, the final dual-target and single-target runs in three subjects, and the final four runs in one subject.
The task was implemented and presented with the Vision Egg software package (Straw, 2008), running on Mac G4 hardware and Mac OS 10.4 software (Apple, Inc., Cupertino, CA).
Behavioral Prescreening Session
Because of the difficulty of the T1 task and the necessity of acquiring enough T1-correct trials for fMRI analysis, we adopted the same subject inclusion criterion employed by Jolicoeur (1998), using a prescreening session to identify participants who performed the task with at least 70% accuracy.
In the prescreening session, participants performed three task conditions: T2-only, T1-only, and dual-target. T1-only and T2-only conditions were included to give participants an opportunity to learn the target response mappings (which were identical to those used during fMRI scanning). During the T2-only runs, participants were instructed to disregard all the items but the T2 letters and to respond to these as quickly and as accurately as possible. The instructions and trial presentation procedure for T1-only and dual-target runs were identical to those described for the fMRI scanning session. The only exception was the ITI, which was shortened to 2 sec for the first half of the runs to reduce testing time. The remaining runs preserved the long ITIs to provide the participants with practice with the same parameters used during the fMRI session. Each participant performed six behavioral prescreening runs containing 28 trials (one T2-only, one T1-only, and four dual-target runs). Prescreening was carried out approximately 1 week before fMRI testing.
Behavioral Data Analyses
t Tests were employed to analyze T2 performance, because this stimulus was only identified in the dual-target condition, which included only two levels of a single factor (short SOA and long SOA). ANOVAs were used for analyzing T1 accuracy data, because T1 was identified in both single- and dual-target conditions, which each also included the SOA factor. T1 RT data are not analyzed because T1 responses were made off-line (and unspeeded) and, thus, are of minimal interpretative value.
fMRI Data Acquisition
fMRI data was acquired using a 3T Philips Intera Achieva scanner at the Vanderbilt University Institute of Imaging Science. The visual display was presented on an LCD panel and back-projected onto a screen positioned at the back of the magnet. Participants lay supine in the magnet and viewed the display on a mirror positioned above them. Responses were recorded using two 5-key keypads (right keypad for the T2 response and left keypad for T1; Rowland Institute of Science, Cambridge, MA). We acquired 3-D T1-weighted high-resolution images using a conventional protocol and functional (T2*-weighted) images with the following parameters: TR =300 msec, TE = 30 msec, field of view = 220 × 220 mm, 64 × 64 matrix. Because of the fast TR, only five axial slices (8-mm thick, 0.5-mm gap) were acquired parallel to the AC–PC plane to cover the ventral portion of the visual cortex (from the ventral surface of the occipital lobe to the superior occipital cortex).
fMRI Data Analysis
We analyzed the data from 11 participants (data from 3 of the original 14 participants were excluded from analysis because of large motion artifacts) for this slow event-related fMRI experiment. Image analysis was performed using Brain Voyager QX 1.8 (Brain Innovation, Maastricht, the Netherlands) and custom MATLAB software (Mathworks, Natick, MA). Data preprocessing included 3-D motion correction (aligned to initial EPI volume), slice-scan timing correction, and linear trend removal. Data were spatially smoothed using an 8-mm smoothing kernel and were aligned to the T1-weighted anatomical data, which were transformed into standardized Talairach space (Talairach & Tournoux, 1988).
We used a multiple regression analysis to create SPMs, defining regressors for the single-target (T1-only) long SOA, single-target short SOA, dual-target (T1–T2) long SOA, and dual-target short SOA conditions (only correctly answered trials were included in the analysis). For each subject, we contrasted dual-target trials with single-target trials (collapsed across SOAs) and identified in left middle occipital gyrus (LMOG; BA 18 and BA 19) the voxel that showed peak sensitivity (dual > single, Z > 1). We focused on the LMOG, as this region corresponds to the source of attenuated T2 attention-related signal identified by Sergent et al. (2005) and is specifically activated by letter discrimination tasks (Pernet et al., 2004). To avoid the selection of deactivated voxels that may surround the site of activation in visual cortex (e.g., Hein, Alink, Kleinschmidt, & Müller, 2009; Bressler, Spotswood, & Whitney, 2007), we selected only voxels that demonstrated positive activation across conditions (Zs > 0). All positively activated (dual > single) voxels within a 5.3-mm3 area (150 mm) of the peak voxel were then included in the ROI. One of the 11 participants failed to meet these criteria and was excluded from further analysis.
For each subject's ROI, we extracted time courses for the four conditions (Task × SOA). These time courses were then curve-fitted with a single gamma function to reduce the influence of noise on our analysis. This gamma function had the following parameters (baseline: lower bound = −0.05, upper bound = 0.05; peak amplitude: lower bound = 0, upper bound = inf; delay: lower bound = 6 sec, upper bound = 8 sec; dispersion: lower bound = 0.5 sec, upper bound = 2.5 sec; onset: lower bound = −4 sec, upper bound = 4 sec), which were held constant across all subjects.
To isolate the hemodynamic response specific to directing top–down attention to T2, we followed the logic outlined by Vogel and Luck (2002): for each participant, we subtracted the single- from dual-target waveforms at each SOA, then averaged the resulting subtracted waveforms across participants. To examine the influence of SOA on T2 attention-related BOLD response, we submitted the individual subject's subtracted waveforms at short and long SOAs to cross-correlation analysis and identified the peak positive cross-correlation coefficient as the SOA-related phase delay for each participant. The effect of SOA on the onset and peak latencies of the T2 waveforms was also analyzed. Latency of peak amplitude was identified as the time point at which percent signal change was highest for the gamma-fitted subtraction waveform, and onset latency of the waveform as the time point at which 10% of the total area under the curve had been reached (Hansen & Hillyard, 1980).
Experiment 2 investigated whether a T1 task with minimal WME demands continued to influence the latency of T2-related activation. If WME delayed attentional processing of T2 in the first experiment, then considerably reducing WME load in a second experiment should reduce or eliminate that delay. Eleven individuals participated in this experiment (six men, ages 20–28 years). The methods were identical to those used in Experiment 1 with the following exceptions. Rather than encoding the identity and spatial position of the T1 probe items, participants simply encoded whether or not T1 items appeared (80% of trials; this manipulation left a total of 44 trials in each condition that were subjected to analysis). During “T1 absent” trials, the T1 frame remained blank but was still followed by the T1 mask. The “T1 probe” stimulus was identical to that used in Experiment 1; in Experiment 2, however, participants used their left index finger to indicate that a T1 probe had occurred and their left middle finger to indicate that no T1 probe had occurred.
fMRI Data Analysis
fMRI data analysis methods employed for Experiment 2 were identical to those employed for the first experiment. In Experiment 2, however, three participants produced negative subtraction waveforms (between dual- and single-target) that could not be meaningfully subjected to cross-correlation analysis and, thus, were excluded from further analyses. Importantly, even when the cross-correlation data from these participants were included in the analysis, the pattern of results we observed did not change.
To establish that WME demands, rather than on-line T2 response execution processes, drove the results of Experiment 1, we conducted a third experiment in which T2 responses were postponed until the end of the trial. Five individuals participated in Experiment 3 (two men, ages 20–28 years). The methods were identical to those used for Experiment 1 with the following exceptions. On dual-target trials, participants did not make a speeded response to the T2 item. Instead, they encoded the identity of the T2 letters and withheld their response until the end of the trial. Immediately after the offset of the T1 probe, a T2 probe (HSHS) appeared. This probe prompted participants to make an unspeeded response via keypress (using their right index or middle finger) to the identity of the T2 item (H or S). In addition, the ITI was reduced from 12 to 11 sec, and the number of trials was decreased from 14 to 10 per SOA in each fMRI run. For the first three participants, an additional pilot condition was included in each run. This condition was dropped for the final two participants, allowing their data to be collected in shorter scanning sessions by doubling the number of trials per run and halving the number of runs. All participants performed 40 trials of each task-relevant condition.
fMRI Data Analysis
fMRI data analysis was identical to that of Experiments 1 and 2. One participant's subtraction waveforms were negative and, therefore, could not be meaningfully subjected to cross-correlation analysis. The pattern of results presented in this manuscript was preserved even when the data of this participant were included.
Experiment 1: The Effect of T1 WME Load on the Latency of T2 Activation
We used paired t tests to examine the RTs to T2 to confirm that WME consolidation delayed behavioral responses to the second target under dual-target conditions. As predicted, T2 RT was faster under long SOA (986 msec) than short SOA (1109 msec) conditions (paired t test, t(9) = 4.17, p = .002), indicating that T2 processing was delayed by T1 processing. This RT effect is not because of a speed accuracy tradeoff, as T2 accuracy was comparable (t(9) = 1.1; p = .3) under long (99%) and short SOA (97%) conditions. A repeated measures ANOVA (using the factors target number and SOA), however, revealed an effect of SOA on T1 accuracy. Specifically, T1 accuracy was greater under long SOA (87%) than under short SOA (82%) conditions (F(1, 9) = 5.98; p = .037). The interaction between target number and SOA, however, was not significant (F(1, 9) = 2.83, p = .126), suggesting that the influence of SOA on T1 processing may have been primarily driven by the physical characteristics of T2 (which were present in all target conditions) rather than the top–down attention directed to T2 (which varied between target conditions). Such a finding is consistent with other data indicating that the physical appearance of T2 may slightly interfere with T1 identification (Ptito, Arnell, Jolicoeur, & Macleod, 2008). Finally, T1 was more accurately remembered (F(1, 9) = 8.32; p = .018) in single-target (88%) than dual-target trials (81%); a result that may reflect competition between targets for representations and/or resource allocations under dual-target conditions (e.g., McLaughlin, Shore, & Klein, 2001).
fMRI Data Results
Consistent with prior findings, the behavioral results revealed that T1 encoding delays RT to T2 at the short SOA relative to the long SOA (Ouimet & Jolicœur, 2007). Does this delay, however, reflect a postponement of top–down related activation to T2 in visual cortex? To answer this question, we first dissociated T2-related activity from both T1-related activity and activation related to the physical presentation of T2—as both contribute to the response in dual-target trials—by subtracting the hemodynamic response obtained in single-target trials from that obtained in dual-target trials for each SOA. The resulting gamma-fitted waveforms were then compared for latency shifts in activation.
Consistent with our hypothesis, the short SOA waveform showed a mean phase delay of 1.2 sec relative to the long SOA waveform (Figure 2), a delay that was significantly greater than 0 (one-sample t test, t(9) = 2.59, p < .05). Moreover, this delay reflected a shift in both peak (short SOA = 7.41 sec, long SOA = 6.24 sec, t(9) = 2.34, p = .044) and onset (short SOA = 5.07 sec, long SOA = 4.44 sec, t(9) = 2.21, p = .054) latencies of the short SOA relative to the long SOA time course.
These findings indicate that reducing the SOA between T1 and T2 delays the hemodynamic response to T2 in visual cortex. Correspondingly, the behavioral results showed that reducing the T1–T2 SOA led to an increase in the RT to T2 (see above). We have hypothesized that the increased RT at short T1–T2 SOA results from the same processing delay that postponed top–down attention to T2. If this is the case, then this RT increase should be related to the hemodynamic response delay that we observed in visual cortex. We tested this hypothesis by assessing whether participants' RT delay (short T2 RT − long T2 RT) correlated with their phase differences in the hemodynamic response between the short and long SOA conditions in LMOG. Consistent with our hypothesis, there was a significant correlation between T2 phase delay and RT delay (r = .68, p = .03; see Figure 3), a striking result given that the phase delay was much greater than the RT delay (see Figure 2). One possible explanation for the scaling difference between SOA and RT may lie in the well-documented nonlinearity of the BOLD response in visual cortex to repeated stimulation at short interval; during an approximately 2 sec window following the presentation of an initial visual stimulus, BOLD signal in visual cortex often shows a sluggish and delayed response to further stimulation (Huettel, Obembe, Song, & Woldorff, 2004; Inan, Mitchell, Song, Bizzell, & Belger, 2004; Huettel & McCarthy, 2000, 2001). Such nonlinearity might account for how small differences in the latency of top–down activation could translate into larger differences in the latency of an indirect measure of that activation, namely the BOLD signal. In any case, the results from Experiment 1 provide clear evidence that WME delays attention-related activation in extrastriate visual cortex. Thus, it appears that the central bottleneck not only limits late stage decision-making processing but also early attentional modulation of visual information.
Experiment 2: The Effect of Minimal T1 Working Memory Demands on the Latency of T2 Activation
The results of the first experiment are consistent with the hypothesis that the demands of encoding a target into working memory delay top–down attentional processing of a subsequent target. If this hypothesis is correct, this delay should be reduced if the WME load of T1 is alleviated or reduced. We tested this prediction in a follow-up experiment that was identical to Experiment 1, except that the difficult T1 encoding task was replaced with a simple stimulus detection task. We predicted that any demands on WME should consequently be very short lived, thereby eliminating the SOA-related latency difference in attention-related extrastriate activation evoked by T2.
The RTs to T2 were slower at the short SOA (886 msec) than at the long SOA (797 msec) (t(6) = 3.45, p = .01). However, T2 accuracy was not affected by this factor (t(6) = 0.8, p = .45, 96% at the long SOA and 95% at the short SOA), and there were no main effects of SOA or target number on T1 accuracy (mean = 97%, ps > .26). Although we did not predict that our minimally demanding T1 task would produce an SOA-related difference in T2 RT, recent data from a skeletal attentional blink (AB) paradigm also reveals that a simple T1 detection task can produce SOA-related impairments in T2 identification (Nieuwenstein, Van der Burg, Theeuwes, Wyble, & Potter, 2009; see further discussion below).
There were no phase differences in the subtracted hemodynamic responses obtained at short and long SOAs (mean = −0.34 sec, t(6) = 0.62, p = .56; Figure 3). Similarly, peak latency (short SOA = 6.39 sec, long SOA = 6.81 s, t(6) = 0.67, p = .53) and onset latency (short SOA = 4.41 sec, long SOA = 4.54 sec, paired t(6) = 0.24, p = .81) were also unaffected by the SOA manipulation. These results indicate that T2-related activation is not delayed at the short SOA when T1 WME demands are minimal.
Although we found no influence of SOA on the latency of the BOLD response associated with top–down attention to T2, we did find that reducing the T1–T2 SOA still led to an increase in the RT to T2 despite the minimal T1 WME demands. This RT difference, however, did not correlate with the SOA phase difference in visual cortex activation (r = .1, p > .55; Figure 3). In experiment 2, then, we did not find the relationship between the behavioral measure of task performance and visual cortex activation that we observed in the first experiment. We , therefore, conclude that low T1 WME demands do not significantly affect the latency of top–down attentional processing of T2. As such, this second experiment provides further support for the hypothesis that the encoding of a target in working memory delays the deployment of top–down attention to another target. By the same token, the finding that the SOA manipulation still affected RTs in the absence of effects in visual cortex suggests that a simple detection task is sufficient to engage processes that delay the execution of the T2 response but is not sufficient to affect top–down attentional processing of T2.
Comparison between Experiments with High and Low WME Demands
To further confirm that manipulations of WME load underlie the SOA-related differences in latency of visual cortex activation, we directly compared the results from Experiments 1 and 2. We found a significant effect of WME demands on SOA-related phase delay (t(15) = 2.59, p = .04); the delay between short and long SOA trials was greater when WME demands were high compared with when they were low (compare Figures 2 and 3). The differential effect of WME load on visual cortex activity was not significantly reflected in the behavioral data, however. Specifically, although the effect of SOA on T2 RT was larger for high WME load (Experiment 1) than for low WME load (Experiment 2), this difference was not significant (p = .41). There was, however, a main effect of WME demand on overall T2 RT (high WME load = 1.04 sec, low WME load = 0.842 sec, F(1, 15) = 5.16, p = .038), indicating that higher WME demands generally slowed responses to T2. An analysis of T1 performance using a mixed model ANOVA with factors of experiment (low, high WME load), SOA (short, long), and target condition (single, dual) revealed a main effect of encoding demand on T1 accuracy (F(1, 15) = 16.2, p = 001). T1 accuracy was higher under conditions of low WME demand (97%) than high WME demand (85%), thereby confirming that the high WME load task was more difficult than the low WME task. In addition, WME demand interacted with target condition (F(1, 15) = 5.54, p = .033); T1 accuracy was higher for single (88%) than for dual (71%) target trials under conditions of high WME demand but not low WME demand (97% for both conditions). This finding further conveys the performance costs of the working memory load manipulation. WME demand did not interact with any other factors to influence T1 accuracy (p > .13).
What does the RT delay observed in Experiment 2 reflect if not WME-related delays in the action of top–down attention on T2 representations? First, our results are consistent with a previous study showing that even minimal T1 detection requirements in a skeletal rapid serial visual presentation paradigm are sufficient to impair the identification of a subsequently presented masked target (Nieuwenstein et al., 2009). Our behavioral data extend these findings by demonstrating that minimal T1 detection requirements delay, rather than impair, the identification of unmasked T2 material. However, our fMRI data suggest that these delays are unrelated to the timing of the attentional enhancement of T2 sensory signals. Analogous dissociations between the neurobiological and behavioral consequences of manipulating T1 difficulty on the T2 response have been observed in electrophysiological studies (Giesbrecht, Sy, & Elliott, 2007; Dell'Acqua et al., 2006). Instead, these findings may reflect a delay (caused by stimulus detection or task switching) in late stages of information processing (such as response execution) that do not affect top–down attention. Such an interpretation is consistent with studies showing that changes in T1 difficulty can affect electrophysiological components of target processing without changing the behavioral performance (Giesbrecht et al., 2007; Dell'Acqua et al., 2006) and with data suggesting that the elimination of a processing delay in a relatively early stage of information processing can unmask a delay in a later stage (Scalf, Banich, Kramer, Narechania, & Simon, 2007).
Experiment 3: Controlling for T2 On-line Response Execution Requirements
The finding in Experiment 1 that SOA-dependent delays in visual cortex activation correlate with SOA-related delays in T2 RT is consistent with the hypothesis that WME of one stimulus postpones the action of top–down attention onto a second item. Our experimental paradigm, however, included an “on-line” (speeded) response to T2, the preparation for which (rather than WME) may have modulated activation in the sensory areas representing the task-relevant stimulus (Toni et al., 2002). It is, therefore, possible that the visual cortex activation observed in our subtraction analysis was, at least in part, influenced by the neural systems supporting the execution of a speeded response made to T2, rather than the attentional processing of this item. To rule out this possibility, we carried out an additional experiment in which the T2 response was performed off-line (see Methods). If the SOA-related delays we observed in Experiment 1 were related to directing top–down attention to T2 rather than to executing a speeded response to T2, they should still be present even when response execution is delayed by several seconds. Because the purpose of this experiment was to replicate the results of our first experiment in the absence of a possible motor execution confound, we examined data from a small number of participants using one-tailed tests.
Because the T2 task in Experiment 3 was off-line and unspeeded, there were no effect of SOA on T2 RT (long SOA = 0.758 sec, short SOA = 0.741 sec, t(3) = 1.6, p = .79), although we did find a marginal effect of SOA on T2 accuracy (long SOA = 0.98, short SOA = 0.92, t(3) = 1.8, p = .085). There was no effect of target number (F(1, 3) = 1.05, p = .19) or SOA (F(1, 3) = 0.0083, p = .465) on T1 accuracy (mean = 81%) nor did these factors interact (F(1, 3) = 0.46, p = .27).
We found a significant phase delay between the short and long SOA conditions (mean delay = 1.28 sec, t(3) = 2.96, p = .03; Figure 4). This delay reflected a significant shift in both the peak latency (short SOA = 8.7 sec, long SOA = 7.05 sec, t(3) = 5.28, p = .006) and onset latency (short SOA = 6.9 sec, long SOA = 5.55 sec, t(3) = 3.13, p = .026). These results strongly suggest that the delay of T2-related visual cortex activation at the short SOA observed in Experiment 1 was not caused by execution of the T2 response but instead reflects WME-delayed top–down attention to visual cortex.
Our data demonstrate that the action of top–down attention on extrastriate cortex is delayed while WME processes are occupied with previously presented task-relevant items. In Experiment 1, we showed that reducing the interval between an initial target and a second target delayed an extrastriate signature of top–down attention to that second stimulus. Experiment 2 showed that this delay was abolished when the WME demands for T1 were reduced. Finally, in Experiment 3 we replicated our results from Experiment 1 in the absence of T2 speeded response requirements. To our knowledge, these findings are the first to indicate that occupation of WME processes by an initial target results in the serial postponement of top–down attention to the extrastriate representations of subsequent targets. By the same token, our results extend the work of de Fockert et al. (2001) by identifying one specific stage of working memory—WME—that interferes with top–down attention.
Given that our experimental manipulations affected the WME demands of T1 rather than directly modulating T2 processing, it is reasonable to question whether the BOLD effects observed in extrastriate cortex were related to top–down attention to T2. Our subtraction technique, similar to that employed by Vogel and Luck (2002), should have eliminated any contribution associated with the physical presentation of T2 and, most importantly, the processing of T1. Admittedly, this technique relies on the assumption of pure insertion (Sternberg, 1969), which may have been violated by our finding that the execution of a speeded T2 response slightly altered accuracy performance of the first task. We note, however, that the key finding of our study is based on SOA-related phase differences in the subtraction waveforms. If such phase differences reflect differences in T1 processing between the single- and dual-target conditions, then one would expect to find an interaction between target number (single, dual) and SOA on T1 accuracy, but this interaction, was not significant (p > .13). Furthermore, in Experiment 3 (in which responses to T2 were performed off-line), we continued to observe the SOA-related phase difference seen in Experiment 1 although T1 accuracy was not affected by SOA (T1 accuracy at short SOA: 80%, at long SOA: 81%). Admittedly, Experiment 3 included a small number of subjects, limiting our interpretation of the null hypothesis. Experiment 1, however, produced significant SOA-related T2 RT differences that correlated with SOA-related phase differences. Furthermore, we observed in Experiment 3 a marginal SOA-related T2 accuracy difference. In both experiments, then, we observed evidence of SOA-related differences in T2 performance that were unique to the dual-target condition and no evidence of an SOA-related difference in T1 performance unique to dual-target trials. We, therefore, believe that it is more parsimonious to conclude that the SOA-related phase differences we report reflect T2 processing delays rather than those associated purely with T1 operations.
As stated above, a WME-related delay in top–down modulation of extrastriate activation is consistent with the hypothesis that WME and top–down attention rely, at least to some extent, on a common neural process (Mayer et al., 2007; de Fockert et al., 2001). It has been suggested that concurrent increases in the demands placed on WME and attentional selection produce an underadditive BOLD response in posterior parietal cortex as well as a delayed response in inferior/middle frontal gyri, suggesting that visual–spatial attention and working memory compete for resources within these structures (Mayer et al., 2007). Working memory load has also been shown to disrupt attention-related suppression of visual cortex response to task-irrelevant stimuli (de Fockert et al., 2001). These results have been taken to indicate that top–down attention is controlled by working memory. Our study provides a crucial test of this hypothesis by demonstrating that the engagement of WME postpones the ability to exert top–down down-modulation of visual cortex activation. In addition, the present results are consistent with theories asserting that WME and top–down attention depend on a common “central executive” resource whose engagement with one operation would render it unavailable to any other similarly dependent operation; such models include the central interference theory (Brisson & Jolicoeur, 2007; Robitaille et al., 2007), perceptual load theory (Lavie, Hirst, de Fockert, & Viding, 2004), the biased competition model of attention (Desimone & Duncan, 1995), and the global work space model (Dehaene, Sergent, & Changeux, 2003).
Top–down attention is believed not only to enhance the representations of task relevant items but also to suppress the representations of task-irrelevant distractors (Pinsk, Doniger, & Kastner, 2004; Smith, Singh, & Greenlee, 2000; Rees, Frith, & Lavie, 1997). As mentioned previously, the amplitude of signal evoked by stimuli irrelevant to an attentionally demanding task may be increased if the demands of a concurrent working memory task are also increased; this change presumably reflects a reduction in the working memory resources available to control attention and suppress material irrelevant to the task (de Fockert et al., 2001). What happens to the timing of attention-related suppression of distractor-evoked signal during performance of a second task while WME resources are occupied with an initial task? Interestingly, different models of distractor inhibition predict different patterns of results. If distractor suppression and target enhancement are dependent on common neural mechanisms (e.g., Dalton, Lavie, & Spence, 2009; Brand-D'Abrescia & Lavie, 2007; Lavie & De Fockert, 2005), then the same WME conditions that delay attention-related enhancement of a subsequent target would also delay attention-related suppression. Other models of distractor suppression, however, propose that it may proceed independently from target enhancement if target and distractor processing are dependent on different neural systems (e.g., Park, Kim, & Chun, 2007; Kim, Kim, & Chun, 2005). By this account, one would expect to see WME-related delays in distractor suppression only under conditions in which the WME task and distractor processing rely on a common neural resource. Future investigations of the WME conditions that delay distractor suppression may help to clarify the relationship between distractor suppression and target enhancement.
WME does not only appear to affect top–down attention; it has also been implicated in awareness. This is best exemplified by WME's putative role in AB, which refers to the deficit in the perception of the second of two temporally proximate targets presented in a rapid serial visual presentation of distractor items (Chun & Potter, 1995; Raymond, Shapiro, & Arnell, 1992). A large volume of AB work (for a review, see Dux & Marois, 2009) has suggested that WME of an initial target item postpones the processing of subsequent target items (Sergent et al., 2005; Dehaene et al., 2003; Vogel & Luck, 2002; Jolicoeur, 1998, 1999; Chun & Potter, 1995), leaving the latter item vulnerable to interference by a trailing mask (Giesbrecht & Di Lollo, 1998). In line with this hypothesis, neurobiological studies of AB have generally concluded that T2 deficit primarily occurs because the second item does not gain access to central stages of information processing in frontal and parietal cortex (Kranczioch, Debener, Schwarzbach, Goebel, & Engel, 2005; Dehaene et al., 2003; Marois, Chun, & Gore, 2000; Luck, Vogel, & Shapiro, 1996). More recent studies, however, indicate that striate and extrastriate representations of T2 are modulated during AB (Slagter, Johnstone, Beets, & Davidson, 2010; Hein et al., 2009; Stein, Vallines, & Schneider, 2008; Williams, Visser, Cunnington, & Mattingley, 2008; Sergent et al., 2005; Marois, Yi, & Chun, 2004). In particular, Williams et al. (2008) showed that the visual cortex response evoked by a masked, attended T2 item is suppressed when the T1–T2 SOAs are short and T1 is attended. In the present study, we found that short T1–T2 SOAs delayed attentional modulation of T2-related activity in visual cortex when T2 stimuli were unmasked. Taken together, these findings support the idea that top–down attention is unavailable to T2 whereas WME processes are occupied with T1; processing of an unmasked T2 is, therefore, likely to be delayed (present study), whereas processing of a masked T2 is likely to be interrupted (Williams et al., 2008).
Limits in WME are not considered to be the only source of AB deficit (see Dux & Marois, 2009). Other AB models propose that T1 processes unrelated to WME, such as task switching (Potter, Chun, Banks, & Muckenhoupt, 1998), post-T1 stimulus suppression (Olivers & Meeter, 2008; Raymond et al., 1992; see also Dux & Harris, 2007; Dux, Coltheart, & Harris, 2006), and attentional filter disruption (Di Lollo, Smilek, Kawahara, & Ghorashi, 2005), can also impair responses to T2. Furthermore, changes in the difficulty of the T1 task can shift the psychological locus at which responses to T2 are impaired (Giesbrecht et al., 2007; Dell'Acqua et al., 2006). One or more of these other processing limitations may explain the results of our second experiment; low T1 WME demands still produced SOA-related delays in unmasked T2 report, although they eliminated SOA-related delays in the top–down enhancement of visual cortex activity. These findings are consistent with those of other researchers: the reduction or elimination of T1 processing demands has been shown to eliminate electrophysiological evidence of SOA-related attenuation of T2 perceptual processing in the face of preserved deficits in responding to T2 (Giesbrecht et al., 2007; Dell'Acqua et al., 2006). The implications of our data for models of AB, however, are somewhat constrained by our use of an unmasked T2 item (which resulted in SOA-related delays in T2 identification rather than the SOA-related impairment in T2 processing). When considered across all three experiments, however, our results generally support a multifactorial account of the AB (Dux & Marois, 2009; Kawahara, Enns, & Di Lollo, 2006).
In summary, our data suggest that the consequences of serial postponement of access to central stages of information processing are widely distributed throughout the brain. These consequences can be observed not only at the putative sources of the central processing limitations in prefrontal and parietal cortex (Sigman & Dehaene, 2008; Dux, Ivanoff, et al., 2006; Marois & Ivanoff, 2005) but also at the sites of stimulus representation on which these processing limitations operate. Consequently, these operations appear to be truly “central” to the successful implementation of multiple cortical processes that support conscious perception.
This work was supported by National Science Foundation grant 0094992 and NIMH grant R01 MH70776, both to R. M., and by a P30-EY008126 grant to the VVRC. P. E. D. was supported by an Australian Research Council APD fellowship (DP0986387).
Reprint requests should be sent to Paige E. Scalf, Beckman Institute, University of Illinois at Urbana Champaign, 405 N Mathews, Urbana, IL 61801, or via e-mail: firstname.lastname@example.org.