Performing multiple tasks concurrently places a load on limited attentional resources and results in disrupted task performance. Although human neuroimaging studies have investigated the neural correlates of attentional load, how attentional load affects task processing is poorly understood. Here, task-related neural activity was investigated using fMRI with conventional univariate analysis and multivariate pattern analysis (MVPA) while participants performed blocks of prosaccades and antisaccades, either with or without a rapid serial visual presentation (RSVP) task. Performing prosaccades and antisaccades with RSVP increased error rates and RTs, decreased mean activation in frontoparietal brain areas associated with oculomotor control, and eliminated differences in activation between prosaccades and antisaccades. However, task identity could be decoded from spatial patterns of activation both in the absence and presence of an attentional load. Furthermore, in the FEFs and intraparietal sulcus, these spatial representations were found to be similar using cross-trial-type MVPA, which suggests stability under attentional load. These results demonstrate that attentional load may disrupt the strength of task-related neural activity, rather than the identity of task representations.
The ability to allocate attention appropriately is a critical component of cognitive control and is essential for goal-directed behavior. To achieve a desired outcome, attention is required to select and maintain behaviorally relevant configurations of neural processes or task sets (Sakai, 2008; Monsell, 2003) while filtering out irrelevant task sets, thoughts, emotions, and sensations. In everyday life, multiple tasks are often performed simultaneously or in rapid alternation, and multiple task sets must be maintained. However, neural resources and information processing in the brain are limited, and the presence of additional tasks places a load on attention and cognitive control.
When two tasks are prepared for or performed concurrently, an increased load on task processing is thought to increase errors and RTs for one or both tasks (Stuss, Shallice, Alexander, & Picton, 1995; Pashler, 1994). fMRI studies examining the neural basis of task interference using dual tasks have shown mixed results. Brain areas involved in single-task performance have been found to have greater activation during dual-task performance (Nijboer, Borst, van Rijn, & Taatgen, 2014; Mizuno, Tanaka, Tanabe, Sadato, & Watanabe, 2012; Erickson et al., 2005; Adcock, Constable, Gore, & Goldman-Rakic, 2000; Bunge, Klingberg, Jacobsen, & Gabrieli, 2000; Klingberg, 2000). Increased activation suggests that impairments in performance are caused by competition for region-specific attentional resources. Alternatively, some dual tasks elicit a decrease in activation compared with single-task performance (Nijboer et al., 2014; Mizuno et al., 2012; Just, Keller, & Cynkar, 2008; Newman, Keller, & Just, 2007; Just et al., 2001; Klingberg, 1998) and suggest a limit on globally available attentional resources. Nonetheless, the neural mechanisms involved in attentional load and how attentional load affects task processing are poorly understood.
Specifically, the concurrent preparation of two tasks requires the simultaneous maintenance of both task representations in the brain. Whether or not these representations are disrupted during simultaneous task preparation is unclear. Conventional univariate analysis with fMRI, which typically examines task-related changes in mean regional activation levels, may lack the sensitivity to capture distinct representations. On the other hand, multivariate pattern analysis (MVPA), which takes into account fine-grained spatial patterns of voxel activation, has recently emerged as a more sensitive approach to investigate how information is neurally encoded (Haynes et al., 2007; Haynes & Rees, 2006; Kamitani & Tong, 2005; Haxby et al., 2001). Consequently, MVPA may detect aspects of task processing under attentional load that would be missed using univariate analysis.
In this study, participants performed a prosaccade task, where they generated a saccade toward a peripheral stimulus, and an antisaccade task, where they generated a saccade away from a peripheral stimulus to the mirror opposite location (Munoz & Everling, 2004; Hallett, 1978). These tasks have distinct stimulus–response associations, which make them useful for investigating task representations. To create an attentional load, participants performed a rapid serial visual presentation (RSVP) task during the preparatory period for prosaccades and antisaccades (Chan & DeSouza, 2013; Joseph, Chun, & Nakayama, 1997; Raymond, Shapiro, & Arnell, 1992). How attentional load affects task processing was examined using conventional univariate analysis and MVPA. This unique approach enabled us to investigate task-related brain activity and task representations in the absence and presence of an attentional load.
All procedures were reviewed and approved by the York University Human Participants Review Committee and the Queen's University Human Research Ethics Board. Eight participants (four women) with a mean age of 26.6 years (range = 20–40 years) provided written informed consent and completed the experiment. All participants had normal vision or corrected-to-normal vision, and no known neurological, psychiatric, or visual disorders.
Apparatus and Data Acquisition
Stimuli were generated using Presentation Version 12.1 (Neurobehavioral Systems, Inc., Albany, CA) and projected onto a screen at the back of the bore of the MRI scanner using a NEC LT265 DLP projector (NEC, Tokyo, Japan) with a resolution of 1024 × 768. Participants viewed the stimuli through a mirror mounted onto the head coil. Infrared eye-tracking was conducted at 120 Hz with an ISCAN eye-tracking system (ISCAN, Inc., Woburn, MA). Button responses were collected using a fORP-optic button joystick (Current Designs, Inc., Philadelphia, PA) placed in the right hand.
All imaging was conducted using a 3-T Magnetom Trio with Tim (Siemens Medical Systems, Erlangen, Germany) whole-body MRI scanner with a 12-channel head coil. Before functional image acquisition, structural images were collected for each participant using a T1-weighted sequence (repetition time = 1760 msec, echo time = 2.6 msec, flip angle = 9°) with 176 1-mm-thick slices of 480 × 512 pixels at a 0.5 mm × 0.5 mm in-slice resolution. The BOLD signal was measured using T2*-weighted EPI (repetition time = 1970 msec, echo time = 2.6 msec, flip angle = 78°, volume acquisition time = 2.0 sec) with thirty-two 3.3-mm-thick slices at a 3.3 mm × 3.3 mm resolution. Nine runs of BOLD fMRI were acquired per participant. The first six runs were the main experiment (see Experimental Paradigm section). The last three runs were used to identify participant-specific ROIs (see Functional Localizer section), independent from the experimental data. Univariate analysis and MVPA were performed in these ROIs for the experimental runs.
Prosaccades and antisaccades were performed in a block design similar to a previous study (DeSouza, Menon, & Everling, 2003). Here, an RSVP task was simultaneously presented during the prosaccade and antisaccade instruction cues to create an attentional load.
Prosaccade and antisaccade blocks were interleaved and alternated with fixation blocks, and each run began and ended with a fixation block (Figure 1A). Each trial began with a black background and a central fixation point (gray box subtending a visual angle of 1.7°). After 500 msec, the fixation point changed color to either green or red. Participants were instructed to recognize one color as a cue to perform a prosaccade toward a peripheral stimulus and the other color as a cue to perform an antisaccade away from a peripheral stimulus. Instruction cue color was counterbalanced between participants. An RSVP task was presented at 10 letters per second inside the green or red instruction cue (Figure 1B), and participants were asked to immediately report the appearance of a single number on each trial with a button press using their right index finger. The instruction cue was presented for 1000 msec, and the RSVP target could not appear in the first or last 100 msec. After 1000 msec, the instruction cue and RSVP task were extinguished, and a peripheral stimulus (gray box subtending a visual angle of 1.7°) was randomly presented 27° to the left or right of center. Participants had 1500 msec to make the appropriate eye movement. Participants were instructed to complete both the RSVP task and prosaccade or antisaccade task as quickly and accurately as possible. Each trial lasted 3 sec, and there were five trials in each 16-sec prosaccade and antisaccade block (each block was 8 volumes in length). Five prosaccade and five antisaccade blocks were performed for each functional run. Fixation blocks lasted 16 sec and consisted of a central fixation point.
Each participant performed three functional runs of this experiment. Subsequently, an additional three functional runs were performed, and participants were instructed to ignore the RSVP task. Ignoring the RSVP task during a dual task has been found to be similar to performing a single task (Joseph et al., 1997). Thus, attentional load was removed and experimental stimuli were identical for both experimental conditions. Before functional scanning, participants practiced 20 trials with RSVP and 20 trials without RSVP (10 prosaccades and 10 antisaccades each). Participants were aware of the alternating order of prosaccade, antisaccade, and fixation blocks before performing the experiment.
Eye tracking confirmed that participants maintained fixation on the central fixation point during task performance. Trials were marked as correct if the first movement from the fixation point was toward the peripheral stimulus for prosaccades or away from the peripheral stimulus for antisaccades. Trials were marked as direction errors if the first movement was in the incorrect direction. Saccadic reaction time (SRT) was defined as the time from the appearance of the peripheral stimulus to the initiation of the saccade on correct trials. Two-tailed t tests were conducted on mean error rates and mean SRTs to compare between saccade type (prosaccades vs. antisaccades) and attentional load condition (with RSVP vs. without RSVP).
All participants performed a separate event-related prosaccade and antisaccade experiment in addition to the main experiment. This task was expected to evoke bilateral activation of the FEFs, supplementary eye fields (SEFs), intraparietal sulcus (IPS), and higher visual cortex (HVC) and was consequently used to functionally define these ROIs in individual participants. Participants performed prosaccades and antisaccades, with an RSVP task simultaneously presented during the instruction cues. Unlike the main experiment, trials were presented individually, interleaved with rest periods, rather than in blocks of multiple trials. These brain areas are reliably and similarly activated by both prosaccades and antisaccades (Jamadar, Fielding, & Egan, 2013) and are known to activate more strongly for antisaccades compared to prosaccades during the preparatory period (Brown, Vilis, & Everling, 2007; Ford, Goltz, Brown, & Everling, 2005; DeSouza et al., 2003). We therefore compared activation during antisaccade preparatory periods to rest periods to identify ROIs.
Each trial began with a dark screen and a central fixation point (gray box subtending a visual angle of 1.7°). After 6 sec, the fixation point changed color randomly to green or red. Participants were instructed to recognize one color as a cue to perform a prosaccade toward a peripheral stimulus and the other color as a cue to perform an antisaccade away from a peripheral stimulus. Instruction cue color was counterbalanced between participants. During the instruction cue, which lasted for 12 sec, an RSVP task was simultaneously presented at 10 letters per second, and participants were asked to report the appearance of three numbers on each trial with a button press using their right index finger. The RSVP target could not appear in the first or last 100 msec of the instruction cue. Subsequently, the instruction cue and RSVP task were extinguished, and a peripheral stimulus (gray box subtending a visual angle of 1.7°) was randomly presented 27° to the left or right of the fixation point. Participants had 2 sec to make the appropriate eye movement. Participants were instructed to complete both the RSVP task and prosaccade or antisaccade task as quickly and accurately as possible. These runs consisted of 10 prosaccade and 10 antisaccade trials that were pseudorandomly interleaved, and each participant completed three runs.
Preprocessing and Analysis
Functional localizer data were preprocessed and analyzed using conventional general linear model (GLM) approaches in FSL v5.0.1 (Jenkinson, Beckmann, Behrens, Woolrich, & Smith, 2012). Preprocessing for each fMRI run included deletion of the first 4 volumes, motion correction (MCFLIRT), brain extraction (BET), spatial smoothing (8 mm kernel, FWHM), high-pass temporal filtering (0.01 Hz cutoff), and linear registration (FLIRT) among functional space, T1-weighted structural space, and 2-mm standard MNI 152 space. For each run, a first-level GLM with FILM prewhitening was performed with three explanatory variables and their temporal derivatives, each convolved with a gamma hemodynamic response function: prosaccade instruction periods (12-sec duration per trial), antisaccade instruction periods (12-sec duration per trial), and saccade periods (2-sec duration per trial). A contrast for antisaccade instruction minus rest was performed. Resulting parameter estimate images were entered into a second-level fixed effects GLM for analysis across runs within individuals. Finally, statistical images obtained from the second-level GLM were entered into a third-level, whole-brain fixed effects GLM in 2-mm MNI 152 space across all eight participants (FWE-corrected thresholding: Z > 2.3, cluster-based p < .05 based on peak intensities and spatial extent as implemented in FSL).
The thresholded group-level Z image for the antisaccade preparatory period minus rest contrast included L-IPS as well as bilateral FEF, SEF, and HVC. When we removed the cluster-based threshold, a R-IPS cluster that appeared homologous to the L-IPS was present. Therefore, for all ROIs except R-IPS, we split the cluster-corrected image into separate clusters to obtain images for each ROI. We created the R-IPS image by splitting the cluster-uncorrected image. All group-level MNI space ROIs obtained from these procedures are shown in Figure 2. L-FEF and R-FEF were functionally defined using the most strongly activated voxels along the precentral sulcus. As a result, FEF is defined more inferior than its conventional location at the junction of the precentral and superior frontal sulci.
We next defined ROIs specific to individual anatomy by extracting from the group-level ROIs the top 120 voxels most strongly activated within a given individual (based on second-level GLM results), excluding voxels at the edge of the brain that were deleted using two iterations of the fslmaths –ero command which removes a 3-mm3 kernel on each iteration. This number of voxels is similar to that used in previous MVPA studies of functionally defined ROIs (Zhang, Kriegeskorte, Carlin, & Rowe, 2013; Zhang & Kourtzi, 2010; Haynes & Rees, 2005). In six cases (two occurrences each in three participants), fewer than 120 voxels were found in an ROI after registration from group level to individual images when excluding brain edge voxels, so slightly smaller ROIs were used (range = 77–114 voxels). To validate all MVPA results and account for factors such as differences in head motion and baseline activation between tasks, we additionally defined control ROIs of (a) a 5-mm-radius sphere in deep white matter (WM) (centered at MNI coordinates x = 26, y = −12, z = 34) defined in standard MNI space and then transformed to each participant's cross-run aligned functional image space (see Preprocessing and Voxel Pattern Preparation section) and (b) 120 contiguous voxels in nonbrain (NB) space outside the brain defined individually in each participant's cross-run aligned functional image space (see Preprocessing and Voxel Pattern Preparation section). The nonbrain ROI was always defined 20–25 mm to the right of the participant's head as a 5 × 5 × 5 (125) voxel cube with five random edge voxels removed.
For comparison and to aid interpretation of MVPA results, we performed conventional univariate GLM analyses to assess mean activation levels within ROIs. Although a sample of eight participants may result in a lack of power and false negatives, this sample size is similar to previous block design and event-related design studies with prosaccades/antisaccades and univariate analyses (Brown, Goltz, Vilis, Ford, & Everling, 2006; Ford et al., 2005; DeSouza et al., 2003). Preprocessing involved the same procedures as those used for functional localizer runs (described above), except that the first 2 (instead of 4) volumes of each run were deleted because of differences in onset time of the first task trial. A first-level GLM with FILM prewhitening was performed within runs with two explanatory variables (and their temporal derivatives), each convolved with a gamma hemodynamic response function: 16 sec prosaccade with RSVP (Runs 1–3) or prosaccade (Runs 4–6) blocks and 16 sec antisaccade with RSVP (Runs 1–3) or antisaccade (Runs 4–6) blocks. Resulting parameter estimate images for activations associated with each condition were entered into second-level fixed effects GLMs for analysis across runs within individuals (separately for Runs 1–3 and Runs 4–6). The mean of contrast of parameter estimate (i.e., beta) values was extracted from each ROI for each condition and converted to percentage signal change. Across participants, two-tailed paired t tests were used to compare mean percentage signal change in each ROI for prosaccades versus antisaccades, prosaccades with RSVP versus antisaccades with RSVP, prosaccades versus prosaccades with RSVP, and antisaccades versus antisaccades with RSVP. Statistical significance was accepted at p < .00625 (i.e., Bonferroni-corrected for eight ROIs for each comparison).
Multivariate Pattern Analysis
For MVPA, preprocessing was performed in FSL v5.0.1, and analyses were conducted at the single participant level with the Pattern Recognition for Neuroimaging Toolbox (PRoNTo v.1.1; Schrouff et al., 2013) and custom Matlab (Mathworks, Inc., Natick, MA) code.
Preprocessing and Voxel Pattern Preparation
The mean of the middle volumes across runs was calculated, and each run was aligned to the resulting image with MCFLIRT. This procedure was performed to minimize any effects of changes in head position between runs on ROI location, as reported previously (Jimura & Poldrack, 2012). High-pass temporal filtering (cutoff: 0.01 Hz) was then performed within runs to remove linear trends/signal drift. The concatenated runs were linearly registered (FLIRT) to T1-weighted anatomical scans and the standard MNI 152 brain. Individual-specific ROIs obtained from the functional localizer runs were linearly transformed from MNI 152 space to cross-run aligned functional space. No spatial smoothing was performed, so fine-grained patterns of activity within ROIs were retained.
We defined individual trials (“examples”) for MVPA as eight consecutive fMRI volumes corresponding to each block, with the first of these volumes at 4 sec after block onset and the last of these volumes at 4 sec after block onset to account for hemodynamic delays. This resulted in 15 trials (corresponding to 15 blocks) each for 4 conditions: prosaccades, antisaccades, prosaccades with RSVP, and antisaccades with RSVP. For each ROI, we created a kernel matrix consisting of pairwise similarity between examples.
Binary Classification of Prosaccades versus Antisaccades
Linear support vector machine classifiers (constant cost parameter, C = 1) were used on kernels with mean-centered features (where the voxel-wise mean is subtracted) to classify prosaccades versus antisaccades as well as prosaccades with RSVP versus antisaccades with RSVP. A “leave-one-stimulus-pair-out” cross-validation approach was used with 15 folds. Each fold used two left-out blocks for testing the classifier (i.e., one prosaccade block and one antisaccade block) and the remaining blocks for training the classifier (i.e., 14 prosaccade blocks and 14 antisaccade blocks). Within each fold, each of the eight volumes for a given left-out block was tested on separately to maximize the data available for calculating accuracy, such that 16 tests (eight for each condition) were performed. While individual volumes in a block corresponded to slightly different periods of a saccade trial, volumes were treated equally because this variability could not be disentangled given the block design and hemodynamic delay. Overall, the final decoding accuracy value was the fraction of 240 tests (15 folds × 16 iterations per fold) that were labeled correctly. Mean decoding accuracy was calculated for each ROI and entered into a group level analysis. To test whether mean decoding accuracy across participants for an ROI was significantly higher than 50% chance decoding, we conducted a two-tailed t test. As a control, mean decoding accuracy across participants for each task-related ROI was also compared to the white matter ROI using a two-tailed paired t test. Statistical significance was accepted at p < .00625 (i.e., Bonferroni-corrected for eight ROIs).
To determine whether tasks were encoded using similar representations in the absence and presence of an attentional load, we conducted cross-trial-type MVPA. Here, one set of trials was used to train the support vector machine classifiers, whereas the other set was used to test the classifiers. First, we trained classifiers on prosaccades versus antisaccades and tested on prosaccades with RSVP versus antisaccades with RSVP (single-to-dual task decoding). Next, this analysis was repeated, but the classifier was trained on prosaccades with RSVP versus antisaccades with RSVP and tested on prosaccades versus antisaccades (dual-to-single task decoding). In contrast to the “leave-one-stimulus-pair-out” cross-validation scheme described for binary classification, we used all available data for both classifier training and testing (Gallivan, McLean, Smith, & Culham, 2011; Smith & Muckli, 2010). Consequently, one fold each of cross-validation was performed for single-to-dual task and dual-to-single task decoding. In single-to-dual task decoding, we trained the classifier on all 30 blocks for prosaccades versus antisaccades and tested on all 30 blocks for prosaccades with RSVP versus antisaccades with RSVP. In dual-to-single task decoding, we trained the classifier on all 30 blocks for prosaccades with RSVP versus antisaccades with RSVP and tested on all 30 blocks for prosaccades versus antisaccades. Each fold included 240 tests (i.e., 30 blocks × 8 volumes per block), and decoding accuracy was calculated as the fraction of correct classifications for these tests, separately for the two folds of cross-trial-type decoding. The mean of decoding accuracy values for single-to-dual and dual-to-single task decoding was considered as the final decoding accuracy for each participant. Significance of cross-trial-type decoding accuracy across participants was assessed as in the binary classification analysis.
Attentional Load Increases Error Rates and RTs
Figure 3A shows mean error rates for prosaccade and antisaccade trials without and with RSVP. Error rates on antisaccades with RSVP were significantly greater than prosaccades with RSVP (p < .01). There was a trend for error rates on antisaccades without RSVP to be significantly greater than prosaccades without RSVP (p = .07). Error rates on antisaccades with RSVP were significantly greater than without RSVP (p < .05), and this trend was present for prosaccades (p = .09).
Figure 3B shows SRTs for prosaccade and antisaccade trials without and with RSVP. SRTs on antisaccades without RSVP were significantly greater than prosaccades without RSVP (p < .01). Similarly, SRTs on antisaccades with RSVP were significantly greater than prosaccades with RSVP (p < .01). SRTs on prosaccades and antisaccades with RSVP were significantly greater than without RSVP (p < .05 for both). RTs for the RSVP task during prosaccade trials (444 ± 24 msec) were not significantly different from RTs during antisaccade trials (434 ± 27 msec; p > .05).
Prosaccade and Antisaccade Task Discrimination
Figure 4A shows mean activation for prosaccades and antisaccades in each ROI. L-IPS (p = .00032) and R-IPS (p = .0056) showed significantly greater activation for antisaccades compared with prosaccades. The FEF and SEF also showed greater activation for antisaccades compared to prosaccades but did not reach significance at the Bonferroni-corrected level (L-FEF: p = .048; R-FEF: p = .0066; L-SEF: p = .024; R-SEF: p = .0095). In contrast, there were no clear differences in mean activation levels for L-HVC (p = .60) or R-HVC (p = .75).
Task decoding accuracies revealed by MVPA are shown in Figure 4B. BOLD activity patterns in L-FEF (57.3%, p = .0022), L-IPS (58.8%, p = .004), R-IPS (59.4%, p < .0001), and R-HVC (62.6%, p = .0015) accurately predicted task identity above 50% chance decoding. In addition, decoding accuracies for L-FEF (p = .0051), L-IPS (p = .0018), R-IPS (p = .0001), and R-HVC (p = .0017) were significantly greater than the decoding accuracy for the white matter ROI (48.2%). Although decoding accuracies did not reach significance at the Bonferroni-corrected level for the other ROIs, mean decoding accuracy for each of these was above 50% chance decoding (R-FEF: 54.8%, p = .0085; L-SEF: 52.9%, p = .0076; R-SEF: 54.6%, p = .029; L-HVC: 59.8, p = .0493) and above the decoding accuracies for the white matter (48.2%, p = .1592) and nonbrain (50.6%, p = .7241) ROIs.
To determine whether there was a relationship between decoding accuracies for different ROIs, decoding accuracy for each ROI was correlated with decoding accuracy for each of the other ROIs using the Pearson correlation coefficient. Decoding accuracies for ROIs were not significantly correlated with each other (p > .0101 for all, statistical significance accepted at p < .0018, Bonferroni-corrected for 28 comparisons).
To determine whether differences in mean activation between prosaccades and antisaccades contributed to decoding accuracy, mean activation for prosaccades was subtracted from mean activation for antisaccades in each ROI and correlated with decoding accuracy using the Pearson correlation coefficient. For all ROIs, the difference in mean activation between prosaccades and antisaccades was not significantly correlated with decoding accuracy (p > .05 for all, statistical significance accepted at p < .00625, Bonferroni-corrected for eight ROIs).
Attentional Load Affects Mean Activation but Not Task Decoding
Figure 5A shows mean activation for prosaccades with RSVP and antisaccades with RSVP in each ROI. Unlike the comparison of prosaccades to antisaccades without RSVP, L-HVC (p = .0028) and R-HVC (p = .00063) showed significantly greater activation for prosaccades with RSVP compared to antisaccades with RSVP. Furthermore, no significant differences in mean activation between prosaccades with RSVP and antisaccades with RSVP were found in brain areas commonly associated with oculomotor task performance (L-FEF: p = .91; R-FEF: p = .53; L-SEF: p = .97; R-SEF: p = .30; L-IPS: p = .75; R-IPS: p = .58).
Overall, mean activation was greater within all ROIs for prosaccades and antisaccades without RSVP, compared to prosaccades and antisaccades with RSVP. Prosaccades, compared to prosaccades with RSVP, were associated with significantly greater activation in L-FEF (p = .0035), R-FEF (p = .00045), and R-SEF (p = .0032). L-SEF (p = .013), L-IPS (p = .018), R-IPS (p = .0089), L-HVC (p = .0080), and R-HVC (p = .010) showed greater activation but did not reach significance at the Bonferroni-corrected level. Antisaccades, compared with antisaccades with RSVP, were associated with significantly greater activation in R-FEF (p = .00052), L-SEF (p = .0040), R-SEF (p = .0012), L-IPS (p = .0024), R-IPS (p = .0017), L-HVC (p = .0013), and R-HVC (p = .0011). L-FEF (p = .0067) showed greater activation but did not reach significance at the Bonferroni-corrected level.
However, task decoding accuracies revealed by MVPA for prosaccades and antisaccades with RSVP were similar to the results for prosaccades and antisaccades without RSVP (Figure 5B). BOLD activity patterns in R-FEF (55.2%, p = .0043), L-IPS (59.1%, p = .0018), R-IPS (58.8%, p = .0011), and L-HVC (61.6%, p = .0012) accurately predicted task identity above 50% chance decoding. In addition, decoding accuracies for L-IPS (p = .0022), R-IPS (p = .0013), and L-HVC (p = .0053) were significantly greater than the decoding accuracy for the white matter ROI (50.5%). Although decoding accuracies did not reach significance at the Bonferroni-corrected level for the other ROIs, mean decoding accuracy for each of these was above 50% chance decoding (L-FEF: 55.3%, p = .099; L-SEF: 53.8%, p = .1976; R-SEF: 54.2%, p = .0378; R-HVC: 61.5%, p = .01) and above the decoding accuracies for the white matter (50.5%, p = .6507) and nonbrain (49.2%, p = .4987) ROIs.
To determine whether there was a relationship between decoding accuracies for different ROIs, decoding accuracy for each ROI was correlated with decoding accuracy for each of the other ROIs using the Pearson correlation coefficient. Decoding accuracies for ROIs were not significantly correlated with each other (p > .0106 for all, statistical significance accepted at p < .0018, Bonferroni-corrected for 28 comparisons).
To statistically compare decoding accuracies with RSVP to decoding accuracies without RSVP, an ANOVA analysis was performed with factors for Attentional load and ROIs. There was a significant difference for decoding accuracies between ROIs (p < .0001), but no significant difference between decoding accuracies with RSVP and without RSVP (p > .05). In addition, there was no significant interaction between Attentional load and ROI (p > .05).
Stable Task Representations Revealed Using Cross-trial-type Decoding
Although MVPA revealed that prosaccades and antisaccades with RSVP could be distinguished based on activity patterns in R-FEF, L-IPS, and R-IPS, despite no differences in mean activation, the prior analyses do not shed light on whether the patterns in these ROIs are similar or different from the patterns that distinguish prosaccades versus antisaccades without RSVP. We therefore performed a cross-trial-type decoding analysis to test the similarity between activity patterns distinguishing prosaccades and antisaccades, without and with attentional load.
Task decoding accuracies revealed by MVPA for single-to-dual and dual-to-single task decoding were similar (Table 1) and combined for analyses. Figure 6 shows decoding accuracies for task identity using cross-trial-type decoding. BOLD activity patterns in R-FEF (52.6%, p = .0057), L-IPS (55.5%, p = .0057), and R-IPS (54.9%, p = .0017) accurately predicted task identity above 50% chance decoding and may occur independent of the absence or presence of an attentional load. In addition, decoding accuracies for L-IPS (p = .0052) and R-IPS (p = .0033) were significantly greater than the decoding accuracy for the white matter ROI (50.6%). Although decoding accuracies did not reach significance at the Bonferroni-corrected level for the other ROIs, mean decoding accuracy for each of these was above 50% chance decoding (L-FEF: 52.0%, p = .06; L-SEF: 51.5%, p = .1795; R-SEF: 51.6%, p = .0354; L-HVC: 55.1%, p = .0094; R-HVC 53.9%, p = .0515) and above the decoding accuracies for the white matter (50.6%, p = .3031) and nonbrain (48.9%, p = .1303) ROIs.
|.||L-FEF .||R-FEF .||L-SEF .||R-SEF .||L-IPS .||R-IPS .||L-HVC .||R-HVC .||White Matter .||Nonbrain .|
|.||L-FEF .||R-FEF .||L-SEF .||R-SEF .||L-IPS .||R-IPS .||L-HVC .||R-HVC .||White Matter .||Nonbrain .|
To determine whether there was a relationship between decoding accuracies for different ROIs, decoding accuracy for each ROI was correlated with decoding accuracy for each of the other ROIs using the Pearson correlation coefficient. Decoding accuracies for ROIs were not significantly correlated with each other (p > .0161 for all, statistical significance accepted at p < .0018, Bonferroni-corrected for 28 comparisons).
To statistically compare decoding accuracies using cross-trial-type decoding with decoding accuracies using binary classification, two ANOVA analyses were performed, one for binary classification without RSVP and one for binary classification with RSVP. In both analyses, the two factors examined were Decoding method and ROIs. Decoding accuracies using cross-trial-type decoding were significantly lower than decoding accuracies using classification without RSVP (p < .0001) and with RSVP (p < .01). In addition, there was a significant difference for decoding accuracies between ROIs in the analysis for classification without RSVP (p < .0001) and with RSVP (p < .0001), but no significant interaction between Decoding method and ROIs in either analysis (p > .05 for both). To determine whether ROIs that significantly decoded task identity using cross-trial-type decoding had significantly different decoding accuracies using binary classification, post hoc two-tailed t tests were conducted for these ROIs. Decoding accuracies for L-IPS and R-IPS were not significantly different between cross-trial-type decoding and binary classification without RSVP (p > .05 for both L-IPS and R-IPS, Bonferroni-uncorrected). Similarly, decoding accuracies for R-FEF, L-IPS, and R-IPS were not significantly different between cross-trial-type decoding and binary classification with RSVP (p > .05 for all, Bonferroni-uncorrected).
In this study, we investigated the effects of attentional load on task processing using conventional univariate fMRI analyses and MVPA. When prosaccades and antisaccades were performed with an attention-demanding RSVP task, error rates and RTs increased, and activation in frontoparietal brain areas commonly associated with oculomotor control showed decreased activation compared to single-task performance. Performing the RSVP task eliminated mean activation differences between prosaccades and antisaccades in these brain areas but did not affect the ability to decode task identity using MVPA. In addition, multivoxel patterns of task representation were found to be similar using cross-trial-type MVPA, which suggests that patterns of activity are stable under attentional load. Taken together, these results suggest that attentional load may disrupt the strength of task-related activity, rather than the identity of task representations.
Previous studies using dual tasks with univariate fMRI have shown that attentional load increases activation in brain areas that are involved in single-task performance (Nijboer et al., 2014; Mizuno et al., 2012; Erickson et al., 2005; Adcock et al., 2000; Bunge et al., 2000; Klingberg, 2000). These results may reflect competition for limited resources in a particular brain area. Activation may increase if both tasks rely on the same neural population or if the tasks rely on different but adjacent populations that inhibit each other (Klingberg, 2000). Recently, Watanabe and Funahashi (2014) found that the same population of lateral pFC neurons was recruited for a spatial memory task and a spatial attention task, and that neural responses were attenuated when these tasks were performed concurrently. Thus, increased BOLD activation in the presence of an attentional load may reflect inhibitory processing.
In contrast to these studies, we found that attentional load decreased activation in task-related brain areas. Although performing an RSVP task during the preparatory period for prosaccades and antisaccades may not resemble a traditional dual task where task responses are processed and performed simultaneously, our results are consistent with another set of dual task experiments (Nijboer et al., 2014; Mizuno et al., 2012; Just et al., 2001, 2008; Newman et al., 2007; Klingberg, 1998). The discrepancy in changes in activation as a result of attentional load may be related to the tasks used. Increases in activation were observed when concurrent tasks involved similar stimuli, processing, and responses. For example, Erickson et al. (2005) asked participants to distinguish visual stimuli based on color or letter and respond with a button press. In contrast, decreases in activation were observed with dissimilar tasks (Nijboer et al., 2014; Mizuno et al., 2012; Just et al., 2001, 2008; Newman et al., 2007; Klingberg, 1998). Dissimilar tasks may activate distinct networks of brain areas or distinct processing streams within the same area. In this study, the generation of prosaccades and antisaccades requires a frontoparietal oculomotor network (Ford et al., 2005; DeSouza et al., 2003), whereas target detection in the RSVP task involves the right TPJ (Shulman, Astafiev, McAvoy, d'Avossa, & Corbetta, 2007; Corbetta, Kincade, Ollinger, McAvoy, & Shulman, 2000). Although RSVP tasks have been shown to activate frontoparietal brain areas (Han & Marois, 2014), this activation and processing may be driven by visual attention and be distinct from the activation and processing associated with saccade preparation.
Although we did not identify brain areas related to the RSVP task, decreased activation in frontoparietal brain areas may reflect the division of resources between two distinct functional processing streams. Decreased activation may be explained by a limit on global attentional resources available at any point in time (Just et al., 2008; Newman et al., 2007) or by the need to share time between tasks, which imposes a local rather than global limit on resources (Nijboer et al., 2014). Importantly, dual-task performance can simultaneously increase and decrease activation in different brain areas, depending on how the areas are involved in the tasks performed (Nijboer et al., 2014; Mizuno et al., 2012). Thus, changes in activation may reflect different types of processing rather than the amount of processing in a brain area.
In addition to examining changes in activation due to attentional load, investigating the ability to represent tasks can provide valuable information about task processing. When prosaccades and antisaccades are performed alone, they consistently activate an oculomotor network that includes the FEF, SEF, and IPS, and the task performed can be differentiated based on mean activation in these areas, with activation greater for antisaccades than prosaccades (Brown et al., 2006, 2007; Ford et al., 2005; DeSouza et al., 2003; Kimmig et al., 2001). These cortical activation differences arise predominately from the preparatory period, rather than the stimulus–response period (Ford et al., 2005; DeSouza et al., 2003), and are thought to reflect task-specific neural processing. FEF and SEF neurons begin to show task-related differences in firing rate between prosaccades and antisaccades after the instruction is presented (Everling & Munoz, 2000; Schlag-Rey, Amador, Sanchez, & Schlag, 1997), and single neuron activity in the IPS suggests a role in appropriately mapping the saccade to the stimulus (Zhang & Barash, 2000; Gottlieb & Goldberg, 1999). Consistent with previous studies, we found that activation in the IPS was significantly greater for antisaccades than prosaccades and a trend for activation to be higher for antisaccades in the FEF and SEF. Notably, the presence of an attentional load during the preparatory period eliminated differences in mean activation between prosaccades and antisaccades. Although an attentional load changes the ability to differentiate tasks based on mean activation, the nature of the content of activation is difficult to determine.
MVPA, on the other hand, increases the sensitivity of neuroimaging and is better suited to investigating how perceptual, cognitive, and motor information is represented in brain activity. Previous MVPA studies have demonstrated that task representations can be decoded from frontoparietal brain areas (Woolgar, Thompson, Bor, & Duncan, 2011; Bode & Haynes, 2009). In particular, Woolgar et al. (2011) investigated different task features that underlie task decoding in frontoparietal cortex and found that stimulus–response mapping was the most strongly represented. Accordingly, we found that saccade task identity was significantly decoded in the L-FEF, L-IPS, and R-IPS, with trends in the remaining ROIs. Although differences in mean activation between prosaccades and antisaccades, irrespective of patterns of activation, could have contributed to MVPA decoding, this was unlikely because differences in mean activation were not significantly correlated with decoding accuracy and mean-centered classifiers were used. In addition, task identity could still be decoded from frontoparietal cortex using MVPA in the presence of an attentional load despite the elimination of mean activation differences between prosaccades and antisaccades. This suggests that MVPA task discrimination results from task-specific neural processing, rather than general processing that may reflect different degrees of cognitive control or task difficulty. Overall, task identity can be predicted by patterns of brain activity, and these representations are maintained in the presence of an attentional load.
Notably, task representations using spatial patterns of activation without attentional load were found to be similar to task representations when an attentional load was present using cross-trial-type decoding. Furthermore, in ROIs that significantly decoded task identity, cross-trial-type decoding accuracies were not significantly different from decoding accuracies using binary classification without and with attentional load. These results suggest that task representation is stable under attentional load. Although no other studies, to our knowledge, have investigated task representations under attentional load, research has been conducted with working memory representations. Like the identity of a maintained task representation, the identity of a single item maintained in visual STM can be decoded from cortical activity using MVPA (Emrich, Riggall, LaRocque, & Postle, 2013; Harrison & Tong, 2009). Recently, Emrich et al. (2013) showed that increasing the number of items maintained in STM decreased the decoding accuracy for item identity in sensory cortex. The contrast between task decoding and item decoding under attentional load can be explained by the tasks that we used. Unlike items in visual STM, our two concurrent tasks may have been represented in relatively distinct functional processing streams. Consequently, the neural representations for these two tasks did not interfere with each other.
Spatially distinct, task-specific responses of neural populations may underlie prosaccade and antisaccade representations detected in frontoparietal cortex by MVPA. In the FEF, MVPA decoding may be driven by neurons that discharge differently for prosaccades and antisaccades (Munoz & Everling, 2004; Everling & Munoz, 2000). Preparatory fixation-related neuron activity is enhanced and saccade-related neuron activity is reduced for antisaccades compared to prosaccades. Saccade-related neurons in the FEF contralateral to the peripheral stimulus are responsible for generating a prosaccade toward the stimulus, whereas saccade-related neurons in the ipsilateral FEF are responsible for generating an antisaccade away from the stimulus. Given the proximity of our functionally defined FEF ROIs to the inferior frontal junction and inferior frontal sulcus, MVPA decoding may also be driven by the encoding of rule representations (Woolgar et al., 2011; Brass, Derrfuss, Forstmann, & von Cramon, 2005). In the IPS, MVPA decoding may reflect differences in how neurons encode stimulus-saccade mapping for prosaccades and antisaccades (Zhang & Barash, 2000) and rules (Stoet & Snyder, 2004). However, task-specific responses of neurons may not be sufficient for MVPA decoding in the SEF. The SEF is thought to be less directly involved in generating prosaccades and antisaccades than the FEF, and increased neural activity and BOLD activation for antisaccades compared to prosaccades may reflect increased cognitive control (Stuphorn & Schall, 2006) rather than rule representation. Nonetheless, our results suggest that task-specific responses of distinct neural populations in the FEF and IPS may be unaffected by attentional load.
Instead, attentional load may interfere with other aspects of task processing, as indicated by the decreases observed in mean activation. BOLD activation may reflect changes in the magnitude of neural outputs. Watanabe and Funahashi (2014) showed that dual-task performance decreased the task-related neural responses. How this affects BOLD activation and whether this can be generalized to concurrent tasks that involve two distinct functional processing streams is unknown. Alternatively, inputs to task-related brain areas may be disrupted. In the FEF, both decreases in the magnitude of task-specific signals and a lack of input may prevent neural activity from reaching thresholds required to generate correct and timely prosaccades and antisaccades (Munoz & Everling, 2004). Although our block design study could not examine the relationship between trial-by-trial performance and activation changes, previous studies have found that lower activation during the preparatory period in the FEF, SEF, IPS, dorsolateral pFC, and ACC are associated with erroneous compared to correct antisaccades (Ford et al., 2005; Curtis & D'Esposito, 2003). Thus, changes in mean activation and the strength of neural activity, rather than changes in task representation, may explain the increases in error rates and RTs observed for prosaccades and antisaccades during dual-task performance (Chan & DeSouza, 2013; Stuyven, Van der Goten, Vandierendonck, Claeys, & Crevits, 2000).
Unfortunately, the effects of attentional load on mean activation may have been confounded by fatigue or practice over time because the order of runs with and without RSVP was not counterbalanced between participants. Although the effects of fatigue on BOLD activation over time are unclear, practice is associated with increased neural efficiency and decreased activation in frontoparietal brain areas (Kelly & Garavan, 2005). However, we found that runs without RSVP at the end of a session had greater activation than runs with RSVP at the beginning. This is inconsistent with the effects of practice, but consistent with dual task experiments (Nijboer et al., 2014; Mizuno et al., 2012; Just et al., 2001, 2008; Newman et al., 2007; Klingberg, 1998). Alternatively, greater activation without RSVP may have been influenced by participants actively ignoring the RSVP stimuli. Nonetheless, our behavioral results suggest that performing prosaccades and antisaccades without RSVP was associated with a decrease in attentional load and changes in attentional load contribute to changes in mean activation.
A limitation of the block design of this experiment was the inability to examine BOLD activation on separate trials or during specific periods of a trial. As a result, the relative contributions of the preparatory period and stimulus–response period to task representations revealed by MVPA remain unclear. Differences in mean activation between prosaccades and antisaccades suggest that both the preparatory period (Ford et al., 2005; DeSouza et al., 2003) and stimulus–response period (Brown et al., 2006; Curtis & D'Esposito, 2003) can influence task decoding, although some studies have attributed differences in the stimulus–response period to carryover from the preparatory period. Preparatory period contributions to task decoding in this study may be partly driven by color because colored cues were used to indicate the current task and alleviate working memory load during blocks of trials. However, rule representations are more strongly encoded than color in frontoparietal cortex (Woolgar et al., 2011) and may primarily drive preparatory period task decoding. Studies using event-related designs will be required to elucidate the time course of MVPA task decoding over the duration of a trial.
In conclusion, we investigated how limited neural resources are distributed for task processing. The use of MVPA with conventional univariate analysis revealed that attentional load affects BOLD activation and patterns of activation differently in task-related brain areas. Limited attentional resources may constrain the strength of task-related neural activity while leaving the identity of task representations unaffected.
We thank B. Coe and S. David for technical support. This research was supported by funding from the York University, Faculty of Health, and the Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery Grant to J. F. X. D. J. L. C. was supported by an NSERC USRA and a Canadian Institutes of Health Research (CIHR) MD/PhD studentship. A. K. was supported by a CIHR doctoral research award.
Reprint requests should be sent to Jason L. Chan, Robarts Research Institute, University of Western Ontario, 100 Perth Drive, London, Ontario, Canada N6A 5K8, or via e-mail: firstname.lastname@example.org.