Areas in frontoparietal cortex have been shown to be active in a range of cognitive tasks and have been proposed to play a key role in goal-driven activities (Dosenbach, N. U. F., Fair, D. A., Miezin, F. M., Cohen, A. L., Wenger, K. K., Dosenbach, R. A. T., et al. Distinct brain networks for adaptive and stable task control in humans. Proceedings of the National Academy of Sciences, U.S.A., 104, 11073–11078, 2007; Duncan, J. The multiple-demand (MD) system of the primate brain: Mental programs for intelligent behavior. Trends in Cognitive Sciences, 14, 172–179, 2010). Here, we examine the role this frontoparietal system plays in visual search. Visual search, like many complex tasks, consists of a sequence of operations: target selection, stimulus–response (SR) mapping, and response execution. We independently manipulated the difficulty of target selection and SR mapping in a novel visual search task that involved identical stimulus displays. Enhanced activity was observed in areas of frontal and parietal cortex during both difficult target selection and SR mapping. In addition, anterior insula and ACC showed preferential representation of SR-stage information, whereas the medial frontal gyrus, precuneus, and inferior parietal sulcus showed preferential representation of target selection-stage information. A connectivity analysis revealed dissociable neural circuits underlying visual search. We hypothesize that these circuits regulate distinct mental operations associated with the allocation of spatial attention, stimulus decisions, shifts of task set from selection to SR mapping, and SR mapping. Taken together, the results show frontoparietal involvement in all stages of visual search and a specialization with respect to cognitive operations.
One of the striking properties of the human brain is its ability to flexibly implement goals that fit a specific context, irrespective of the exact stimuli. For example, we can judge the size of an object or its color depending on instructions or need. Alternatively, we can search among a set of items for our keys, a folder, or a favorite pen, again depending on our needs. We can even set abstract rules, such as picking out the largest object, or the object different from the rest, and do this regardless of the exact objects. The same flexibility and context specificity apply to our actions. On finding our keys, we might just grab them with our dominant hand, but we might use our nondominant hand or simply tell a friend where they are depending on the circumstances.
The brain mechanisms that underlie this flexibility have been associated with a network of regions in frontal and parietal cortex variously labeled as the multiple-demand (MD) region (Crittenden, Mitchell, & Duncan, 2016; Duncan, 2010), flexible cognitive hub (Cole et al., 2013; Cole & Schneider, 2007), task-set system (Nelson et al., 2010; Dosenbach et al., 2006, 2007), and task-activation cortex (Hampshire & Sharp, 2015; Seeley et al., 2007). We will concentrate principally on MD and potentially larger activation across frontoparietal cortex. MD encompasses frontal regions, including mid-dorsolateral, ventrolateral, and dorsal anterior cingulate regions, as well as regions in parietal cortex, notably the inferior parietal sulcus (IPS). MD has been shown to play a central role in goal attainment across a range of domains (Crittenden et al., 2016; Hampshire & Sharp, 2015; Cole et al., 2013; Fedorenko, Duncan, & Kanwisher, 2013; Duncan, 2010; Cole & Schneider, 2007; Seeley et al., 2007). Current evidence is somewhat conflicting as to whether MD is functionally homogeneous. Some studies find widespread activation throughout this region regardless of specific cognitive demands (e.g., Woolgar, Afshar, Williams, & Rich, 2015). In the main, though, evidence favors a functional specialization within MD, such that regions respond more to some manipulations than others (Shenhav, Botvinick, & Cohen, 2013). For example, Crittenden et al. (2016) manipulated the complexity of response rules that had to be learned and found that one subset of MD, including regions in dorsolateral prefrontal cortex (dlPFC), inferior frontal sulcus (IFS), and IPS, showed strong representations of task rules. Another subset of MD regions including the anterior insula (AI), anterior pFC, and dorsal ACC showed weaker rule representation. The results of Crittenden et al. (2016) suggest that patterns of activity within MD reflect a highly interconnected network of functionally separate brain areas whose activity depends on the exchange of information specific to the demands of a given task (see also Molinari et al., 2013). Moreover, localized regions of frontal cortex have been shown to be preferentially active in manipulations of response complexity, notably bilateral activity in dlPFC, dorsolateral premotor cortex, ACC, superior parietal and inferior parietal (IP) cortex, and precuneus (e.g., Schumacher, Cole, & D'Esposito, 2007; Jiang & Kanwisher, 2003; Schumacher, Elston, & D'Esposito, 2003).
Our interest here is in the role of MD and, more broadly, frontoparietal cortex in visual search. Visual search is the process of finding a predefined target embedded in a set of irrelevant distractor items. In daily life, it could be looking for one's keys among papers, phones, folders, and loose change on the desk; spotting threats in airport baggage screening; or locating tumors in medical images. Visual search can be functionally decomposed into a target selection stage, followed by a stimulus–response (SR) mapping stage. Each stage can be further decomposed into more elementary cognitive operations. During target selection, objects are attended to in sequence and compared with the properties that define the target, such as a particular feature (e.g., red), a conjunction of features (e.g., red circles), or a semantic attribute (e.g., animals). Thus, in the target selection stage, shifts of spatial attention alternate with stimulus decision-making until an item matching the target specification is found. Once found, the task rules that define the target must be replaced by those that dictate the response, so that the SR mapping stage can be completed. Such switching requires mental operations associated with executive control. In addition, the SR stage itself requires executive control in the application of response rules along with preparation and execution of the motor response. Visual search then has potential for the involvement of frontoparietal cortex in the application of two separate sets of task rules to complete the task—rules that specify what distinguishes targets from distractors, along with a second set that defines the response (e.g., find the blue diagonal line and then press the left button if it is angled left or the right button otherwise). Because both stages share a common dependency on task-specific rules, stimulus decisions in target selection may share neural resources with response decisions in SR mapping. However, the application of task rules differs in the two stages. A stimulus decision involves matching target specification against an internal representation of an attended object, whereas SR mapping rules specify a motor response. Object representation and response planning likely invoke separate neural systems. Thus, it may be possible to observe localized activity in MD, and frontoparietal cortex more broadly, attributable to only one of these stages, both stages, or localized preferences for some operations over others. Here, we investigate this issue by independently varying the difficulty of target detection and SR mapping.
As noted above, the target selection stage consists of repeated comparisons of attended objects to target specifications followed by voluntary shifts of attention for nonmatches. Consistent with shifts of spatial attention, imaging studies of visual search (Makino, Yokosawa, Takeda, & Kumada, 2004; Müller et al., 2003; Nobre, Coull, Walsh, & Frith, 2003) typically find activity in frontoparietal areas associated with the dorsal attention network (DAN), which underlies the endogenous orienting of spatial attention (Corbetta & Shulman, 2001). For example, Nobre et al. (2003) compared brain activity when participants searched for targets that differed by a single feature or a feature conjunction, both during efficient and inefficient search. Search is efficient when search times are constant with increasing number of distractors but inefficient when times increase. The main effect of search efficiency revealed activity in the right (R) and left (L) intraparietal sulci, R dlPFC, and R and L ventrolateral pFC. Similar results have been found in studies using cues to direct spatial attention in advance to better isolate activity associated with spatial attention (Makino et al., 2004; Müller et al., 2003). Although a relatively consistent pattern of activity in portions of the DAN has been observed for attention shifts during visual search, the existing literature is less clear about brain activity underlying other mental operations in target selection, such as stimulus comparison, or response processing. Activity in portions of frontal cortex not associated with response mapping, especially L pFC, has been attributed to target identification (Müller et al., 2003). A firm association between regions within MD and specific control actions in visual search has yet to be established owing, in part, to the difficulty of inference from different paradigms and manipulations. This study independently manipulates target selection and SR mapping difficulty using the same stimulus displays within the same paradigm to provide a clearer picture of how task rules operate at two functional levels of visual search. This should enable a clearer picture of specialization within the MD system and the role of frontoparietal cortex in visual search.
Participants in the present experiment were presented with search arrays and had to select two targets (easy or difficult), apply an SR mapping rule (easy or difficult), and execute a response. Easy target detection consisted of a “pop-out” search, in which targets were two uniquely colored items against distractor items in a different, uniform color (see Figure 1). The difficult detection required selection of one of the uniquely colored items along with an item in the background color, ignoring the more salient but irrelevant uniquely colored item. Easy SR mapping consisted of a single-component mapping rule, whereas difficult mapping consisted of two components. We used multivoxel pattern analysis (MVPA) because of its sensitivity in discriminating between experimental conditions (Norman, Polyn, Detre, & Haxby, 2006) and employed a well-established functional connectivity analysis to assess MD connectivity changes (O'Reilly, Woolrich, Behrens, Smith, & Johansen-Berg, 2012; Friston et al., 1997). We also employed a whole-brain univariate analysis to assess directional amplitude involvement of non-MD regions in selection and SR mapping. Because the search task we developed allowed for independently varying demands on target selection and SR mapping under identical visual input, differences in brain activity could arise only from differences in the complexity of target selection or SR mapping and not from differences in the visual displays themselves.
Twenty-three right-handed participants (16 women; age: mean = 24 years, SD = 3.2 years) were recruited for the study. They provided informed written consent and were reimbursed for their time ($20/1.5 hr). Of these 23 individuals, data from two participants were excluded because of excessive head movement (>10-mm translation and/or 6° rotation), and data from another three participants were excluded because their accuracy in the behavioral task was below our a-priori criterion of 70% correct. The study was approved by The University of Queensland Human research ethics committee.
Experimental Task and Procedure
Participants were scanned while performing a visual search task that included three operations: target selection, SR mapping, and response execution (Figure 1A). On each trial, participants were required to (1) select two target stimuli from a visual search array, (2) apply the correct SR rule to map the targets onto the correct response, and (3) execute a motor response via an appropriate keypress. The task was organized in mini blocks of four trials. At the start of each mini block, target selection and SR instructions were displayed on the screen for 6 sec, followed by a sequence of four trials consisting of fixation for 1 sec, search display for 1.5 sec, and a blank screen for 1.5 sec. Responses were recorded from the onset of the search display until the offset of the blank screen (3 sec). Critically, difficulty of target selection and SR mapping were independently manipulated while visual input was identical across conditions.
Apparatus and Stimuli
The task was created using the Psychophysics Toolbox (Version 3.0.8) (Brainard, 1997; Pelli, 1997) for MATLAB (MathWorks, release 2012b). Responses were recorded with an MR-compatible fiber-optic two-button response box, and stimuli were projected onto a screen at the top end of the scanner bore that was viewed through a mirror mounted to the head coil.
We created 96 search displays that consisted of nine bars arranged in a 3 × 3 grid and subtended a visual angle of ∼3° (see Figure 1A and B). Six bars in each display were tilted at a 45° angle (three leftward and rightward each) and all colored green or blue with equal probability (distractors). The remaining three bars in each display were oriented horizontally or vertically (equiprobably1) and colored red (RGB values = 247, 0, 0), green (24, 145, 0), and blue (0, 50, 254). Across search displays, color, orientation, and location were controlled. Stimuli were rendered on a light gray background.
Task difficulty was manipulated independently for target selection and SR mapping, resulting in easy versus difficult conditions. Targets in the easy target selection conditions were two uniquely colored items that popped out of the display because of their distinct color (e.g., red and blue among green distractors; see Figure 1A and B, top row). Targets in the difficult selection condition consisted of one of the pop-out items (red) along with a distractor-colored item of the specified orientation. Conditions of difficult search, thus, required effortful search and suppression of the salient distractor (see red and green targets in Figure 1B, middle row). For SR mapping, the easy conditions required only that participants press the “Yes” key if the two uniquely colored bars had the same orientation or the “No” key if they were oriented differently. In the difficult SR mapping condition, responses were contingent on both color and orientation; participants pressed the “Yes” key if the display contained either a red horizontal pop-out colored bar and a green vertical pop-out bar or a red vertical pop-out colored bar and a blue horizontal pop-out colored bar (and otherwise the “No” key). We included a control condition in which participants had to determine whether a central bar was tilted leftward or rightward (in identical displays). Participants were tested in four conditions: (1) Control, (2) Easy (Easy Selection/Easy SR), (3) Selection (Difficult Selection/Easy SR), and (4) SR (Easy Selection/Difficult SR). We did not test the Difficult Selection/Difficult SR, as MD activity would likely be similar to that of the SR condition, and we were not interested in how the difficulty manipulations combined (additively or otherwise). Instead, our analyses focused on activations in the Selection and SR conditions. For similar reasons, the MVPA assessed the regions of frontoparietal cortex by contrasting SR and Selection against the Control condition, not the Easy condition. As noted above, MD activity is widespread across tasks. The Easy conditions would, therefore, likely provide a noisy background against which to detect unique contributions of frontoparietal locations that distinguish SR and Selection.
The four conditions were interleaved using mini blocks (of 22 sec duration each). Sixteen blocks (four of each condition) were interleaved according to a Latin-square procedure in each run for a total of six runs (Figure 1C). Each visual search display was shown once per condition, and display order was determined by a random permutation of all displays. The total number of trials was 384, and the scanning session lasted about 50 min. Before transferring into the scanner, participants completed a training block of 96 trials (24 of each condition; 15 min).2
Anatomical and functional images were acquired using a 3-T Siemens Trio MRI scanner and a 32-channel head coil. Functional T2*-weighted images were acquired parallel to the AC–PC plane using a gradient-recalled echo EPI sequence (repetition time = 1.8 sec, echo time = 30 msec, flip angle = 80°, field of view = 192 × 192, matrix = 64 × 64, in-plane resolution = 3 × 3 mm). Each volume consisted of 29 slices with a thickness of 3 mm and a 0.3-mm interslice gap. At the start of the session, we collected a T1-weighted anatomical image using a magnetization prepared rapid gradient echo sequence (repetition time = 1.9 sec, echo time = 2.32 msec, flip angle = 9°, field of view = 230 × 230 × 230, resolution = 0.9 mm3). Each run consisted of 196 volumes. The first three images of each run were discarded to account for T1 equilibration effects.
Images were preprocessed and analyzed with Brain Voyager (Version 20.2; Brain Innovation) and custom-written MATLAB code using the BVQX Toolbox v08d. Images were slice-scan time corrected, realigned to the first image, and coregistered with the structural images. Finally, data were normalized to the standard Montreal Neurological Institute template. For the univariate analyses and connectivity (psychophysiological interaction [PPI]) analyses, data were smoothed with an 8-mm FWHM Gaussian kernel. For the MVPA, unsmoothed data were used.
To minimize any theory-driven bias in ROI selection, a single run from each participant's data was selected (equiprobably from Runs 1 to 6) to compute a functional localizer contrast (this run was excluded from subsequent MVPA and connectivity analyses). The localizer contrast showed voxel clusters (minimum size of 10 voxels) where BOLD activity averaged over the SR, selection, and intermediate conditions exceeded that in the control condition (false discovery rate [FDR] q = .01). The contrast yielded 12 activation clusters, all in frontal and parietal cortices. We computed the center of gravity coordinates for each cluster (see Table 1) and created a spherical ROI with a radius of 6 mm around each.3
|ROI Name .||ROI Coordinates .||t Value (FDR-corrected) .||Duncan (2010) .||Dosenbach et al. (2007) .|
|x .||y .||z .||x .||y .||z .||x .||y .||z .|
|L IFS (dlPFC)a||−46||32||29||4.474||−41||23||29||−43||22||34|
|R IFS (dlPFC)a||42||39||22||4.595||41||23||29||43||22||34|
|ROI Name .||ROI Coordinates .||t Value (FDR-corrected) .||Duncan (2010) .||Dosenbach et al. (2007) .|
|x .||y .||z .||x .||y .||z .||x .||y .||z .|
|L IFS (dlPFC)a||−46||32||29||4.474||−41||23||29||−43||22||34|
|R IFS (dlPFC)a||42||39||22||4.595||41||23||29||43||22||34|
AI/FO = anterior insula and adjacent frontal operculum; ACC = anterior cingulate; MFG = medial frontal gyrus; IPS = intraparietal sulcus; lFOG = lateral frontal-orbital gyrus; IFS = inferior frontal sulcus; dlPFC = dorsolateral prefrontal cortex; PrG = precentral gyrus; pre SMA = presupplementary motor area.
ROIs that were identified in Duncan (2010) but that failed to reach significance in our functional localizer.
Most identified ROIs were similar to those identified in previous studies (Wen, Mitchell, & Duncan, 2018; Duncan, 2010; Dosenbach et al., 2007; Jiang & Kanwisher, 2003). Table 1 lists the coordinates of ROIs and indicates those corresponding to Duncan (2010) and Dosenbach et al. (2007). We included an additional four ROIs based on MD coordinates reported previously (Duncan, 2010) that were not covered by the 12 ROIs we identified with the functional localizer. Together, these 16 ROIs covered the MD regions that were studied in previous research, and our procedure ensured that other regions with significant task activation were not excluded on a-priori theoretical grounds.
In line with previous studies (Woolgar et al., 2015), we used MVPA to assess condition differences in the pattern of activity across voxels because of its sensitivity in discriminating between experimental conditions (Norman et al., 2006). Subsequently, we employed a well-established functional connectivity analysis (O'Reilly et al., 2012; Friston et al., 1997) to assess whether multivoxel activity pattern differences across the Selection and SR manipulations were accompanied by a differential pattern of connectivity between MD regions. Finally, we conducted a whole-brain univariate analysis to assess the directional amplitude of the effects and involvement of non-MD regions in Selection and SR mapping.
MVPA was performed using BrainVoyager (version 20.2; Brain Innovation) to assess whether voxel activity patterns for different levels of complexity in the target selection and SR mapping stages were distinguishable. In a first step, general linear model (GLM) analyses were performed on each individual's data (five runs), with separate regressors for each condition. Each regressor was modeled as a rectangular function from the onset of the first search display in each block to the end of the fourth trial and convolved with the canonical hemodynamic response function. We derived t values for each voxel in each ROI per mini block. Values were z scored across all voxels to reduce any impact of small differences in univariate activity between Selection and SR conditions on MVPA results.
The pattern discrimination between tasks was then estimated using pairwise classification between SR and Control, and Selection and Control, for each participant. We used a support vector machine with a linear kernel and optimized cost parameter C in a level-split procedure. The classifier was trained on four runs and tested on a different run. This procedure was repeated five times on a leave-one-out basis (fivefold cross-validation) such that each run was used for testing once. Classification accuracies for each ROI were averaged across splits and participants to yield a single classification accuracy.
To assess whether classification accuracy was significantly higher than chance, we ran 200 permutation tests per split with condition labels randomly shuffled and identified the 95th percentile of this distribution. We then assessed whether pattern discrimination with intact labels exceeded the observed 95th percentile of this distribution. Finally, classification accuracy for SR and Selection conditions was compared using paired, two-sided t tests.
Connectivity analyses were performed using a standard PPI analysis (Friston et al., 1997) for the contrast SR versus Selection to compare the patterns of activity in MD to SR and selection difficulty manipulations (using the PPI plug-in for BrainVoyager). The source regions in the preprocessed and smoothed data were assessed for ROIs that showed a difference in classification accuracy in the MVPA for SR and Selection. The potential receiving voxels were restricted to the ROIs introduced in Table 1. Connectivity strength was calculated across mini blocks.
The unique explanatory power of the PPI was assessed by first extracting the time series from the first-level GLM for each source ROI and convolving it with the vector representing the contrast of SR versus Selection. Subsequently, we ran a second-level GLM with all three of the task difficulty variables testing the unique explanatory power of the PPI. Changes in connectivity between SR and Selection are indicated when PPI regressor values differ significantly from zero (FDR q = .01, minimum cluster size: 20 voxels).
Whole-Brain Univariate Analyses
Finally, to test for activations uniquely associated with either target selection or SR mapping in other brain areas, we also conducted a standard whole-brain analysis with the contrasts SR minus Easy and Selection minus Easy (threshold of p-FDR < .01; minimum cluster size: 15 continuous voxels).
Mean RTs in the visual search task differed significantly across conditions, F(3, 17) = 152.02, p < .001, partial η2 = .96 (see Figure 2). Paired, two-tailed t tests showed that RTs in all conditions differed significantly from one another: RTs in the Control condition were faster than those in the Easy condition (Easy Selection/Easy SR), t(19) = 10.89, p < .001, which in turn were faster than those in the SR condition (Easy Selection/Difficult SR), t(19) = 9.78, p < .001, which in turn were faster than those in the Selection condition (Difficult Selection/Easy SR), t(19) = 3.10, p < .001. Mean error percentages are also shown in Figure 2 (black circles). There was no evidence of a speed–accuracy trade-off.
Most of the ROIs showed significant decoding of task difficulty (for SR vs. Control as well as Selection vs. Control) with the average classification accuracy of increased task difficulty over all ROIs being 75%. Figure 3 depicts the mean classification accuracy for SR versus Control (red bars) and Selection versus Control (green bars) and the 95th percentile of the null distribution (black line) that was used as the criterion for significance. All regions decoded both difficulty of target selection and difficulty of SR mapping, although some regions showed significantly better decoding for one condition or the other.
To compare classification accuracy for Selection and SR, we conducted a repeated-measures ANOVA with search stage (Selection vs. SR) and ROI as within-group factors. There was a significant effect of ROI, F(8, 9) = 4.642, p = .017, and a significant interaction between stage of processing and ROI, F(8, 9) = 4.199, p = .016. A series of follow-up t tests showed that one set of ROIs—specifically the AI/FO and ACC—showed comparatively better decoding for the task pair that differed in the SR dimension (see Figure 3; left cluster: AI/FO = 77% [SR] vs. 70% [Selection], p = .009; ACC = 74% [SR] vs. 67% [Selection], p = .013). Medial frontal gyrus (MFG), IPS, and the precuneus showed comparatively stronger decoding for Selection (Figure 3, central cluster): MFG region, 71% (SR) versus 77% (Selection), p = .010; and IPS, 74% (SR) versus 81% (Selection), p = .007. For the precuneus, classification accuracy was lower for SR than Selection, 76% (SR) versus 81% (Selection), p = .038. The remaining ROIs (IFS, PrG, pre-SMA) did not show a significant difference in decoding strength for SR and Selection (see Figure 3, right cluster; IFS: p = .595; PrG: p = .184; pre-SMA: p = .667; R pFC: p = .470). The only frontoparietal ROI that did not show significant decoding for either task pair was the lateral frontal–orbital gyrus.
To assess whether RT differences between conditions influenced the MVPA results, we regressed classification accuracy on RT for each participant and extracted the slopes (β) and intercepts (α). Critically, classification accuracy at intercept—that is, in the absence of any RT differences between conditions—was above chance level for SR and Selection (Figure 4A). Specifically, for the SR–Control and Selection–Control pairs, the classification accuracy at intercept was 64% and 63%, respectively. Two-tailed, Wilcoxon signed rank tests showed that these percentages were significantly greater than chance (ps < .05), thus confirming reliable task decoding across conditions in the absence of RT differences.
The regression slopes provided evidence that RT moderately, but significantly, modulated classification accuracy for target selection (Figure 4B). Specifically, for the SR–Control and Selection–Control pairs, the regression slopes were 0.014 and 0.013, respectively. The former did not differ significantly from zero (p = .103), whereas the latter did (p = .024). Because the mean RT difference between the two conditions of interest was around 100 msec, this suggests that RT differences might have inflated the difference in classification accuracy by ∼1.3%. Thus, RT differences between conditions cannot account for the large and reliable effects observed in our MVPAs.
The PPI analysis showed a change in functional connectivity between several ROIs for the SR and Selection conditions (see Figure 5). Specifically, connectivity between ACC and R MFG (1167 voxels significant, t(17) = 2.783, corrected p = .015) was stronger during SR and weaker during Selection. Similarly, connectivity between ACC and PrG (L PrG: 1309 voxels significant, t(17) = 3.075, corrected p = .008; R PrG: 1309 voxels significant, t(17) = 2.632, corrected p = .019) was stronger during SR and reduced during Selection. The opposite pattern, with relatively stronger functional connectivity during Selection and reduced connectivity during SR, was observed between the PrG and the R precuneus (1238 voxels, t(17) = −2.786, corrected p = .016).
Whole-Brain Univariate Analyses
For Selection (difficult target selection > easy), the whole-brain univariate analysis revealed four regions of gray matter activation (see Figure 6), centered bilaterally on the precuneus (left: −21, −55, 52 [522 voxels]; right: 18, −55, 52 [1906 voxels]; Brodmann's area 7) and the middle occipital gyri (left: −33, −88, 4 [3788 voxels]; right: 30, −91, 10 [7179 voxels]; Brodmann's area 19).
For SR (difficult SR > easy), the whole-brain univariate analysis revealed eight regions of gray matter activation (see Figure 7) in frontal, parietal, and occipital cortices. Frontal activation included the bilateral anterior MFG (left: −37, 59, −11 [12850 voxels]; right: 48, 50, −20 [12665 voxels]; Brodmann's areas 10/11), a more posterior portion of the left MFG (−36, 23, 31 [2340 voxels]; Brodmann's area 9), the middle frontal gyrus (6, 35, 43 [19952 voxels]; Brodmann's area 8), and the R claustrum (30, 23, −5 [1376 voxels]). Parietal and occipital activation included the bilateral IP lobule (left: −37, 64, 46 [2021 voxels]; right: 48, −55, 43 [9986 voxels]; Brodmann's areas 7/40) and the cuneus (−6, −70, 31 [6187 voxels]; Brodmann's area 7). Overall, the whole-brain analysis confirmed the pattern of results observed in previous studies, implicating parietal and occipital cortex in target selection as well as frontal and parietal regions in SR.
Here, we have shown that, during visual search, areas in frontal cortex lying within the region associated with MD flexibly represent information not only relevant to SR decisions but also to target selection. The results of the MVPA suggest that all MD regions represented Selection-related information when target selection was demanding and SR-mapping information when the SR stage of processing was demanding. Moreover, some MD regions showed relatively stronger representations for target selection; and others, for SR mapping. The systems underlying target selection appear to involve circuits that span frontal and parietal areas, whereas preferential activity for difficult SR mappings involved primarily frontal cortical areas. A demanding target selection stage was associated more strongly with activity within a cluster of regions consisting of the MFG and IPS, whereas a demanding SR mapping stage showed strong activity within a more ventral cluster consisting of the AI/FO and ACC. In addition, our connectivity analysis identified two separate circuits: one consisting of correlated activity between L and R PrG and the R precuneus and the other between L and R PrG, ACC, and R MFG. Importantly, our activation patterns cannot be attributed to changes in display characteristics as we held the visual search stimuli constant across conditions. Instead, they reflect differences in the nature of the demands imposed by selection difficulty and SR difficulty.
Overall, the patterns of activity observed in response to SR and selection difficulty correspond to the functional task analysis of visual search we presented in the Introduction and are broadly consistent with existing literature. To relate active regions to possible cognitive operations in visual search, we input coordinates of our ROIs into NeuroSynth (Yarkoni, Poldrack, Nichols, Van Essen, & Wager, 2011) to identify studies finding activity within 6 mm of the centroid of each ROI. For target selection difficulty, the univariate analysis in Figure 6 showed heightened activity in precuneus and occipital sites when targets were less salient. Occipital and parietal activity has been observed with shifts of spatial attention in visual search (Makino et al., 2004; Müller et al., 2003; Nobre et al., 2003) and forms part of the DAN, a neural orienting network underlying voluntary shifts of attention (e.g., Corbetta & Shulman, 2001). Heightened activity in the precuneus has been found with shifts of attention between object features (Becker, Grubert, & Dux, 2014; Nagahama et al., 1999, 2001), consistent with the cognitive operations needed to iteratively match attended items to target finding features. Activity in the precuneus has also been found in tasks requiring mental imagery or episodic memory retrieval (Bray, Almas, Arnold, Iaria, & MacQueen, 2015; Becker et al., 2014; for a review see, Cavanna, 2007; Cavanna & Trimble, 2006; Makino et al., 2004; Giesbrecht, Woldorff, Song, & Mangun, 2003). In the MVPA, activation in MFG and IPS led to better classification rates for target selection than SR mapping. Our MFG coordinates put it close to the FEFs, which are also part of the DAN (Corbetta & Shulman, 2001), just outside the confidence intervals given by Paus (1996) but well within the range of locations active during shifts of spatial attention (Jiang, 2004; Makino et al., 2004; Woldorff et al., 2004; Giesbrecht et al., 2003; Müller et al., 2003; Nobre et al., 2003). In summary, the function associated with active locations in the univariate, multivoxel pattern, and connectivity analyses corresponds to the cognitive operations associated with repeated shifts of attention and comparison of attended items to target specifications.
For SR mapping difficulty, the univariate analysis revealed a more anterior pattern of activation involving prefrontal sites and activity in supplementary motor cortex, regions associated with response processing (Molinari et al., 2013; Cavina-Pratesi et al., 2006; Boettiger & D'Esposito, 2005; Jiang, 2004; Jiang & Kanwisher, 2003; Johnson-Frey et al., 2003). The MVPA classification accuracy was higher for SR mapping than target selection in ACC and AI/FO. Regions in ACC and AI/FO have been strongly linked to executive and task control (Nelson et al., 2010; Dosenbach et al., 2006, 2007).
Perhaps, the most novel outcome of the experiment is the identification of dissociable circuits underlying SR mapping and target selection difficulty seen in the connectivity analysis. Although most ROIs within MD were active in decoding both selection and SR difficulty, the connectivity analysis tells a slightly more nuanced story by examining correlated patterns of firing between regions. Difficult target selection revealed highly correlated activity patterns between the precuneus and bilateral precentral gyrus (PrG). Difficult SR mapping revealed correlations involving bilateral PrG, this time with ACC. ACC also showed correlated activity with the L MFG. It is premature to draw strong conclusions about the role of these circuits in visual search, especially because each location has been associated with multiple functions in studies with varying results, stimulus materials, and paradigms. Nonetheless, when examined in light of previous studies, the connectivity does suggest that we are dealing with two separate control circuits, one for attention and stimulus decisions and the other for response planning, in which task rules are applied separately in the two stages of visual search.
Evidence for two separate control circuits involved in manipulating and applying task rules comes from studies showing that regions in close proximity to our PrG coordinates are active in tasks that require manipulation of task goals, especially the instantiation and shifting of task set. Bilateral PrG activity has been observed in studies that require a shift in task set, as for example, transitioning from one stimulus to another in dual-task paradigms (Jiang, 2004), learning SR mapping rules (Boettiger & D'Esposito, 2005), and task switching (Nagahama et al., 2001). These regions are also active when observing objects being manipulated (Johnson-Frey et al., 2003), suggesting that they participate in planning upcoming actions.
Although both L PrG and R PrG activity has been observed, the two appear to deal with different aspects of goal-driven behavior. Unilateral activity in R PrG has been observed in difficult working memory conditions (Bray et al., 2015) and shifts of attention (Dosenbach et al., 2006, 2007; Müller et al., 2003). Müller et al. (2003), for example, cued target location using circles around one, two, or four items in a search array. Participants were instructed to shift attention to the indicated region. The authors found transient activity in coordinates near our R PrG (x, y, z = 36, −3, 33) shortly after the cue was presented as well as immediately on presentation of the search array. No activity of L PrG was reported. Dosenbach et al. (2006) analyzed fMRI data from 10 tasks to identify activity within the default mode network (for a review, see Raichle, 2015) associated with the start of a trial, sustained throughout a trial, and in response to error feedback. They observed increased activity in a region close to the coordinates of our R PrG (x, y, z = 42, 2, 37) only in the start-cue condition. They argue that regions active in the start-cue condition reflect the instantiation of attentional goals. Activation in R PrG has also been shown to increase in the incongruent condition of the Stroop task (Pardo, Pardo, Janer, & Raichle, 1990; x, y, z = 46, 7, 21), reinforcing the idea that the R PrG is closely involved in goal-directed attention. Given this, it is reasonable to hypothesize that the connection between the precuneus and R PrG mediates attention shifts in target selection.
Unilateral activity in regions corresponding to our L PrG suggests a role in task set, but specifically verbal working memory. Hanakawa et al. (2002), for example, had participants engage in a verbal, numeric, or spatial incrementing task. They observed activity in an area corresponding to our L PrG for numeric (x, y, z = −56, 8, 46) and verbal (x, y, z = −56, 4, 42) addition tasks as well as a verbal rehearsal task (x, y, z = −64, 0, 40), but not for a spatial task. Giesbrecht et al. (2003) used letters to cue either location or color and found activity in areas corresponding to our L PrG (x, y, z = −49, −4, 30; −34, 8, 50; −45, 8, 35; and −45, 4, 30), but not R PrG. Molinari et al. (2013) found activity in areas corresponding to L PrG (x, y, z = −48, 6, 28; −49, 6, 30) when participants were instructed to actively monitor functional hand movements. Similar results were observed by Dinstein, Hasson, Rubin, and Heeger (2007) when participants were instructed to either imitate (x, y, z = −39, −2, 29) or simply observe (x, y, z = −45, −3, 32) motor actions. Interestingly, they also found that repetition of a motor response led to reduced L PrG activity (x, y, z = −43, 0, 26), leading them to argue that activity in this area was related to motor memory, which would also be consistent with planning upcoming task-appropriate actions. One possibility is that the connection between the precuneus and L PrG mediates matching between an attended stimulus against a set of task rules specifying target properties. The match may even involve a verbal (phonetic) representation of task rules.
In the SR mapping condition, the connection of the ACC with both L PrG and R PrG appears to be part of an executive control circuit, involved in the transition from target detection to response mapping. The ACC has long been associated with cognitive control (see Bush, Luu, & Posner, 2000, for a review; Fellows & Farah, 2005) in Stroop (Pardo et al., 1990) and task switching (Johnston, Levin, Koval, & Everling, 2007; Liston, Matalon, Hare, Davidson, & Casey, 2006). Evidence suggests that ACC represents and maintains current goals at an abstract, task-independent level (Lopez-Garcia et al., 2016; Munakata et al., 2011), consistent with the proposal that the PrG and ACC together monitor progress toward an active goal (Benn et al., 2014). Other accounts, however, posit that control is not achieved through such strategic monitoring but through conflict monitoring (Shenhav, Cohen, & Botvinick, 2016; Shenhav et al., 2013; Botvinick, Cohen, & Carter, 2004; Carter et al., 2000). In either case, it is reasonable to posit that the ACC–PrG connection represents a cognitive control circuit that supports the transition from target selection to response planning.
It could be argued that, related to its executive control function, the ACC activity in our study reflects error monitoring. We cannot completely rule out this account. Our mini-block design does not allow us to eliminate error trials without eliminating the entire four-item mini block, which would result in an underpowered analysis. Nonetheless, aspects of the data strongly suggest that error monitoring cannot be the primary driver of ACC activity in the MVPA. Although the ACC does provide a weak predictive value in the Selection condition, it shows substantial preference for SR. Despite this, error rates in Selection, although a little less than in SR (black dots in Figure 2), are 10%–15% higher than in Control. If error rate differences were the primary driver, then we should have expected to see a roughly equivalent role for ACC in SR and Selection. In the univariate analysis, which is less sensitive than MVPA to error asymmetries, we see the same pattern: ACC activation in the SR, but not Selection, condition. The pattern suggests that both the MVPA and univariate analyses reflect functional specialization within ACC that goes beyond error monitoring. The functional specialization we find, moreover, is consistent with existing literature on the role of ACC in executive control more broadly, not simply error monitoring.
The connection between ACC and R MFG may, at first glance, seem odd in the context of the SR mapping circuit, given its position within accepted bounds for FEF (e.g., Giesbrecht et al., 2003; Jiang & Kanwisher, 2003). However, our R MFG is in the region of supplementary motor cortex shown to be active in conditions requiring incompatible or unusual response mapping (Bray et al., 2015; Jiang, 2004; Jiang & Kanwisher, 2003; Schumacher & Jiang, 2003), as well as spatial (but not verbal) working memory (Majerus et al., 2016; Giesbrecht et al., 2003; Hanakawa et al., 2002; Nagahama et al., 2001). Hanakawa et al. (2002), for example, found R MFG (x, y, z = 20, 0, 66) involvement only when participants mentally transitioned from one cell in a matrix to another, not for a numeric or verbal incrementing task. Similarly, Giesbrecht et al. (2003) presented a numeric cue that indicated either the location or color of a target. Activity in R MFG (x, y, z = 23, 0, 50) was observed only with the location cue. Although we cannot definitively rule out a role for the R MFG–ACC connectivity in shifts of spatial attention, in the context of our SR mapping difficulty, it is more likely to be associated with response processing.
Our target selection manipulation increased the perceptual difficulty involved in finding the target by keeping the color of one of the target elements identical to the surrounding items, effectively reducing its salience. In keeping with this, the univariate analysis found increased activity in occipital and parietal areas, but we also noted that this perceptual manipulation led to activity within MD regions in frontal cortex. What role, then, does the region in frontoparietal cortex identified as MD play in perceptual processing? Our view is that MD and other frontal sites are active only when the perceptual difficulty triggers task-level processing. By task-level processing, we refer to cognitive operations that can only be carried out in reference to the state of ongoing goal-directed processing, such as decisions about where to fixate next, whether a stimulus matches target specifications, or which response should be made. This is in line with claims that MD is associated with resource-limited processing, not data-limited processing (Wen et al., 2018; Han & Marois, 2013). In the dual-task literature, resource-limited processing refers to cognitive operations that benefit from the application of limited-capacity resources. In contrast, data-limited processing is associated with perceptual manipulations that affect stimulus quality and are insensitive to the application of additional resources (Pashler, 1984; Norman & Bobrow, 1975; Welford, 1952). Data-limited processing engages sensory processing, but not the integration of the sensory signal with task goals. A key feature of resource-limited processing is its serial nature; resource limitations make difficult any parallel processing of multiple stimuli even for simple tasks (Pashler, 1984, 1994; Pashler & Johnston, 1989). Such limitations are consistent with widespread activity seen in MD, and frontoparietal cortex in general, in response to our difficulty manipulation. Despite the specialization within MD, communication between multiple regions was evidently needed in our task.
Investigations of the role of MD in perceptual discrimination support the idea that it is associated with task-level operations. Wen et al. (2018) had participants judge brief periods of coherent random motion in random dot kinematograms. No MD activity was detected in conditions of reduced contrast or variations in motion coherence, although performance was poorer. In seeming contradiction to this finding, Jiang and Kanwisher (2003, Experiment 1) observed MD activity in a perceptually more demanding discrimination task, in which participants had to judge whether a discrepant line was shorter or longer than the other line segments. Although these two perceptual discriminations may seem at first glance similar, one could argue that the line discrimination task in Jiang and Kanwisher (2003) did not make it more difficult to perceptually resolve the stimuli but more difficult to judge whether the line was longer or shorter. Similarly, our salience manipulation in target selection not only made it more difficult to detect the target stimulus against the background but also increased the difficulty of target decision-making, involving task-level operations. Clearly, more examples are required before making strong generalizations about the kinds of manipulations that drive MD responses and those that do not.
The link between MD and resource-limited processing (Wen et al., 2018; Han & Marois, 2013) brings up the larger issue of whether MD may be the principal brain region associated with resource-demanding portions of task processing—what has been called “central processing” in the dual-task literature (Pashler, 1984; Welford, 1952). The central processor is assumed to flexibly apply task rules to incoming stimuli that describe what decisions are to be made and to map arbitrary responses to task-relevant aspects of the stimulus. The observed pattern of MD activity across a wide range of tasks is consistent with the kind of task and stimulus independence required of a central processor.
We have identified two separate circuits underlying target selection and SR mapping in visual search. These circuits interact with neural centers previously shown to be important for the manipulation and application of task-specific rules. The observation that both circuits communicate with the same regions of frontal cortex suggests that task rules for target selection and SR mapping are not represented in widely separate brain regions but draw on the same neural resources.
Roger William Remington: Conceptualization; Formal analysis; Funding acquisition; Investigation; Methodology; Project administration; Supervision; Writing - Original Draft. Joyce M. G. Vromen: Data curation; Formal analysis; Software. Stefanie I. Becker: Conceptualization; Data curation; Formal analysis; Methodology; Software; Supervision; Writing - Review & Editing. Jason Mattingley: Formal analysis; Funding acquisition; Investigation; Methodology; Project administration; Resources; Supervision; Writing - Review & Editing.
This work was supported by an Australian Research Council (ARC) Discovery Grant awarded to R. W. R. and S. I. B. (DP120103721). S. I. B. was supported by an ARC Discovery Grant (DP170102559). O. B. was supported by a grant from the National Health and Medical Research Council (APP1098862). J. B. M. was supported by an ARC Laureate Fellowship (FL110100103) and by the ARC Centre of Excellence for Integrative Brain Function (Centre Grant CE140100007).
Reprint requests should be sent to Roger W. Remington, Department of Psychology, University of Minnesota, 75 East River Road, S204 Elliott Hall, Minneapolis, MN 55455, or via e-mail: email@example.com.
With the restriction that they could not all three be of the same orientation.
At the end of each trial, accuracy feedback was provided. If a participant's accuracy rate was below 70% in any of the four conditions, participants had to complete the training block a second time. The only other divergence in procedure from the scanner block was that instructions were displayed for 12 sec.
Peak voxel coordinates for the single run localizer were representative of all six runs, with the distance between the peak coordinates observed for a single run and those observed when using all six runs being less than 3 voxels (x mean: 1.8, y mean: 2.6, z mean: 1.8).