Single-cell and fMRI experiments indicate that task-relevant features are enhanced globally across the visual field (VF). Moreover, this global feature-based attention can spread to task-irrelevant features of the attended object. Here we ask whether a task-irrelevant feature, by virtue of being bound to a task-relevant feature, can also be enhanced at a task-irrelevant location. Specifically, we asked whether attending to the color of moving dots in one VF would influence the motion signal to colored moving dots in the other VF. Participants attended to either red or cyan dots, superimposed and moving in opposite directions. Critically, the color and motion of dots present in the opposite VF varied as a function of the attended dots such that they were either the same color/same direction, same color/opposite direction, opposite color/same direction, or opposite color/opposite direction as the attended dots. We found greater activity in ventral visual cortex when either the color or direction of motion matched the color or direction of motion at the attended location. Similar effects were found for direction of motion in human medial temporal/medial superior temporal cortex. Moreover, the color and motion effects did not interact in any region. Together, these results suggest that the coselection of an object's features modulates those features independently beyond the selected object.
At its most basic level, attention serves to highlight important information in our visual world as defined by current goals. Depending on the nature of those task goals, attention can bias the perceptual system toward a special location (e.g., home plate), a specific feature (e.g., the color red), or multidimensional objects (e.g., a red, bouncing kickball). Unlike spatial attention, which enhances features at a specific location, feature-based attention appears to operate in parallel across the visual field (VF; e.g., White & Carrasco, 2011; Bichot, Rossi, & Desimone, 2005; Saenz, Buracas, & Boynton, 2002; Treue & Martinez-Trujillo, 1999); that is, the attended feature is enhanced relative to other features wherever it is encountered. Once a target object has been located, however, task-irrelevant features that are spatio-temporally bound to the attended feature appear to be coselected (e.g., Katzner, Busse, & Treue, 2009; Melcher, Papathomas, & Vidnyánszky, 2005; Sohn, Chong, Papathomas, & Vidnyanszky, 2005). We ask whether that coselection is more akin to spatial selection in which information at a single location is enhanced or whether the coselection relies on more feature-based mechanisms that enhance the task-irrelevant features independently beyond that object's location.
Converging evidence from single-cell, neuroimaging, and behavioral studies has revealed that feature-based attention operates in a global fashion. For instance, Treue and Martinez-Trujillo (1999) used single-cell measurements in macaque monkeys to show that features can be attentionally modulated independently of spatial attention. Using moving random dot patterns as stimuli, the authors showed that the neural tuning curves of cells in motion-sensitive medial-temporal (MT) area responding to a coherent direction were enhanced in a multiplicative manner when the monkeys attended to the preferred direction of those cells. Importantly, this attentional enhancement occurred to the same extent whether the monkeys directed their attention toward a stimulus falling inside or outside the receptive field. In other words, the attentional enhancement was not specific to the attended location. Additionally, when the monkeys attended to a stimulus outside the cell's receptive field that was moving in the opposite direction to the cell's preferred direction, the activity of the cell was suppressed relative to its response to a null direction stimulus. Together, these results indicate that features (as opposed to a spatial location) can guide behavioral selection (i.e., attention) through gain enhancement of cells' responses to relevant target features and suppression of cells' responses to opposing features. On the basis of these data, the authors proposed a “feature similarity gain model” to explain how the similarity between the given response sensitivity of neurons and behaviorally relevant features may drive feature-based attention. Importantly, this model allows for concurrent analysis of features that may not fall within the same receptive field, allowing for a more global, as opposed to spatially specific, attentional bias.
Similarly, human fMRI results have shown greater activity for attended as opposed to unattended features at task-irrelevant locations (e.g., Serences & Boynton, 2007; Saenz et al., 2002). Interestingly, Serences and Boynton (2007) found evidence of such enhancement even in the absence of visual stimulation in task-irrelevant locations. Using multivoxel pattern recognition, they were able to decode the attended direction of motion at a task-irrelevant location whether or not the task-irrelevant location contained a stimulus. These results support the suggestion that the feature-based enhancement seen in earlier studies reflects a top–down global enhancement rather than grouping-related enhancement among like stimuli. Finally, global feature-based enhancement has been shown to have behavioral consequences (White & Carrasco, 2011; Arman, Ciaramataro, & Boynton, 2006; Melcher et al., 2005). For instance, attending to a specific direction of motion in one location can modulate the duration of the motion aftereffect elicited by a similarly moving stimulus at a task-irrelevant location (Arman et al., 2006).
What happens once an object is selected based on a feature-based strategy? Objects are multidimensional, and although we may have used a single dimension to orient to them, ultimately we may select items as whole objects. In fact, a number of studies have shown that attending to an object results in coselection of task-irrelevant features spatio-temporally bound to the task-relevant feature (e.g., Schoenfeld et al., 2003; O'Craven, Downing, & Kanwisher, 1999; Duncan, Humphreys, & Ward, 1997; Duncan, 1984). For instance, using fMRI, O'Craven et al. (1999) found that attending to a moving object that happened to be a face or a house not only led to increased activity in the motion-sensitive MT+ cortex but also in the cortical region associated with the category of the object (i.e., fusiform face area when the moving object happened to be a face). Importantly, the object category was irrelevant to this particular task, suggesting that the coselection of the moving object's category happened automatically. Additionally, Schoenfeld et al. (2003) demonstrated that this spread of attention to task-irrelevant features occurs quickly enough to reflect the initial feature-binding process in object-based attention. These neural findings are consistent with behavioral work by Duncan (1984), showing that participants were able to judge two attributes of a briefly presented object as accurately as a single object attribute.
What are the mechanisms behind such coselection of features? Is the enhancement of task-irrelevant features specific to the attended location, or might those features be selected more globally? Several studies have found both neural (Boehler, Schoenfeld, Heinze, & Hopf, 2011; Katzner, Busse, & Treue, 2009) and behavioral (Arman et al., 2006) consequences of task-irrelevant dimensions beyond the location of the attended object, suggesting that task-irrelevant dimensions of attended objects are modulated globally. For instance, Katzner et al. (2009) showed that the response of MT cells in macaques to a spatially distant, task-irrelevant dot field increased when the dots moved in the same versus different direction as the attended dot field, although the monkeys were only rewarded for monitoring the color (a task-irrelevant dimension) of the attended dot field.
One unresolved issue is whether such global modulation of task-irrelevant dimensions is based on a distinct global object-based mechanism or whether it is a consequence of multiple feature-based mechanisms induced by the features (task-relevant or not) of the attended object. If there is a global “object-based” mechanism, then we should see that enhancement at a task-irrelevant location is specific to or at least greater for a conjunction of the task-relevant and task-irrelevant features. In other words, it is the object that is being boosted globally rather than the individual features per se.
Alternatively, because observers are presumably not actively searching for the object, the global boosting observed may instead be the result of two independent feature-based mechanisms, one for each of the features being coselected at the attended location. In other words, any enhancement seen at the task-irrelevant location may simply be a consequence of the fact that multiple features are being attended at the task-relevant location, and those feature mechanisms are necessarily global. For example, if an observer is attending to both color and motion in the attended object, two global enhancements, one for color and one for direction of motion, may be set up simultaneously but independently. In this case, the enhancement of the task-irrelevant dimension would not interact with the enhancement of the task-relevant dimension.
In the following experiment, we ask whether two features (color and motion) are modulated at task-irrelevant locations independently or in a conjoined fashion. If the effects of color and motion interact, such that the enhancement is greatest when both features are present, this would indicate that enhancement is occurring in a conjoined fashion consistent with the presence of a global object-based attention. However, if coselection of task-irrelevant dimensions relies on multiple but separate feature-based mechanisms, then the two dimensions should be modulated independently at a task-irrelevant location; that is, modulation of a task-irrelevant dimension should not be contingent on the presence of the relevant dimension in the task-irrelevant location.
More specifically, we asked whether attending to a single dimension of a moving random dot pattern (containing both color and coherent direction of motion) would modulate the neural response to a spatially distant, task-irrelevant dot field that shared either (i) both features (same color and direction), (ii) a single feature (either same color or same direction), or (iii) no features (opposite color and direction) with the attended stimulus. On the basis of previous work on global feature-based attention (e.g., Treue & Martinez-Trujillo, 1999), we predicted first that activity should be enhanced at the task-irrelevant location when the stimulus contains the attended feature (color). Second, based on studies showing global consequences of task-irrelevant dimensions (e.g., Katzner et al., 2009), we predicted that the irrelevant dimension (motion) bound to the attended color at the attended location would also be modulated at the task-irrelevant location. Importantly, in our design, we can also ask whether each feature is modulated independently at the task-irrelevant location or whether such modulation of the task-irrelevant dimension depends on the presence of the task-relevant feature. Specifically, if participants attend to the color of a moving dot field in one location, is the modulation of task-irrelevant motion direction in a spatially distant dot field greater when the ignored stimulus contains the same versus different color as the attended dot field, or is the amount of direction modulation the same regardless of the color of the ignored stimulus?
Finally, because we are scanning the whole visual brain, we can ask whether the spatially distant effects of color and motion are restricted to the ventral and dorsal stream, respectively. Previous work either only presented one of the features from the attended object at the task-irrelevant location (Boehler, Schoenfeld, et al., 2011; Katzner et al., 2009) or limited their analysis to a single area (e.g., MT, Katzner et al., 2009); thus, it is unclear how restricted these effects are. The integrated competition model of attention (Duncan, 2006; Duncan et al., 1997) predicts that when an object is selected, multiple systems will converge to select all features of the object. Thus, we predicted that any enhancement of task-relevant or task-irrelevant dimensions would be present in both the dorsal (MT+, V3a) and ventral visual streams (V1, V2, VP, and V4).
One author (A.L.) and eight naive participants (7 men; ages 25–32 years) participated in the experiment. All participants had normal or corrected-to-normal visual acuity and were in good health. Each participant gave informed written consent, which was approved by the institutional review board of the University of Illinois at Urbana-Champaign and were paid for participating.
The stimuli were presented via goggles (Magnetic Resonance Technologies, Willoughby, OH) with a refresh rate of 60 Hz and resolution of 800 × 600. The experiment was programmed using Matlab and the Psychophysics Toolbox (Brainard, 1997; Pelli, 1997) and was controlled by a Windows PC. During stimulus presentation, two square apertures (9° × 9°) appeared 6° to the left and right of and 1° above the center of the screen, where a gray fixation cross (0.88° × 0.88°) was presented. Each aperture contained 100 dots (radius = 0.08°); the right aperture (attended field) always contained 50 cyan and 50 red dots, whereas the dots in the left aperture (task-irrelevant location) were either all red or all cyan, depending on the condition. In the attended field, 100% of the attended dots (either red or cyan) moved in a coherent direction (either up, down, left or right) for 4 sec and then changed direction, rotating through the four possible directions in a random order. The initial direction was chosen randomly, and the order of directions was counterbalanced across all conditions. The ignored dots (either cyan or red) in the attended field always moved coherently in the opposite direction as that of the attended dots. Importantly, because the attended location always contained two surfaces, any modulation of visual cortex would be based on effects of feature-based attention and not on low-level visual differences between conditions. The dots in the task-irrelevant location either all moved in the same or opposite direction as the attended dots, depending on the condition (Figure 1). All dots in both apertures moved at a rate of 8.77°/sec.
At the beginning of each run, participants were instructed to detect a slight luminance change (dim) in either the red or cyan dots (target color specified before each run) in the attended field (right VF). The target color alternated each run, and the initial target color was counterbalanced across participants. Participants were instructed to maintain their gaze on the centrally presented gray cross present throughout the run. During experimental blocks (16 sec), participants were asked to attend to the aperture to the right of fixation while ignoring the left aperture. Experimental blocks were interspersed with 16-sec baseline fixation blocks, and each run began and ended with a 16-sec baseline fixation period. In the attended aperture only, dots in the target and nontarget color dimmed for 500 msec either zero, one, or two times (but never simultaneously) within each 16-sec block. When two dims occurred within the same block, they were separated by 4-, 8-, or 12-sec intervals. Participants responded to dims in the target color by pressing a single button on a button box. During each block, the dots in the left, task-irrelevant location contained dots of a single color moving in a coherent direction that were related to the target dots in four possible ways (as shown in Figure 1): the dots in the task-irrelevant location were the (1) same color, same direction (SColSDir), (2) the same color, opposite direction (SColODir), (3) the opposite color, same direction (OColSDir), or (4) the opposite color, opposite direction (OColODir) of the attended dots. Each of these four block conditions appeared three times during each run in a pseudorandomized order, such that the same condition never appeared twice in a row. There were a total of eight runs lasting 6 min and 40 sec, each containing 12 experimental blocks and 13 baseline blocks.
Following the eight experimental runs, we presented two localizer runs designed to identify the most spatially selective voxels responding to the task-relevant and task-irrelevant aperture locations. Participants viewed gray moving dots in either the left or right aperture (in the same stimulus locations as the experiment), with the location alternating between the left and right VFs every block (16 sec). The dots moved in a coherent direction at a speed of 8.77°/sec for 4 sec (up, down, left, or right), then rotated through the rest of the possible directions. Each run contained 10 blocks (5 blocks with the left aperture and 5 with the right aperture).
Retinotopic Mapping Procedure
To delineate functional maps of early visual cortex, we ran a separate retinotopic mapping session. Participants passively viewed flickering black and white checkerboard images that stimulated various regions of the VF. In half of the runs, the checkerboards were alternately presented in the upper and lower VFs in a blocked fashion, with each block lasting 20 sec. For the other half of the runs, the checkerboards were alternately presented in the horizontal and vertical meridians, again in a blocked fashion. We also collected multiple high-resolution T1-weighted anatomical images (MPRAGE) to create flat maps of visual cortex.
Data Acquisition and Analysis
Imaging data were acquired in a 3-T head-only scanner (Allegra; Siemens, Norwood, MA) using a standard head coil. For the experimental and localizer runs, we collected echo-planar images (EPIs) from the entire brain using a gradient-echo sequence (repetition time = 2000 msec, echo time = 20 msec, flip angle = 90°, image matrix = 64 × 64, field of view = 180 × 180 mm, slice thickness = 3 mm, no gap, voxel size = 2.8 × 2.8 × 3 mm) in 40 ascending transverse slices, obliquely oriented between coronal and axial orientations so as to provide maximum coverage of the brain. We collected 200 images for each of the eight experimental runs and 96 images for each of the two localizer runs. To assist in registering images to anatomical space, we collected high-resolution, T1-weighted MPRAGE anatomical images (repetition time = 2000 msec, echo time = 2.22 msec, flip angle = 8°, image matrix = 192 × 192, field of view = 240 × 240 mm, voxel size = 1.25 × 1.25 × 1.25 mm) in the sagittal plane. The functional data from the retinotopic mapping session were combined with ROIs from the localizer scan in the previous session and overlaid onto inflated and flattened anatomical images created with Freesurfer software (Fischl, Sereno, & Dale, 1999).
Data from the experiment, localizer and retinotopy sessions were analyzed using the AFNI software suite (Cox, 1996). The functional data from all sessions were motion-corrected, despiked, spatially smoothed (6 mm FWHM Gaussian blur), and normalized to the temporal mean of each run before being submitted to a regression analysis. Four square-wave regressors were constructed to model periods in which the dots in the task-irrelevant location were (1) the same color and direction, (2) the same color and opposite direction, (3) the opposite color and same direction, or (4) the opposite color and direction as the attended stimulus. These regressors were then convolved with a standard gamma function (default AFNI parameters: time-to-peak = 4.7 sec, FWHM = 3.8 sec) to model the hemodynamic response and submitted to a multiple regression in the framework of the general linear model. Additionally, we ran a separate general linear model for the localizer runs using two regressors, convolved by the same gamma function, that corresponded to stimulus presentations in the left and right VFs.
To test for differences among the four conditions in the task-irrelevant location, we first identified ROIs in visual cortex that responded more to task-irrelevant than task-relevant locations. Spatially selective voxels for the task-irrelevant location were identified by contrasting blocks in which the moving dot field appeared in the contralateral versus ipsilateral VF during the localizer runs. Because the task-irrelevant stimulus always appeared in the upper left visual quadrant, all ROIs associated with the task-irrelevant stimulus were located in the right hemisphere. For MT+ and V3a, we selected groups of contiguous voxels (minimum cluster size was 10 voxels, because MT+ was rather small for some participants) using, for each participant, the lowest statistical threshold (i.e., highest p value) at which these regions were spatially separated from one another and separated from the ventral visual areas, resulting in a different threshold for each individual participant that ranged from p = .001 to p = 5.0 × 10−7. We then used the lowest statistical threshold defined above to define our ventral visual cortex ROIs (including V1 through V4). Although the contrast we used to identify the task-irrelevant ROIs (left VF stimulus > right VF stimulus) was chosen to primarily identify voxels responsive to the task-irrelevant location, V3a and MT+ do not respond exclusively to the contralateral hemifield (Tootell, Mendola, Hadjikhani, Liu, & Dale, 1998; Tootell et al., 1997), and thus, there was some concern that the activity profiles in V3a and MT+ could contain some input from the attended, ipsilateral field. To check for this possibility, we created a second, more restricted set of V3a and MT+ ROIs in which the threshold was increased to isolate the most spatially selective voxels for the contralateral field, such that these restricted ROIs had approximately 50% of the voxels of the original ROIs for each individual (see Table 1 for details). Our logic was the following: If contamination from the ipsilateral (attended) region was driving the effects we saw in the original V3a and MT+ ROIs, then the effects should be weakened when the ROIs are made more spatially selective. Similar or stronger effects in the restricted dorsal ROIs would be taken as evidence that the effects were primarily driven by the contralateral (task-irrelevant) stimulus.
|MT+ Restricted Voxels|
|V3a Restricted Voxels|
|MT+ Restricted Voxels|
|V3a Restricted Voxels|
Area MT+ activity was identified on the basis of its location in posterior, dorsolateral occipital cortex. Area V3a was identified, and the ventral visual cortex activity was further subdivided into V1 through V4 on the basis of the separate retinotopic mapping scans. Briefly, these regions were delineated by contrasting the horizontal versus vertical meridians in the upper and lower VFs. The retinotopic areas were then combined with a large ventral visual ROI that was extracted from the localizer scan to assign this activity to its corresponding visual area for the final ROIs. In addition to creating ROIs in the right hemisphere, we created comparable ROIs in the left hemisphere corresponding to the upper right VF quadrant to test whether the task-irrelevant stimulus affected processing of the attended stimulus, using the same voxel selection, retinotopic delineation and thresholding methods described above. Finally, for each participant we extracted the beta weights for each task-irrelevant stimulus condition (SColSDir, SColODir, OColSDir, and OColODir) from each ROI. This data was then submitted to a within-subject ANOVA with Color (same or opposite), Direction (same or opposite), and Size (unrestricted versus restricted) as factors.
Overall average accuracy on the luminance detection task was 82%, indicating that participants were sufficiently able to perform the main task. Furthermore, neither accuracy nor false alarm rates on the detection task differed as a function of the color [accuracy, F(1, 8) = 1.4, p = .3; false alarms, F(1, 8) = 0.1, p = .8] or the direction [accuracy, F(1, 8) = 0.3, p = .6; false alarms, F(1, 8) = 0, p = 1.0] of the task-irrelevant stimulus, suggesting that participants' behavior was not significantly influenced by this stimulus.
Figure 2A shows percent signal change in V1, V2, VP, V4, V3a, and MT+ for each condition in response to the contralateral, task-irrelevant dot field. An ANOVA performed on the average activity in each ROI revealed a main effect of Color in all regions except MT+ [V1, F(1, 8) = 10.3, p = .01; V2, F(1, 8) = 20.1, p = .002; VP, F(1, 8) = 20.5, p = .002; V4, F(1, 8) = 17.9, p = .003; V3a, F(1, 8) = 5.5, p = .047; MT+, F(1, 8) = 2.28, p = .2], with the task-irrelevant dot field eliciting a greater response when the dots' color matched the attended color. Importantly, there was also a main effect of Direction in all regions except V2, but even in this area the effect approached significance [V1, F(1, 8) = 7.7, p = .02; V2, F(1, 8) = 5.3, p = .051; VP, F(1, 8) = 8.7, p = .02; V4, F(1, 8) = 9.3, p = .02; V3a, F(1, 8) = 13.5, p = .006; MT+, F(1, 8) = 8.1, p = .02]; that is, we saw clear modulation of the task-irrelevant dimension at the task-irrelevant location. As shown in Figure 2A, the task-irrelevant dot field elicited higher activity when the dots moved in the same direction as the attended dots. Interestingly, there were no interactions between Color and Direction in any region [V1, F(1, 8) = 0.97, p = .4; V2, F(1, 8) = 1.3, p = .3; VP, F(1, 8) = 1.6, p = .2; V4, F(1, 8) = 0.05, p = .8; V3a, F(1, 8) = 0.5, p = .2; MT+, F(1, 8) = 0.17, p = .7], indicating that the modulation of the task-irrelevant feature did not depend on the presence of the task-relevant dimension. In other words, the two features were modulated independently at the task-irrelevant location.
Because the receptive fields of some cells in MT+ and V3a include regions of the ipsilateral VF (e.g., Tootell et al., 1998), we also performed a restricted voxel analysis for these regions in which a subset of more spatially selective voxels comprised the ROIs (Figure 3; see Methods for details on this analysis). If the results we saw were due to ipsilateral signals from the attended stimulus, then restricting the number of voxels should reduce the direction effect we observed. The main effect of Direction was significant in both restricted regions [restricted V3a, F(1, 8) = 9.5, p = .02; restricted MT+, F(1, 8) = 6.2, p = .04]. Importantly, the direction effects in the restricted ROIs were unchanged relative to the larger ROIs [V3a, F(1, 8) = .004, p = .9; MT+, F(1, 8) = 0.03, p = .9, for the interaction between direction effect and number of voxels]. The same was true of the color effects; there was a significant main effect of Color in restricted V3a, F(1, 8) = 6.7, p = .03, but not in restricted MT+, F(1, 8) = 1.9, p = .2, and these effects were similar across the unrestricted and restricted regions [V3a, F(1, 8) = 0.2, p = .7; MT+, F(1, 8) = 0.01, p = .9, for the interaction between color effect and number of voxels]. Finally, there were no interactions between color and direction in the restricted ROIs [V3a, F(1, 8) = 0.04, p = .8, MT+, F(1, 8) = 0.25, p = .6]. Taken together, these results indicate that the color and direction effect sizes did not diminish in the restricted ROIs; in fact, the average effect sizes were highly similar across the restricted and unrestricted ROIs (average color effect: V3a = 0.12, V3a restricted = 0.11; MT+ = 0.048, MT+ restricted = 0.048; average direction effect: V3a = 0.11, V3a restricted = 0.11; MT+ = 0.088, MT+ restricted = 0.088). Importantly then, this restricted analysis indicates that the effects from the larger ROIs could not have been primarily driven by ipsilateral activity.
As shown in Figure 4, the average color and direction difference scores were not significantly different from each other in any ROI, indicating that the task-relevant and task-irrelevant dimensions were modulated to the same extent in the task-irrelevant location. However, the time course of the color and direction effects for V4 and MT+ (Figure 5, based on unmodeled data, normalized to the temporal mean) appear to differ, with the color effect peaking before the direction effect in V4 and vice versa in MT+. We note, however, that because the hemodynamic response tends to rise more steeply with greater activation levels, these patterns may be due to the overall increased color versus direction effect in V4 and vice versa in MT+; however, these findings raise the possibility that the boost from the nonpreferred dimension (e.g., direction in V4) appears later, possibly as a result of feedback from areas better suited to process that dimension (e.g., MT+ for motion direction). Further research using a methodology with finer temporal resolution will be needed to determine if this is in fact the case.
Finally, we also assessed activity in each ROI responding to the attended location. First, we compared overall mean activity in the left (task-relevant) and right hemispheres (task-irrelevant) for each ROI to assess whether, in keeping with task instructions to attend to the stimulus in the right VF, the task-relevant stimulus did in fact elicit greater neural response in the left versus right hemisphere. In V1–VP, mean activity was similar across the two hemispheres [V1, t(8) = 1.93, p < .09; V2, t(8) = 0.09, p < .9; VP, t(8) = 0.33, p < .75]. However, in V4, restricted V3a, and restricted MT+, where attention effects are typically larger (Beck & Kastner, 2009), mean activity was significantly greater in the left hemisphere, corresponding to the attended location, than in the right hemisphere [V4, t(8) = 3.37, p < .01; V3a restricted, t(8) = 2.6, p < .03; MT+ restricted, t(8) = 3.9, p < .005]. The greater activity for the attended location cannot be due to the fact that the attended location contained two dot fields moving in opposite directions as opposed to the single dot field in the task-irrelevant location, as superimposed dot fields with opposing motion typically elicit less activity than a single dot field (due to suppressive or competitive interactions), at least under conditions of inattention (e.g., Britten & Heuer, 1999). Thus, we interpret the overall increased response to the task-relevant stimulus as an index of higher priortization of the task-relevant stimulus over the task-irrelevant stimulus.
We next assessed whether responses to the task-relevant stimulus were influenced by the color or direction of the task-irrelevant stimulus (Figure 2B). In the ventral left hemisphere (lh), there was a significant main effect of Color of the task-irrelevant stimulus in lhV1, F(1, 8) = 5.7, p = .04, and a trend for a main effect of Color in the other ventral regions [lhV2, (F(1, 8) = 4.8, p = .06; lhVP, F(1, 8) = 4.2, p = .07; lhV4, F(1, 8) = 3.8, p = .09]. There was a trend for a main effect of Direction of the task-irrelevant stimulus in lhV1, F(1, 8) = 4.8, p = .06, and there were significant main effects of Direction in lhV2, F(1, 8) = 9.1, p = .02, lhVP, F(1, 8) = 8.3, p = .02, and lhV4, F(1, 8) = 7.7, p = .02. In the left dorsal stream regions, there were no effects of Color [lhV3a, F(1, 8) = 1.9, p = .2; lhMT+, F(1, 8) = 0.26, p = .6], but there was a significant effect of Direction in lhV3a, F(1, 8) = 12.6, p = .008, and a trend toward a significant effect of Direction in lhMT+, F(1, 8) = 4.96, p = .06. As in the right hemisphere, there were no interactions between Color and Direction in any region [lhV1, F(1, 8) = 1.2, p = .3; lhV2, F(1, 8) = 0.8, p = .4; lhVP, F(1, 8) = 0.6, p = .5; lhV4, F(1, 8) = 0.7, p = .4; lhV3a, F(1, 8) = 0.06, p = .8; lhMT+, F(1, 8) = 2.6, p = .1]. Similar to the restricted analysis of dorsal ROIs responding to the task-irrelevant stimulus, we performed a restricted analysis of lhV3a and lhMT+ (using the same procedure described above) to reduce any influence from the ipsilateral hemifield, which in this case contained the task-irrelevant stimulus. Consistent with the unrestricted lh ROIs, there was a significant effect of Direction in restricted lhV3a, F(1, 8) = 8.4, p = .02, a trend for a direction effect in restricted lhMT+, F(1, 8) = 4.75, p = .06, and no interactions between the number of voxels and the direction effect in either region [lhV3a, F(1, 8) = 0.04, p = .8; lhMT+, F(1, 8) = 0.5, p = .5]. Again, there were no main effects of Color [restricted lhV3a, F(1, 8) = 1.1, p = .3; restricted lhMT+, F(1, 8) = 0.4, p = .5] and no interactions between Color and Direction in either of the restricted lh ROIs [lhV3a, F(1, 8) = 0.3, p = .6; lhMT+, F(1, 8) = 3.8, p = .09]. Thus, the effects observed in the left dorsal ROIs were also not primarily driven by spill from the ipsilateral, task-irrelevant stimulus.
Taken together, these results suggest that both the ventral and dorsal responses associated with the attended stimulus benefited from having the same direction and (to a weaker extent) color as the attended surface present in the task-irrelevant location. However, this neural result was not mirrored in the behavioral results, because neither accuracy nor false alarms changed as a function of the direction or color of the task-irrelevant stimulus. It is still possible, however, that a more sensitive behavioral measure may benefit from a matching stimulus in a task-irrelevant location.
To test whether the color and direction effects were comparable among the left and right ROIs, we performed a three-way ANOVA with Hemisphere, Color of the task-irrelevant stimulus, and Direction of the task-irrelevant stimulus as factors. There were no interactions between Hemisphere and Color for V1, F(1, 8) = 0.46, p = .5, or restricted MT+, F(1, 8) = 2.1, p = .2, and no interactions between Hemisphere and Direction for V1, F(1, 8) = 1.6, p = .2, VP, F(1, 8) = 2.7, p = .14, V4, F(1, 8) = 1.5, p = .3, or restricted MT+, F(1, 8) = 0.13, p = .7. However, there were significant interactions between Hemisphere and Color for V2, F(1, 8) = 7.3, p = .03, VP, F(1, 8) = 19.9, p = .002, V4, F(1, 8) = 22.8, p = .001, and restricted V3a, F(1, 8) = 8.8, p = .02, with larger color effects in the right (corresponding to task-irrelevant location) versus left hemisphere. Finally, there was a trend toward a significant interaction between Hemisphere and Direction in V2, F(1, 8) = 4.1, p = .08, and a significant interaction between Hemisphere and Direction in restricted V3a, F(1, 8) = 5.4, p = .048, again with larger direction effects in the right versus left hemisphere. These results suggest that, whereas the left hemisphere was influenced by the conditions of the task-irrelevant stimulus, the effects of a feature match between task-relevant and task-irrelevant locations were either similar or greater in the right hemisphere, corresponding to the task-irrelevant location. In other words, the act of attending to a particular color (or direction) in the task-relevant location had greater effects on the task-irrelevant location than the identity of the task-irrelevant stimulus had on the task-relevant stimulus.
Our goal for this study was to determine whether task-relevant (color) and task-irrelevant (motion) properties of an attended object (moving colored dots) would be modulated independently at a task-irrelevant location. Previous research has shown both that an attended feature (i.e., task-relevant feature) is enhanced throughout the VF (e.g., Treue & Martinez-Trujillo, 1999) and that a task-irrelevant feature of an attended object is enhanced at the location of the object (O'Craven et al., 1999). Here, we further asked whether this enhancement of the task-irrelevant feature depends on the presence of the task-relevant feature at the task-irrelevant location. In keeping with previous research, we found a significant main effect of the task-relevant feature of color in regions corresponding to the task-irrelevant location in visual areas V1 through V4 and V3a, such that the dots in this location elicited greater activity when they matched the color of the attended object (same color conditions: SColSDir and SColODir) than when they matched the color of the ignored object (opposite color conditions: OColSDir and OColODir). In other words, we observed an effect of the attended color at a task-irrelevant location, consistent with previous single-cell (e.g., Treue & Martinez-Trujillo, 1999) and neuroimaging studies (e.g., Serences & Boynton, 2007; Saenz et al., 2002) that also observed global, enhanced modulation in visual cortex during feature-based attention.
Importantly, although both motion and the dot field location under investigation were irrelevant to the task, we also found a significant effect of motion direction in visual areas V1 through V4, V3a and in MT+, such that the task-irrelevant dot field elicited greater responses when it was moving in the same direction (SColSDir and OColSdir) than the opposite direction (SColODir and OColOdir) as the task-relevant dot field. Consistent with prior neural evidence (e.g., Boehler, Schoenfeld, et al., 2011; Katzner et al., 2009), this result suggests that attending to one dimension of an object enhances processing of task-irrelevant features bound to that object in not only the same location (as demonstrated by O'Craven et al., 1999) but also in a task-irrelevant location in the opposite VF.
Finally, although there were significant main effects of Color and Direction in ROIs responding to the task-irrelevant location, we did not observe an interaction between Color and Direction in any region. This lack of an interaction suggests that the task-relevant and task-irrelevant dimensions of the attended object were modulated independently at the task-irrelevant location, which is consistent with the idea that global effects of object-based attention are due to multiple feature-based effects that occur independently for each feature of an attended object; that is, attention did not modulate activity globally for the object per se, which would have predicted that the effect of direction would have been specific to or at least greater when both features were present at the task irrelevant location. Instead, we observed equivalent effects of direction when the task-irrelevant stimulus was the same or opposite color as the attended object. This suggests that spatially distant effects of task-irrelevant dimensions, although presumably initially mediated by object-based mechanisms, may ultimately be based on global feature-based attentional mechanisms.
We should point out, however, that these results are also consistent with an object-based mechanism (or template) that can evoke graded responses based on a partial match to the template. For instance, it is possible that the brain creates a global template for a conjoined, attended object, and the response to a task-irrelevant item increases with the number of shared features between the task-irrelevant item and the object template. Of course, one way to implement such a template capable of eliciting a graded response would be to modulate the features independently as described. It then becomes an issue of semantics as to whether the failure for the features to interact at the task-irrelevant location should be considered as evidence of an object-based template or simply two feature-based mechanisms. Certainly our data suggest there is no added benefit, above and beyond the benefit accrued by each feature, of the task-irrelevant stimulus matching the attended stimulus in both color and direction.
One benefit of the stimuli used in the current study was that they simultaneously activated regions in the ventral and dorsal streams, allowing us to assess the integrated competition model of attention (Duncan, 2006; Duncan et al., 1997). This theory proposes first that visual objects compete with one another such that increased response in cells coding relevant objects is accompanied by decreased activity in cells coding irrelevant objects. This competition among objects is then integrated across multiple brain systems such that, as one object wins the competition in one system, properties of that object dominate in other brain systems. The result is that all target-related features, including spatial location and any other task-irrelevant property, are preferentially processed over other nontarget features throughout the brain.
In the context of the current study, such a model could explain why various visual areas were modulated by motion direction in a task-irrelevant location despite the fact that motion was task-irrelevant and why such effects would be present in both the ventral and dorsal streams. By virtue of attending to the color of the relevant dot field as a whole, participants may have automatically coded the direction of motion associated with the attended dots, consciously or not. Once this motion direction was selected, albeit indirectly, the modulation spread to a task-irrelevant location, because such feature-based selection is ultimately a global mechanism (e.g., Treue & Martinez-Trujillo, 1999). Moreover, the integrated competition model posits that once both the task-relevant and task-irrelevant features are coselected, their enhancements are propagated throughout the brain systems involved in processing those features. In keeping with the integrated competition model, Boehler, Schoenfeld, et al. (2011) showed that although initially only the task-relevant feature was modulated by attention, soon (80 msec) after the object was selected they began to see global modulation of the task-irrelevant feature. Because they were using ERP, however, it was unclear whether this effect was actually integrated across multiple brain areas or not. Here, we show global modulation of task-irrelevant features in multiple ventral areas as well as dorsal areas, providing support for one of the critical tenets of the integrated competition model. In short, our results are not only consistent with the integrated competition idea but also provide further insight into the specific mechanisms by which task-irrelevant features of attended objects come to dominate neural systems across the brain.
Is Object-based Enhancement Obligatory?
Although our results suggest that attending to an object can result in automatic selection of task-irrelevant features, it is important to note that this automatic coselection may not always occur. For instance, task-relevant features can be prioritized over task-irrelevant features if task conditions favor such a strategy (Wegener, Ehn, Aurich, Galashan, & Kreiter, 2007). Although our task required that subjects prioritize color over motion when the stimulus first appeared (so he or she would know which dots to attend to), once they had selected the appropriate dots there was no need to prioritize color over motion and thus no reason to suppress the task-irrelevant dimension. Our data should not, therefore, be taken as evidence that task-irrelevant features must be coselected but rather it assesses the neural consequences of such coselection when it does occur.
What Is the Source of the Spatially Distant, Task-irrelevant Feature Modulation?
Presumably, initially top–down signals from fronto-parietal cortex serve to boost the attended color in ventral visual cortex. How then does this enhancement spread to a motion direction? In predicting that the direction of the attended dots would be modulated in the task-irrelevant location, one might have expected that this effect would be primarily observed in dorsal regions more typically associated with coding direction of motion, such as MT+ and V3a. In fact, we did find significant main effects of direction in these regions, but we also found comparable effects in ventral regions V1 through V4, which are more commonly associated with recognition of object features such as color, form, and orientation (e.g., Ungerleider & Mishkin, 1982). However, the ventral motion effect is not entirely surprising given that the parvocellular pathway, which predominantly codes color and form information, and the magnocellular pathway, which is more sensitive to contrast and motion information (e.g., Ferrera, Nealey, & Maunsell, 1992) are not completely restricted to the ventral and dorsal streams, respectively. For example, Gegenfurtner, Kiper, and Fenstemaker (1996) showed that both distinct and common cell populations in macaque V2 preferentially respond to multiple features, such as color and direction of motion. Additionally, Ferrera et al. (1992) observed that macaque area V4 receives both parvocellular and magnocellular inputs. Finally, Huk, Ress, and Heeger (2001) demonstrated that areas V1, V2, VP, and V4 are also direction selective in humans using an adaptation paradigm with fMRI. Thus, it is possible that the enhancement of color-sensitive cells spread to motion-sensitive cells within the early ventral regions themselves, either by virtue of the same cells coding both color and motion (e.g., Gegenfurtner et al., 1994) or via lateral connections between cells that code different properties (Gegenfurtner et al., 1996).
An alternative possibility is that ventral visual areas received information about motion direction indirectly via feedback from more dorsal regions such as V3a and MT+ (e.g., Duncan, 2006). In other words, the direction effect in ventral visual areas may not have been mediated by direction-sensitive cells in these regions but rather resulted from an interaction between the ventral and dorsal streams as a result of some binding process. Reciprocal anatomical connections between V1 through V4 and V3a and MT found in primate visual cortex (e.g., Felleman & Van Essen, 1991) make such interactions possible. Moreover, as shown in Figure 4, the size of the direction difference score increased from V1 to V4, consistent with recent evidence that attentional feedback progresses from higher to lower visual areas (e.g., Boehler, Tsotsos, Schoenfeld, Heinze, & Hopf, 2011; Buffalo, Fries, Landman, Liang, & Desimone, 2010); however, we should note that this linear trend was not significant (p = .4). Nonetheless, because there are only a limited number of direction-selective cells in V4 (e.g., Felleman & Van Essen, 1987), local modulations of direction information in V4 might be expected to be weak, making the large direction effect observed in V4 more likely to have arisen from feedback from dorsal regions. Such a feedback mechanism would also be consistent with the observation that the direction effect arose later than the color effect in V4 (Figure 5); however, such arguments are speculative at this point.
It is possible, when conditions are such that it is advantageous to select one feature of an object over another, for feature-based attention to override object-based effects (i.e., only the attended feature is enhanced; Wegener et al., 2007). However, if there is no reason to actively inhibit the task-irrelevant feature, attention tends to work in an object-based manner. We found that not only can attention spread to the task-irrelevant feature of an object, but that feature is then in turn modulated independently at a task-irrelevant location. Although further work is necessary to uncover the specific neural circuitry, our results indicate that object-based attention ultimately manifests as multiple feature-based attention mechanisms, one for each feature of the selected object, that enhance those features beyond the location of selected object. Although serial attention may be necessary for initially integrating multiple dimensions of an object, such as color and motion (e.g., Treisman & Gelade, 1980), it also appears that, once selected, these dimensions can become “unbound” and modulated independently and in parallel.
This work was funded by National Institute of Mental Health grant R03 MH082012 (D. M. B.). We thank Paige Scalf for assistance in data collection and Ana Torralbo for assistance in data analysis.
Reprint requests should be sent to Audrey G. Lustig, 603 E. Daniel St. #514, Champaign, IL 61820, or via e-mail: firstname.lastname@example.org.