Shifting attention from one color to another color or from color to another feature dimension such as shape or orientation is imperative when searching for a certain object in a cluttered scene. Most attention models that emphasize feature-based selection implicitly assume that all shifts in feature-selective attention underlie identical temporal dynamics. Here, we recorded time courses of behavioral data and steady-state visual evoked potentials (SSVEPs), an objective electrophysiological measure of neural dynamics in early visual cortex to investigate temporal dynamics when participants shifted attention from color or orientation toward color or orientation, respectively. SSVEPs were elicited by four random dot kinematograms that flickered at different frequencies. Each random dot kinematogram was composed of dashes that uniquely combined two features from the dimensions color (red or blue) and orientation (slash or backslash). Participants were cued to attend to one feature (such as color or orientation) and respond to coherent motion targets of the to-be-attended feature. We found that shifts toward color occurred earlier after the shifting cue compared with shifts toward orientation, regardless of the original feature (i.e., color or orientation). This was paralleled in SSVEP amplitude modulations as well as in the time course of behavioral data. Overall, our results suggest different neural dynamics during shifts of attention from color and orientation and the respective shifting destinations, namely, either toward color or toward orientation.
Everyday adaptive behavior requires frequent shifts and focusing attention to a certain feature of an object or to a particular location. In recent studies, we investigated temporal dynamics of neural facilitation and suppression of early visual cortex activity during cued shifts of attention to a certain location (Müller, 2008; Müller, Teder-Sälejärvi, & Hillyard, 1998) or to a certain color of two superimposed random dot kinematograms (RDKs; Andersen & Müller, 2010). To this end, we recorded objective electrophysiological measures of neural activity in early visual cortical areas of the human brain (steady-state visual evoked potentials, SSVEPs) that tracked the time course of neural dynamics of attentional shifting (Andersen & Müller, 2010; Müller et al., 1998) in conjunction with behavioral data. Although our previous studies used designs in which participants were cued to shift attention after a neutral baseline period, very little is known about the top–down modulated shifting dynamics in feature-selective attention when participants are cued to shift attention from one attended feature to another. Of particular interest is the question of whether all shifting processes in the feature domain underlie identical temporal dynamics. In other words, would shifting attention from color to color be similar to shifting attention from color to orientation? A prominent model in feature-selective attention is the feature similarity gain model (cf. Treue, 2001; Treue & Trujillo, 1999). This model, however, focuses basically on some principle neural mechanisms, such as global facilitation of a to-be-attended feature. It makes no particular assumptions with regard to temporal dynamics of feature-based shifts of attention. Recently, Buffalo, Fries, Landman, Liang, and Desimone (2010) found a backward progression in monkey ventral stream from higher-order to lower-order areas during top–down shifts of attention. In other words, attentional modulations showed earlier onset latencies in V4 compared with V2 and arrived latest in V1. These differences were substantial with an onset of attentional modulation of neural firing rates in V4 at about 170 msec, in V2 at about 440 msec, and in V1 at about 860 msec, when monkeys needed to detect and respond to a color change of a Gabor patch. Given the anatomical organization of the visual cortex in which color is processed further upstream in the ventral stream in human V4 complex and V8 (cf. Chao & Martin, 1999; Hadjikhani, Liu, Dale, Cavanagh, & Tootell, 1998; Allison et al., 1993; Corbetta, Miezin, Dobmeyer, Shulman, & Petersen, 1990) than orientation (cf. Hubel & Wiesel, 1968),1 one would expect earlier neural facilitation after cue onset when participants shifted attention toward color compared with when they need to shift attention to the orientation of a stimulus.
Evidence for such a backward progression was also found in human visual cortex in spatial-based attention. Martinez and colleagues (2001) reported a negative deflection of the visual evoked potential (VEP) in the time window between 160 and 260 msec after stimulus onset for attended compared with unattended locations in a classical Posner paradigm (Posner, 1980) having its generators in primary visual cortex (V1). This negativity was preceded by the classical finding of an amplitude modulation of an early positive component with a latency of about 80–100 msec (P1) with its generators in extrastriate visual cortex (cf. Martinez et al., 2001; Heinze et al., 1994). The authors interpreted that late negative VEP modulation was attributed to V1 as clear evidence for what they called reentrant loops from higher visual areas. In the light of the study by Buffalo and colleagues (2010), this finding adds on to the idea of top–down modulated backward progression of neural attentional modulation in early visual areas.
Another hint for different temporal dynamics for attention shifts in feature-selective attention comes from visual search. The dimensional weighting account (DWA) proposes that search guidance can be performed on the basis of feature dimensions, such as color, motion, or orientation, rather than emphasizing a certain characteristic such as blue, horizontal, or moving upward (cf. Müller, Reimann, & Krummenacher, 2003; Müller, Heller, & Ziegler, 1995). An attentional weight (Duncan & Humphreys, 1989) will be allocated to a certain feature dimension that constitutes the search object. This enhances the saliency of the target by amplifying its constituent feature dimension (Müller et al., 2003; Found & Muller, 1996). Empirical evidence for DWA was mainly found in experimental designs that presented one object that differed in one feature compared with the other objects in the display (odd one out). Such singleton feature search reveals that presenting an odd-one-out target that varies in the same dimension as the one of the preceding trial (i.e., color: change from red to green) has no influence on the detection speed. If, however, the subsequent odd-one-out target changes its feature dimension (i.e., change from color to orientation) detection speed is slowed down (Müller & Krummenacher, 2006a, 2006b; Müller et al., 1995). DWA explains these costs by claiming that an attention weight needed to be shifted from the “old” (i.e., color) to the “new” target dimension (i.e., orientation). So far, only one behavioral study exists that extended the idea of DWA to a nonsearch task (Müller & O'Grady, 2000). In this study, observers had to make single or dual judgments to one or both of two overlapping (and briefly presented) objects, where dual judgments involved either one dimension (color or form) or both dimensions. Although the study replicated the dual-object-judgment cost (Duncan, 1984), it also found that it was harder (in terms of reduced accuracy) making dual perceptual judgments across, than within, visual feature dimensions, regardless of whether judgments had to be directed to one or two objects.
Taken together, although the studies by Buffalo and Martinez and respective colleagues found some evidence for neural top–down modulated backward progression to early visual areas that resulted in earlier neural facilitation in higher order visual areas, DWA suggests a time-consuming shifting process when attention needs to be shifted from one feature dimension to another. The difference between the two accounts lies in the fact that a top–down modulated backward progression would result in earlier shifts toward color, regardless of its origin (i.e., color or orientation). DWA would claim that all shifts toward a newly to-be-attended feature dimension are more time consuming; in other words, a shift toward color is earlier when started from color compared with the situation when orientation was the origin of the shift, and the same would be true vice versa.
This study set out to investigate neural dynamics in feature-selective attention in situations where participants were instructed to shift attention toward either color or orientation but when attention was already deployed to either color or orientation for several seconds. To this end, we presented our participants with superimposed RDKs that differed in color (red vs. blue) and orientation (slashes vs. backslashes). These RDKs flickered at different frequencies, respectively, to elicit SSVEPs that allowed us to analyze temporal dynamics of shifting processes in early visual cortex because of its ongoing oscillatory nature (Andersen & Müller, 2010; Müller et al., 1998). In previous studies that used a similar display of four RDKs with bars that conjoined color and orientation, we demonstrated that these stimuli elicited reliable SSVEPs with statistically significant amplitude modulations when participants attended to either color or orientation (Andersen, Müller, & Hillyard, 2008, 2015). Given the central presentation of these bars, unsurprisingly, these SSVEPs experienced a focused amplitude maximum at occipital electrodes, and cortical source reconstructions found the generators of the attention effect in early visual cortical areas including V1 (Andersen et al., 2008).
In accordance with a number of earlier studies, feature-selective attention modulated processing for color as well as for orientation. SSVEP amplitude modulation for color was greater compared with orientation. Overall, our results suggest different neural dynamics during shifts of attention from color and orientation toward respective shifting destinations, namely, color or orientation.
Twenty volunteers (18–35 years, mean age = 23 ± 4 years; eight men; 16 right-handed) with normal color vision and normal or corrected-to-normal visual acuity took part in the experiment after giving written informed consent. None of the participants reported a history of neurological disease. The experiment accorded to the Declaration of Helsinki and the guidelines of the local ethics committee. Volunteers received class credit or monetary compensation. Two participants were excluded from data analyses. One performed at chance level in the behavioral task. Another one showed excessive eye movements during EEG recordings.
Procedure and Stimulus Material
Participants were seated comfortably in an acoustically dampened and electromagnetically shielded chamber in front of a 19-in. cathode ray tube screen at a distance of 80 cm. The screen, set to a refresh rate of 120 Hz and a resolution of 640 × 480 pixels with a 32-bit color depth, displayed the experimental stimulation that was composed of four completely overlapping RDKs with a diameter of 13° of visual angle in front of a gray background (luminance = 6 cd/m2). Each RDK consisted of 36 dashes carrying a unique conjunction of features color (red or blue) and orientation (dash tilted by 45° = “slash” or dash tilted by 135° = “backslash”). Resulting RDKs were frequency tagged (see Figure 1A) to elicit SSVEPs that provided distinguishable rhythmic neural signatures in EEG recordings. Frequency tagging was realized by presenting RDKs at respective rates of 10 Hz (blue backslashes), 12.5 Hz (red backslashes), and 15 Hz (blue slashes), with a 50% on–off ratio per cycle. As an exception, the 17.5-Hz stimulation frequency (red slashes) was realized by an approximated sinusoidal luminance modulation for technical reasons. Single dashes subtended an area of 0.44° × 0.08° of visual angle and moved by 0.04° in a randomly chosen direction with each screen duty cycle. Dashes of all RDKs were drawn in random order to prevent depth cues induced by a systematic superposition of one RDK with another. Before the experiment, participants adjusted red and blue colors to isoluminance with the gray background by means of heterochromatic flicker photometry (Wagner & Boynton, 1972).
Experimental trials started with the onset of all four RDKs and a centrally presented fixation cross that, by its appearance (see Figure 1A and B), permanently cued participants to attend to one of the four features: blue, red, slash, and backslash. Participants thus always attended to two of the RDKs while ignoring the other two. For instance, when red was cued, participants simultaneously attended the red-slash and red-backslash RDKs. After a variable time interval between 2400 and 2800 msec, the fixation cross changed to cue attention toward another feature (either blue, red, slash, or backslash). This change did not occur instantaneously, but cues faded over for 200 msec to attenuate change-evoked ERPs that would have influenced the analyses of SSVEP amplitude time courses. For a minimum of 2000 msec after cue change onset, RDK stimulation continued (see Figure 1B). Between trials, a white fixation cross was presented for 1000 msec that indicated participants to blink to reduce eye movements during stimulation.
Trials of all conditions were presented in randomized order in 12 blocks (∼5-min duration) of 48 trials each. For each of the four experimental conditions that entered statistical analysis, namely, shifts from “attend red to attend blue” (R→B), “attend blue to attend backslash” (B→\\), “attend backslash to attend slash” (\\→//), and “attend slash to attend red” (//→R), we presented 144 trials (576 trials in total). Note that our experimental design was deliberately unbalanced because a fully balanced design would have quadrupled net recording time (to about 4 hr).
Participants were instructed to detect brief episodes (400 msec) of dashes moving coherently along the vertical axis (up or down). Only a randomly chosen 75% of dashes in one of the two attended RDKs showed this coherent motion to prevent participants from tracking single dashes. Participants had to discriminate between coherent motion in attended RDKs (targets) and unattended RDKs (distractors) and respond to targets via button press. Responding hand was changed halfway through the experiment with the starting hand counterbalanced across participants. A given trial featured any combination of up to three targets and distractors that could be presented as early as 300 msec after trial onset. Subsequent target and distractor onsets were separated by at least 800 msec. Only 50% of the trials of each condition contained coherent motion events to retain a sufficient amount of target- and distractor-free trials for the analyses of SSVEPs. To allow for analyzing the time course of behavioral data, target and distractor onsets were evenly distributed across 60 bins of 66 msec each for each condition. Bins were evenly distributed around the cue change such that 30 bins measured preshift performance and the other 30 bins measured postshift performance. Each bin contained one target and one distractor, resulting in 120 binned coherent motion events. Given 144 coherent motion events per condition (of 144 trials, 72 had no events, 24 had one event, 24 had two events, and 24 had three events), the remaining 24 events (12 targets and 12 distractors) were randomly assigned to different bins. Before the experiment, participants practiced the behavioral task for at least two blocks. After each training and experimental block, they received feedback on their performance.
EEG Data Recording and Preprocessing
EEG was recorded from 64 Ag–AgCl scalp electrodes and amplified by a Biosemi ActiveTwo amplifier (Biosemi, Amsterdam, Netherlands) set to a sampling rate of 256 Hz. Electrodes were mounted in a nylon cap according to the 10–20 International system. Vertical eye movements and blinks were monitored by a bipolar montage located above and below the right eye. Lateral eye movements were monitored by a bipolar outer canthus montage. Data processing was performed using the EEGLAB toolbox (Delorme & Makeig, 2004) combined with custom routines written in MATLAB (The MathWorks, Natick, MA). Epochs of 5200 msec, starting 2400 msec before the shifting cue, were extracted from continuous data. We discarded epochs containing targets or distractors to avoid signal contaminations because of ERPs evoked by targets/distractors and corresponding motor responses in case of button presses. Trials with blinks or eye movements exceeding a threshold of 19 μV (corresponding to a horizontal eye movement of about 1.5° of visual angle) were automatically rejected. The “Fully Automated Statistical Thresholding for EEG Artifact Rejection” (Nolan, Whelan, & Reilly, 2010) procedure was applied to identify and correct further artifacts. This procedure replaced data from artifact-contaminated channels with spherical-spline interpolations based on data from artifact-free channels. Epochs with more than 12 contaminated channels were excluded from further analysis. On average, 12% of trials per participant and condition were contaminated with artifacts and were excluded. Artifact-free data were rereferenced to average reference. Subsequently, data were averaged across epochs for each participant and experimental condition, separately.
EEG Data Analyses
EEG analyses focused on 12.5- and 15-Hz SSVEPs entirely because, due to the unbalanced design of the experiment, the RDKs frequency tagged with 10 and 17.5 Hz were only subject to attentional shifts in half of the conditions. As an example, the blue-backslash RDK (10 Hz) remained attended in condition B→\\ as well as remained to be ignored in condition //→R. Second, 10- and 17.5-Hz RDKs only experienced shifts within a feature dimension, namely, color-to-color (C→C) and orientation-to-orientation (O→O), which disqualified them from analyses of between-dimension shifts, namely color-to-orientation (C→O) and orientation-to-color (O→C). Although generally irrelevant to our EEG analyses, both RDKs were flickered nevertheless to maintain maximal physical and perceptual similarity to the 12.5- and 15-Hz RDKs.
SSVEP amplitudes were analyzed in two steps. In a first spectral analysis, we retrieved SSVEP amplitudes averaged across preshift and postshift periods to identify clusters of electrodes exhibiting maximal SSVEP amplitude in scalp topographies by means of a fast Fourier transformation. We further investigated amplitude modulations between preshift and postshift periods to control for participants' compliance and evaluate attention effects (i.e., amplitude increases/decreases). A second spectrotemporal analysis of EEG epochs yielded SSVEP amplitude time courses that served to characterize the neural dynamics during attentional shifts.
Fourier transforms of detrended data (mean and first-order linear trend removed) were retrieved from two time windows before and after the shifting cue. The preshift window was extracted from shifting cue onset to 2000 msec before the onset. Previous studies demonstrated that shifting is a rather slow process in the order of about 400–500 msec (cf. Andersen & Müller, 2010; Müller et al., 1998); therefore, we extracted a time window starting 400 to 2400 msec after the shifting cue for postcue period spectral analysis. We averaged across these two time windows and experimental conditions to obtain the scalp topographical distributions of 12.5- and 15-Hz SSVEPs that are depicted in Figure 2A (see Results). In these grand-averaged scalp maps, amplitudes peaked at a cluster of nine central occipital electrodes for both frequencies. From this cluster, we selected five electrodes with greatest SSVEP amplitudes for each participant and frequency individually and averaged across these electrodes for further analysis. This two-stage selection process allowed for some interindividual variance in the exact peak amplitude location.
Applying Equation 1 yielded AMIs for each of the four shift types (C→C, O→O, C→O, O→C) and, from the perspective of a given RDK, both shift directions: For example, when shifting attention from red to blue, both red RDKs were attended preshift and unattended postshift, whereas blue RDKs were unattended preshift and attended postshift. AMIs were entered into a three-way repeated-measures ANOVA with factors of Shift direction (away vs. toward), Shift origin, and Shift destination (both color vs. orientation). Note that the ANOVA was based on absolute AMI values because we were interested in the magnitude of shift-induced attentional modulation rather than its direction (in terms of enhancement vs. suppression). Where appropriate, we further compared magnitudes of systematic main effects.
Averaged artifact-free EEG epochs were detrended within the interval of 2000-msec precue to 2400-msec postcue and subjected to Gabor energy filters (Marroquin, Harmony, Rodriguez, & Valdes, 2004) with a spectral bandwidth of ±0.74 Hz (temporal resolution = ±300 msec) centered at the stimulation frequencies of 12.5 and 15 Hz, respectively. In this way, we obtained the SSVEP time course of the four shifting conditions, namely, color to color (C→C), color to orientation (C→O), orientation to orientation (O→O), and orientation to color (O→C). In the next step, we subtracted attended-to-unattended from unattended-to-attended time courses for each shifting condition to obtain a measure of the selectivity of stimulus processing (Andersen & Müller, 2010). Before subtraction, time courses were divided by the absolute maximum amplitude of the interval of 2000-msec precue to 2400-msec postcue. This normalization procedure accounted for differences in absolute amplitude between 12.5- and 15-Hz SSVEPs. As depicted in Figure 3, if SSVEP amplitude time courses follow the shift of attention (Figure 3A), the difference curve will result in a sigmoidal shape (Figure 3B).
Previous studies have typically tested these time courses against a precue baseline during which participants' attention was allocated to a neutral position or feature to estimate shift onset characteristics (Kashiwase, Matsumiya, Kuriki, & Shioiri, 2012; Andersen & Müller, 2010; Müller, 2008; Müller et al., 1998). In our case, it was not feasible to define such a baseline because participants attended to specific features before and after the cue. Selecting a particular precue or postcue time window as a baseline would have been arbitrary and could have biased our results toward effects originating from selective attention to either feature (dimension). We thus opted for a different approach and took the first-order derivative of each selectivity time course (S′; see Figure 3B) for each participant and condition. In contrast to the original S time courses, S′ indicates the change in stimulus selectivity measured at each time point. Critically, S′ should be close to zero as long as one stimulus is attended but should deviate substantially from zero during a shift from attention to one feature to attention to another because S is changing (see Figure 3B). After the shift, S′ should return to zero because, now, participants attend to the newly cued feature. These characteristics allowed a statistical assessment of shift onset times in each condition: Using a 99% confidence criterion, we tested when S′ started to deviate from zero significantly.
Behavioral Data Analyses
Button presses made between 350 and 1000 msec after target onset were considered hits. Similar responses after distractor onset were considered false alarms. Given the equal distribution of targets in time across trials, we were able to reconstruct the time course of behavioral performance for each condition. To this end, RTs of correct responses were sorted into consecutive time bins of 67 msec each according to the onset times of corresponding targets. For example, when a response occurred to a target presented at time x, its corresponding RT was sorted into the bin that contained time x. An RT corresponding to a response at time x + 1 was sorted into the bin containing time x + 1. This procedure yielded RT time courses that were sampled at a rate of 15 Hz. We interpolated the time courses of these bins for each participant and condition to a sampling rate similar to EEG recordings (256 Hz) by means of a smoothing regression with a Nadaraya–Watson kernel (Nadaraya, 1964) to allow for the analysis of the RT time course with a better temporal resolution (Andersen & Müller, 2010).
On the basis of our previous study that investigated the time course of attentional shifts in feature-based attention (Andersen & Müller, 2010), we expected that, after the cue, participants responded much slower to the newly to-be-attended feature until attention is fully engaged to that feature. The obtained smoothed time courses were then subjected to a jackknife procedure to estimate individual peak latencies of RT increases after the shifting cue. Jackknifing has been shown to outperform peak latency estimation from typically noisy individual data (Smulders, 2010). To this end, N (where N = 18, number of participants) subaverage S′ time courses were computed by an iterative leave-one-out algorithm such that subaverage time course Ni was based on the averaged time courses of all participants excluding participant i. From each of these subaverages, we derived peak values that were subjected to a statistical correction method proposed by Smulders (2010) to estimate individual peak latencies. Peak latencies of the four conditions C→C, C→O, O→O, and O→C were compared by means of a two-way repeated-measures ANOVA with factors of Shift origin and Shift destination (both color vs. orientation).
In addition, we visually identified two periods of relatively stable RTs, one period before and one after the cue, from average time courses. Within each period, we collapsed RTs across attend-color and attend-orientation conditions for each participant. A two-way repeated-measures ANOVA with factors of Period (precue vs. postcue) and task-relevant Feature dimension (color vs. orientation) tested for systematic variations in performance.
Figure 2A depicts the grand mean topographical distribution for 12.5- and 15-Hz SSVEP amplitudes across the entire stimulation period and all trials. For both frequencies, occipito-central maxima are clearly visible. Figure 2B depicts grand-averaged power spectra for preshift and postshift time windows. Peaks correspond to the stimulation frequencies, respectively. As can be seen, SSVEP power was generally greater when either the color or the orientation of the driving RDK was attended. Peak amplitudes in both time windows indicated greater attention effects for color than for orientation resulting in greater modulation during shifts.
A three-way repeated-measures ANOVA on AMIs indeed revealed that attentional modulation depended on original and target features of shifts (main effect, Shift origin: F(1, 17) = 7.55, p = .01, η2 = 0.04; main effect, Shift destination: F(1, 17) = 9.98, p < .01, η2 = 0.10). In both cases, modulation was greater when color was the relevant feature (see Figure 4). Put differently, attentional modulation was greatest when participants shifted from one color to another and smallest when they shifted from one orientation to another. Shifts between feature dimensions (C→O, O→C) led to intermediate modulations. In addition, we found that attentional modulation differed between shifts toward and away from features (main effect, Shift direction: F(1, 17) = 11.10, p < .005, η2 = 0.06), with shifts away leading to greater modulation (see Figure 4). This is well in line with an earlier study reporting greater suppression for unattended stimuli than facilitation for attended stimuli (Andersen & Müller, 2010). Interactions of factors were negligible (all Fs(1, 17) < 2.42, ps > .14). The benefit, that is, greater magnitude, in attentional modulation when participants shifted from color instead of orientation (main effects of Shift origin) was comparable with the benefit when participants shifted toward color (main effect of Shift destination) instead of orientation (t(17) = −0.92, p = .37). Further specific contrasts of AMIs against zero demonstrated that attentional modulation was substantial in all cases depicted in Figure 4 (all |ts(17)| > 4.22, ps < .001).
Figure 5A and B depict time courses of SSVEP amplitudes for 12.5- and 15-Hz RDKs in all four shifting conditions, respectively. For three shifting conditions C→C (R→B), C→O (B→\\), and O→C (//→R), we obtained the expected sigmoidal morphology (compare Figure 3A with Figure 5A and B). However, only small shifting effects were visible when participants shifted attention from one orientation to another (i.e., O→O that is \\→//).
Figure 5C shows the time courses of selectivity (unattended-to-attended shifts minus attended-to-unattended shifts) that resulted in the time courses of the first-order derivative (S′) displayed in Figure 5D. Time courses are displayed again, separated by condition, in Figure 6 additionally showing 99% confidence bounds that indicate significant deviations from zero after cue presentation. During C→C shifts, attentional modulation started as early as 145 msec after cue presentation. Onsets of C→O shifts and O→C shifts occurred over 100 msec later at 271 and 246 msec, respectively. Surprisingly, O→O shifts did not show the expected morphology, which disallowed retrieving a shift onset time for this condition. Note the absence of substantial changes in SSVEP amplitude during O→O shifts (Figure 6C).
As can be seen in Figure 7A, RTs to targets that occurred around the time of the shifting cue showed an increase in all conditions—even as a consequence of O→O shifts that did not result in the expected changes in neural processing as laid out above. Latency analyses of RT maxima in smoothed time courses showed that timing only depended on the Shifting destination (i.e., toward color or orientation; F(1, 17) = 14.10, p < .005, η2 = 0.20) but not from which feature the shift originated (main effect, Shift origin: F(1, 17) < 1). Furthermore, we found no significant interaction between the two factors (F(1, 17) = 2.14, p = .16, η2 = 0.04). As can be seen in the bar graphs depicted in Figure 7B, shifts to color were faster than shifts to orientation. This was largely confirmed by post hoc pairwise comparisons of RT peak latencies (Table 1).
|Comparison .||Differencea .||t(17) .||p .|
|C→C vs. O→C||21 msec||−0.84||.41|
|O→O vs. C→O||33 msec||−1.35||.19|
|C→C vs. O→O||55 msec||−2.98||<.01*|
|O→C vs. C→O||68 msec||−2.52||.02*|
|C→C vs. C→O||89 msec||−3.60||<.005*|
|O→O vs. O→C||35 msec||1.40||.18|
|Comparison .||Differencea .||t(17) .||p .|
|C→C vs. O→C||21 msec||−0.84||.41|
|O→O vs. C→O||33 msec||−1.35||.19|
|C→C vs. O→O||55 msec||−2.98||<.01*|
|O→C vs. C→O||68 msec||−2.52||.02*|
|C→C vs. C→O||89 msec||−3.60||<.005*|
|O→O vs. O→C||35 msec||1.40||.18|
Asterisks mark significant results. C = color; O = orientation.
Absolute mean values.
As depicted in Figure 7A, RTs were relatively stable during a period from −1.2 to −0.2 sec and another period from 0.8 to 1.8 sec relative to the shifting cue. Response speed differed systematically between these precue and postcue periods (main effect, Period: F(1, 17) = 23.22, p < .001, η2 = 0.13). Furthermore, RTs varied between color and orientation targets (main effect, Feature dimension: F(1, 17) = 29.67, p < .001, η2 = 0.43). We found no significant interaction between both factors (F(1, 17) = 1.80, p = .20, η2 < 0.01). Bar graphs in Figure 4C illustrate the main effects. In general, participants responded faster to coherent motion targets when they attended to a color as compared with when they attended to an orientation. Second, RTs were generally faster in the period after the shifting cue.
In this study, we were interested in temporal dynamics of cortical facilitation during cued shifts of feature-selective attention. Contrary to our previous studies, in which we cued participants to shift attention after a neutral baseline period (Andersen & Müller, 2010; Müller et al., 1998), here, participants were cued to shift attention toward either color or orientation of flickering bars while they were already attending to either another color or orientation. Our design allowed us to obtain an objective electrophysiological measure of early visual cortex activity and their temporal dynamics as a function of time after shifting cue onset. In conjunction with the time course of electrophysiological data as measured with SSVEP amplitudes, we obtained a time course of behavioral performance: Participants were instructed to detect short coherent motion events in the to-be-attended RDKs and to press a button, while ignoring such events in the to-be-ignored RDKs.
Our study was motivated by two accounts for temporal dynamics for such shifting processes in feature-selective attention. On one hand, monkey intracranial (Buffalo et al., 2010) and human noninvasive recordings (Martinez et al., 2001) hinted toward a top–down modulated backward progression of attentional facilitation from higher-order to lower-order visual areas, which would result in an earlier neural facilitation when participants shifted toward color, regardless of its origin compared with when they were cued to shift to orientation. On the other hand, based on singleton feature search, the DWA predicts that shifting within a feature dimension (such as from color to color) is less time consuming compared with shifts across feature dimensions (such as from orientation to color; cf. Müller et al., 1995, 2003).
Our results nicely replicated findings from our previous studies that SSVEPs serve as a powerful tool to investigate neural temporal dynamics (Andersen & Müller, 2010; Müller, 2008; Müller et al., 1998) in early visual cortical areas, where the generators of SSVEPs were consistently found (cf. Keitel, Andersen, Quigley, & Müller, 2013; Andersen & Müller, 2010; Di Russo et al., 2007; Müller, Teder, & Hillyard, 1997). Shifting attention toward a stimulus (or two RDKs as in the present experiment) resulted in a significant amplification, and this was true with respect to color and orientation. Similar to our recent shifting experiment (Andersen & Müller, 2010), and to some extent, also visible in Figure 5, reduction in SSVEP amplitudes after the withdrawal of attention was significantly greater compared with amplification when participants shifted attention toward RDKs. Therefore, we can assume that suppression of the to-be-ignored RDKs is greater compared with facilitation of the to-be-attended RDKs. Overall, our electrophysiological as well as behavioral results are supportive for a backward progression of top–down modulated shifts in feature-selective attention. Shifting toward color, regardless of its origin (color or orientation), resulted in earlier and more pronounced cortical facilitation of the newly to-be-attended color after the onset of the cue compared with when participants shifted away from color toward orientation. Behavioral data paralleled our electrophysiological results. RT maxima, that is, the expected increase in RTs as a consequence of the shifting cue, showed earlier peak latencies for shifts toward color, again regardless of its origin.
At first glance, our interpretation of earlier color shifts in terms of reversely progressing attention effects appears based on an outdated dichotomy of cortical area V1 processing stimulus orientation and V4 stimulus color (Zeki, 1993; Hubel & Wiesel, 1968). This conception has long been replaced by vast evidence that neurons selective to either feature populate both visual cortices as well as intermediate areas (cf. Roe et al., 2012). A number of studies have documented robust color-selective responses in primary visual cortex (cf. Wachtler, Sejnowski, & Albright, 2003; Johnson, Hawken, & Shapley, 2001). Human neuroimaging work suggests that stimulus color can be decoded in primary visual cortex as well (cf. Brouwer & Heeger, 2009). Vice versa, orientation selectivity has been documented in V4 (cf. Desimone & Schein, 1987), and attention to orientation modulates orientation-selective units in V4 (Roe et al., 2012; McAdams & Maunsell, 2000). Nevertheless, the present results reconcile readily with the established similarity in color and orientation selectivity across visual cortices. Two reasons speak in favor of our interpretation: our particular stimulus situation and the respective functional characteristics in V1/V2 and V4. First, progressing forward through the visual hierarchy, receptive field sizes of neurons increase from <1° (visual angle) in V1 to 4°–10° in V4 (Roe et al., 2012). As a result, orientation selectivity in V4 neurons with its large receptive fields was linked to figure-ground segregation in object processing (Roe et al., 2012) and orientation constancy, also needed for object processing (Vanduffel, Tootell, Schoups, & Orban, 2002). Given the minute size of single dashes (<0.5°) in our stimulus display, their orientation was likely extracted at very early stages of stimulus processing, supported by the finding that neurons with small receptive field size were highly orientation selective in V1 and those with larger receptive fields were not (Gur, Kagan, & Snodderly, 2005). Furthermore, Vanduffel et al. (2002) reported of a gradient in the proportion of orientation-selective cells in V1 as a function of retinotopic eccentricity: Portions of V1 that represented more parafoveal regions of the visual field contain a higher proportion of orientation-selective neurons (∼70%), whereas foveal regions only contain 40% (Zeki, 1993). Therefore, our experimental stimulation (circular RDKs with a diameter of 13° of visual angle) resulted in the fact that most dashes fell in parafoveal regions that promoted orientation-selective rather than color-selective processing in early visual cortex. Second, one can argue that stimulus color could only be extracted in higher visual cortices with larger RF sizes that allowed for integration of color information across dashes. To some extent, this is supported by the finding that V1 exhibited almost no attentional modulation with colored stimuli, but attentional modulation of the V4 measured BOLD response was highly significant (Kastner et al., 2001; Kastner & Ungerleider, 2001). In these studies, the authors argued that receptive field sizes in V1 are too small to introduce competitive interactions between the colored stimuli, ergo no attentional modulation. However, if V1 would significantly contribute to color processing, attending to a colored stimulus should also increase the BOLD response compared with when that stimulus needs to be ignored. In summary, framing our interpretation in terms of a backward progression of attentional gain effects from “color-selective” V4 to “orientation-selective” V1 is thus qualified by a more intricate interplay of specific neuronal organization and our stimulus situation rather than an overcome modular notion of visual cortex function.
Surprising to us, we were unable to analyze the onset of shifting for shifts from orientation to orientation, despite the significant attention effects we found in attentional gain modulation (Figure 4). Here and in two previous studies in which we used bars that conjoined color and orientation, we found highly significant greater attention effects for SSVEP amplitudes for color compared with orientation (Andersen et al., 2008, 2015). Very likely, this might be due to the well-documented finding of greater attention effects at higher stages of stimulus processing (Bles, Schwarzbach, De Weerd, Goebel, & Jansma, 2006; Kastner, De Weerd, Desimone, & Ungerleider, 1998) such as in V4 compared with V2 or V1 (Kastner & Ungerleider, 2000; Kastner et al., 1998). In the light of our finding, we conclude that pronounced attention effects seem to be imperative for analyzing feature shifts by means of SSVEP amplitude time courses. Future studies may test different conjunctions such as color and shape that may lead to more comparable amplitude modulations during shifts. Nevertheless, the consistent finding of significantly smaller attention effects in SSVEP amplitudes for orientation compared with color demonstrates that at least color is a very powerful and efficient feature for attentional selection. Interestingly, in a recent study, we found that color modulated SSVEP amplitudes identically as spatial attentional selection (Andersen, Fuchs, & Müller, 2011).
The efficiency of color cues in feature-selective attention is also documented by the fact that detection of coherent motion events for color targets was easier compared with orientation targets, as defined by faster RTs for color targets before and after the shift. This parallels very nicely findings from studies that used odd-one-out displays in visual search (Krummenacher, Müller, & Heller, 2002; Found & Muller, 1996) that also reported faster RTs for color. Furthermore, a number of studies reported asynchronous processing of color compared with other features, such as motion or form (cf. Bartels & Zeki, 2006; Viviani & Aymoz, 2001; Moutoussis & Zeki, 1997). Is it now possible that our results are confounded by task difficulty because color is more salient compared with orientation? In other words, was a shift toward color targets linked with less effort because it was the easier task compared with orientation targets? We consider that as very unlikely. First, in a previous study, we have shown that effort or task difficulty did not modulate SSVEP amplitudes (Hindi Attar & Müller, 2012). Second, one would assume that a shift from a putative difficult task to an easier one (i.e., O→C) needs to result in an earlier peak latency of RTs compared with a shift from a putative easy-to-easy (i.e., C→C) or difficult-to-difficult (i.e., O→O) task. However, as listed in Table 1, this was not the case because these two comparisons resulted in insignificant differences. To us, it is more plausible that the observed latencies in SSVEP amplitudes are a result of the observed backward progression and earlier facilitation of color-related areas also related in earlier peaks in the RT maximum. Given the potential power of color as an attentional selection cue, it might be difficult to create displays for color and orientation that result in identical behavioral results because all orientations we used in our previous studies operated on a maximum orientation difference of 90°, that is, are orthogonal to each other. On the other hand, manipulating color hues might result in RDKs that are no longer isoluminant. Another possibility would be to change target durations of the two dimensions. That, however, would violate the basic requirement that all conditions need to be physically identical.
There remains the question, to what extent did the deliberately unbalanced design of our study contribute to the results. In particular, in the color-to-color condition, participants always shifted from red to blue, disallowing analyses of vice-versa shifts from blue to red. However, we consider a systematic bias in terms of an easier shift away from red as very unlikely. In fact, based on a recent study in visual search, one would even expect the opposite. Fortier-Gauthier and colleagues reported that red is special in visual search (Fortier-Gauthier, Dell'Acqua, & Jolicoeur, 2013). They presented a color cue (red or green) 1000 msec before the onset of a stimulus (presented for 400 msec) that consisted of 10 circles, evenly distributed in a circular fashion with two circles being red and green, respectively. They found shorter latencies of the so-called N2pc component for red compared with green targets, leading them to the conclusion that there exists a “red-alert effect in visual search.” If that would be the case, there might be the possibility that a shift away from red might be quite difficult, but a shift to red might be quite easy, resulting in different shifting times. If that were the case, one would expect even faster shifting times for shifts from blue to red. However, in a recent study, we cued participants to shift attention to either a red or blue RDK and found no differences in shifting time between shifts to red or blue (Andersen & Müller, 2010). In addition, even in the visual search literature, such a “red-alert effect” was not reported before, although these studies used a similar design with sudden-onset stimuli of red and a different color such as green or blue as in the study by Fortier-Gauthier and colleagues. This is true with respect to behavioral (cf. Müller, Geyer, Zehetleitner, & Krummenacher, 2009; Müller & Krummenacher, 2006a, 2006b; Krummenacher et al., 2002) and electrophysiological (Gramann, Töllner, & Müller, 2010) data. Another argument that speaks against such a bias in shifting times between red and blue is our finding of a much later onset of shifts when participants shifted from orientation to red. Again, a condition that would cue participants to shift from orientation to blue would result in an even later onset of cortical facilitation after shifting cue onset if a “red-alert effect” would exist and therefore in line with the basic results of our experiment. Last, but not least, there are substantial general design differences between the present and typical visual search experiments. Here, we presented all four RDKs for several seconds. At the time point of the presentation of the shifting cue, the whole stimulus was displayed for at least 2.400 msec and did not change with and after the cue. Therefore, the “system” is already activated by the relevant features of our stimuli and is not activated by a sudden onset that is required in designs that measure ERPs. SSVEPs need some time to develop after the onset of a flickering stimulus (about 400–500 msec, depending on the stimulation frequency). Therefore, the entire feed-forward and feedback loop, which is required to process the relevant stimuli, is established with the occurrence of the oscillatory signal. Under such stimulation conditions, only recently, we were able to show that attentional selection of color (red/blue) and orientation (vertical/horizontal) operates parallel and independently (Andersen et al., 2015). In other words, our results clearly pointed to the direction that the feature dimensions of color and orientation were not operating on a common resource pool that needed to be shared for attentional selection. On the contrary, different feature dimensions appeared to draw on entirely independent resource pools.
Importantly, recent and our present results put some questions on the generalizability of basic predictions from DWA that assumes a shifting of attentional weights from one feature dimension to another, which would be identical for shifts from color to orientation and vice versa (cf. Müller et al., 1995, 2003). However, we found that the shifting onset for C→O was earlier compared with O→C in SSVEP amplitudes that was also paralleled in behavioral data with a significant peak latency difference (see Table 1). In addition, as listed in Table 1, for example, C→C compared with O→C, peak latencies did not differ significantly, and the same was true for O→O compared with C→O and O→O compared with O→C peak latencies, respectively. Future research is needed to directly compare such cross-dimensional shifting in an odd-one-out design and in a paradigm as the present one.
To summarize, we studied neural dynamics in human early visual cortex during cued shifts of feature-selective attention that have not been addressed with electrophysiological measures before. Again, we demonstrated the power of SSVEPs as an objective electrophysiological tool to investigate neural temporal dynamics of attentional shifting. Attending to a feature led to cortical facilitation of the stimuli that carried that feature, whereas ignoring a feature resulted in significantly reduced processing, signifying suppression of that feature. Shifts toward color were always faster compared with shifts toward orientation, regardless of the initially to-be-attended feature. Therefore, our results are more in line with the finding of a reversed progression of top–down attentional modulation.
We thank Renate Zahn for help with data collection. This work was supported by the Deutsche Forschungsgemeinschaft (MU 972/20-1 and, in parts, by DFG Graduiertenkolleg “Function of attention”).
Reprint requests should be sent to Matthias M. Müller, Institute of Psychology, University of Leipzig, Neumarkt 9-19, 04109 Leipzig, Germany, or via e-mail: firstname.lastname@example.org.
Note that this is necessarily a simplified account of the much more intricate distribution of color and orientation-selective neurons across visual cortices from V1 up to V4 (for the complexity of v4, see, e.g., Roe et al., 2012). We further elaborate on this notion in the Discussion section and argue that our paradigm promotes stimulus processing in a way as if this were the case.