Abstract

In an explorative study, we investigated the time course of attentional selection shifts in feature-based attention in early visual cortex by means of steady-state visual evoked potentials (SSVEPs). To this end, we presented four flickering random dot kinematograms with red/blue, horizontal/vertical bars, respectively. Given the oscillatory nature of SSVEPs, we were able to investigate neural temporal dynamics of facilitation and inhibition/suppression when participants shifted attention either within (i.e., color to color) or between feature dimensions (i.e., color to orientation). Extending a previous study of our laboratory [Müller, M. M., Trautmann, M., & Keitel, C. Early visual cortex dynamics during top–down modulated shifts of feature-selective attention. Journal of Cognitive Neuroscience, 28, 643–655, 2016] to a full factorial design, we replicated a critical finding of our previous study: Facilitation of color was quickest, regardless of the origin of the shift (from color or orientation). Furthermore, facilitation of the newly to-be-attended and inhibition/suppression of the then to-be-ignored feature is not a time-invariant process that occurs instantaneously, but a biphasic one with longer time delays between the two processes. Interestingly, inhibition/suppression of the to-be-ignored feature after the shifting cue had a much longer latency with between- compared to within-dimensional shifts (by about 130–150 msec). The exploratory nature of our study is reasoned by two limiting factors: (a) Identical to our precursor study, we found no attentional SSVEP amplitude time course modulation for orientation, and (b) the signal-to-noise ratio for single trials was too poor to allow for reliable statistical testing of the latencies that were obtained with running t tests of averaged data.

INTRODUCTION

Processing sensory information for everyday adaptive behavior like in visual search requires that attentional selection is shifted from one element to the next. Such shifts might result in crossing dimensional borders, such as from color to a certain shape, or stay within a feature dimension, that is, shifting from one color to a different one. However, what are the temporal neural dynamics underlying those attentional shifts in feature selection?

One conceptual idea is framed in the dimensional weighting account (DWA; Müller, Reimann, & Krummenacher, 2003). It suggests that attention can be directed along certain feature dimensions, such as color, orientation, or motion. That is, an attentional weight is assigned to one such dimension thereby boosting its constituent features compared to those of another dimension. Consequently, shifting attention between features of the same dimension should be faster than shifting attention between dimensions; for example, shifts from red to blue items are faster than shifts from red to horizontal items. The theory is backed up by evidence from behavioral search and dual-object paradigms (Müller & Krummenacher, 2006a, 2006b; Müller & O'Grady, 2000; Müller, Heller, & Ziegler, 1995). Another account is based on neural data from monkey intracranial recordings suggesting that attention progressively projects back from higher order to lower order areas (Buffalo, Fries, Landman, Liang, & Desimone, 2010). In their study, the onset of neural responses after top–down shifts was registered first in V4, then V2, and last in V1 when monkeys were presented with differently colored Gabor patches. Given that color is processed further up in the visual stream, that is, V4 (Allison et al., 1993; for a review of V4 functionality, see Roe et al., 2012; Zeki, 1983) and V8 (Hadjikhani, Liu, Dale, Cavanagh, & Tootell, 1998), than, for example, orientation in V1 (Hubel & Wiesel, 1968), shifting attention toward color should thus be faster than shifting attention toward an orientation independent of to what attention was directed before the shift. Support for such a reversed progression mechanism was also found in humans with spatial attention (Martínez et al., 2001).

In light of these competing accounts, our laboratory previously set out to address neural temporal dynamics of feature-selective attention (Müller, Trautmann, & Keitel, 2016) by taking advantage of steady-state visual evoked potentials (SSVEPs), a neural signature generated in early visual cortex such as V1, V2, and V3 including areas V4/8 or hMT (Andersen & Müller, 2010; Müller et al., 1998; Morgan, Hansen, & Hillyard, 1996), elicited by flickering stimuli, such as random dot kinematograms (RDKs). Therefore, SSVEPs serve as a powerful tool to investigate neural temporal dynamics of feature-based attentional shifts. In that previous study, participants had to shift attention toward a color or orientation from another already attended color or orientation in a display of four superimposed flickering RDKs with different colors (blue and red) and orientations (slashes and backslashes), respectively. Results showed that attentional SSVEP amplitude modulation was greater and occurred earlier for shifts toward color compared to orientation, which was complemented by faster RTs when shifting attention toward color compared to orientation. Unfortunately, electrophysiological data for orientation were weak; for example, time course analyses for orientation to orientation shifts remained inconclusive as no significant attentional modulation over time could be observed. Furthermore, the design was unbalanced as a deliberate tradeoff between the possibility to analyze behavioral data and a sensible recording time for participants. That compromise, however, limited analyses of the electrophysiological data to a subset of the presented conditions, and precluded a full-fledged picture of the temporal dynamics of all feature shifts that were involved.

Contrary to our 2016 study, this study has some important expansions: We set up a fully balanced design that would allow us to more conclusively look into the temporal neural dynamics of attentional shifts within and/or between feature dimensions. Given our unbalanced design in the 2016 study, we were not able to look into the time course of neural competitive interactions of the newly to-be-attended feature and inhibition/suppression of the to-be-ignored feature after the shifting cue. There is ample empirical evidence that to-be-ignored stimuli will be suppressed: See, for example, in biased competition (Desimone & Duncan, 1995), in spatial attention with regard to the to-be-attended and the to-be-ignored location as signified by the P1 and N1 component of the visual evoked potential (Hillyard, Vogel, & Luck, 1998), and in feature-based attention as well (Forschack, Andersen, & Müller, 2017; Andersen & Müller, 2010; Maunsell & Treue, 2006). However, what is unknown at present is the temporal relation between facilitation of the to-be-attended and suppression of the to-be-ignored feature/stimulus. Almost all attentional models assume that shifting attentional resources to a stimulus/location has the immediate effect of attentional resources being withdrawn from another stimulus/location because they are limited. As a consequence, facilitation and suppression should be time invariant and occur instantaneously as pouring water from one glass into another one. However, our previous studies demonstrated that facilitation and inhibition/suppression rather follow a biphasic profile, with early facilitation of the to-be-attended and time-delayed inhibition/suppression of the to-be-ignored color (Brummerloh & Müller, 2019; Andersen & Müller, 2010). Frequency tagging and the analysis of SSVEP time courses provide us with a unique tool to investigate these temporal neural dynamics. Given these results, we were curious to find out whether (a) such a biphasic process also exists when attention is already deployed to one feature within a dimension (i.e., shifting attention from color to color), given that DWA would predict a time-invariant process of facilitation/suppression in within-dimensional shifts, and (b) whether the possible biphasic modulation is a time-fixed process, or whether we find longer latencies in facilitation and inhibition/suppression when attention needs to be shifted between feature dimensions (such as from color to orientation). As far as we are aware of, nobody has ever looked into these dynamics. Furthermore, given our surprising finding of no attentional SSVEP amplitude modulation for orientation in our previous study in which we used slashes and backslashes, we turned back to horizontal and vertical bars because we were able to show SSVEP amplitude modulation with horizontal/vertical bars in previous studies (Andersen, Müller, & Hillyard, 2015; Andersen, Hillyard, & Müller, 2008).

METHODS

Participants

Thirty young and healthy adults (18–37 years, mean age = 24 ± 5 years, 18 women, 27 right-handed) with normal or corrected-to-normal vision were recruited from existing databases at the University of Leipzig. The sample size was determined by an a priori power analysis on the minimal effect sizes from our previous experiment (Müller et al., 2016) with a power of 0.8 and an α error probability of 0.05 using G*Power (Faul, Erdfelder, Buchner, & Lang, 2009). We decided to test 30 participants, because the calculated required minimum sample sizes ranged between 21 and 26.

Informed written consent was obtained before the experiment. The study was designed and conducted according to the declaration of Helsinki and was approved by the local ethics committee. All participants received either course credit or monetary compensation of €8 /hr. Two participants were excluded from data analysis because of technical errors with graphics and timing during stimulus presentation.

Materials

We used four superimposed flickering RDKs centered at a fixation cross in the middle of the screen on a gray background (luminance = 6 cd/m2). Each RDK consisted of 75 bars randomly distributed across a circular area spanning 13° of visual angle. Single bars were 0.44° × 0.1° of visual angle, moved randomly by 0.04° per frame, and were drawn in random order to prevent depth cues induced by a systematic superposition of one RDK with another. The four RDKs consisted of unique combinations of the feature dimensions color and orientation and flickered at a different frequency each to allow for the analysis of distinct SSVEPs: blue horizontal (10 Hz), red horizontal (12.5 Hz), blue vertical (15 Hz), and red vertical bars (17.5 Hz, see Figure 1A). The central fixation cross served as a cue to indicate the feature that had to be attended: red, blue, horizontal, and vertical (orientation cues were realized by a black bar crossing a white bar). Consequently, participants always had to attend to two RDKs at the same time; for example, when cued with a blue fixation cross, blue horizontal and vertical bars (10 and 15 Hz) needed to be attended, and when vertical was cued, red and blue vertical bars (15 and 17.5 Hz) needed to be attended, and so on.

Figure 1.

Experimental design. (A) Stimulus display of the four overlapping RDKs, corresponding frequency tags, and possible attention cues on the right; multiple shift conditions form four major conditions of interest: shifts from color–color (C-C), orientation–orientation (O-O), color–orientation (C-O), and orientation–color (O-C). (B) Time course per trial: fixation cross for 1400 msec, duration of RDK stimulation for 4800 msec, vertical dashed lines indicate jittered onset of the shifting cue for 2400–2800 msec. First cue (exemplary in red) fades over to second cue (exemplary in blue) over 200 msec to prevent an ERP.

Figure 1.

Experimental design. (A) Stimulus display of the four overlapping RDKs, corresponding frequency tags, and possible attention cues on the right; multiple shift conditions form four major conditions of interest: shifts from color–color (C-C), orientation–orientation (O-O), color–orientation (C-O), and orientation–color (O-C). (B) Time course per trial: fixation cross for 1400 msec, duration of RDK stimulation for 4800 msec, vertical dashed lines indicate jittered onset of the shifting cue for 2400–2800 msec. First cue (exemplary in red) fades over to second cue (exemplary in blue) over 200 msec to prevent an ERP.

Procedure

Participants were seated 80 cm in front of a 19-in. cathode ray tube screen (640 × 480 pixel resolution, 32-bit color depth, 120-Hz refresh rate) inside an acoustically dampened and electromagnetically shielded chamber. First, each participant individually adjusted the colors red and blue to isoluminance with the gray background by means of heterochromatic flicker photometry (Wagner & Boynton, 1972).

Before and between trials, a white fixation cross was shown for 1400 msec that served as time to blink and preparation for the trial to reduce eye movements during stimulation. Participants were instructed to fixate at the fixation cross, not move their eyes and to avoid eye-blinks during the stimulation period. Each trial started with the four RDKs and the central cue signaling the to-be-attended feature for a jittered period of 2400–2800 msec. Afterward, the cue changed and indicated the new feature to which participants had to shift their attention. The jittered time interval before the change of the shifting cue was introduced to avoid temporal expectancy effects. The shifting cue was not immediate but faded over for 200 msec to prevent a change-induced ERP that would interfere with the SSVEP analyses. New cue and RDK stimulation continued for 2000–2400 msec, respectively, so that the total duration of a trial was always 4800 msec (see Figure 1B). RDKs were phase-locked at cue onset and the shifting cue.

The whole experiment comprised 576 trials, distributed across 12 blocks of 48 trials each (∼6-min duration). We defined four major shift conditions of interest (two within and two between dimensions): color–color (C-C), orientation–orientation (O-O), color–orientation (C-O), orientation–color (O-C). Given our stimulus set, each of these major shifting directions consisted of more specific conditions (12 in total); within-dimension shifts comprised two, between-dimension shifts comprised four conditions, for example, C-C: red-blue, blue-red; C-O: red-horizontal, red-vertical, blue-horizontal, blue-vertical (see Figure 1A for all conditions). Because of this, within-dimension shift conditions were presented twice as often (72 trials each) as between-dimension shift conditions (36 trials each), to match the numbers of trials for the four major conditions of interest (144 trials each).

The participants' task was to detect coherent diagonal motion events of the attended bars, upon which they were to press the space bar with their right hand (see below for individual settings). They were told that there could be either none or up to three such events per trial and that events could also occur at the unattended bars, which were to be ignored. Coherent motion events lasted 400 msec, starting earliest at 300 msec after trial onset, separated by at least 800 msec, and only occurred in 25% of the trials because they only served to ensure participants' compliance with the cue, and to have more event free trials for EEG analysis which was the focus of this study. Only responses made between 350 and 1000 msec after the event were included to calculate accuracy. Participants received feedback on their performance at the end of each block.

Before the main experiment, participants did a minimum of three training blocks of 24 trials each to familiarize with the task and to set individual difficulty to generate an average accuracy of 70% (in main experiment: 69 ± 11%). Difficulty was modulated by individually adjusting the percentage of bars within one RDK that coherently moved during an event (65–95%, mean = 86 ± 8%). During training, participants received auditory feedback on each event, and culminated feedback at the end of each training block.

EEG Acquisition and Preprocessing

We recorded EEG from 64 Ag/AgCl electrodes mounted in an elastic cap using an ActiveTwo amplifier system (BioSemi) with a sampling rate of 256 Hz. Vertical eye movements were monitored with two additional electrodes above and below the right eye, and lateral eye movements were captured by two electrodes placed on the external canthi. EEG data analysis was performed using the EEGLAB toolbox (Delorme & Makeig, 2004) and custom MATLAB scripts (The MathWorks). We extracted epochs from −2400 to 2600 msec relative to the shifting cue. All epochs containing events were excluded from analysis to prevent interference from ERPs to coherent motion, or from motor activity to button presses. EEG data were then detrended, and trials with blinks or eye movements exceeding a threshold of 25 μV were automatically rejected. We employed Statistical Control of Artifacts in Dense array Studies (Junghöfer, Elbert, Tucker, & Rockstroh, 2000) to identify and correct further artifacts; trials with more than 15 contaminated channels were excluded from further analysis. On average, 18% of trials were excluded per participant. Subsequently, data were rereferenced to average reference and detrended again.

Frequency Analyses

Fast Fourier transforms were separately calculated relative to the shifting cue for a preshift window (−2000 to 0 msec) and a postshift window (400 to 2400 msec) with zero padding to extend each electrode time series to 16,384 (= 214) points. The first 400 msec after the shifting cue were not included to avoid possible contamination by the ERP elicited by the cue. Data for all artifact-free epochs were averaged across these two time windows to obtain the scalp topographical distributions for the four SSVEP frequencies (see Figure 2). Oz was unequivocally the best electrode in all participants and all frequencies, which we consequently used in all further analyses.

Figure 2.

Scalp topographies and amplitude spectra for all SSVEPs. (A) Topographical maps for each RDK frequency averaged across conditions and pre- and postshift windows (data in B). The RDK-defining feature is depicted on top of each scalp map (e.g., the 10-Hz RDK consists of horizontal blue bars). Oz was the best electrode for all frequencies and conditions and was therefore used in all analyses. (B) Grand-averaged amplitude spectra of preshift (upper panel) and postshift (lower panel) windows obtained by fast Fourier transforms of the SSVEP waveforms per condition. Peaks are clearly visible at all four stimulation frequencies showing that SSVEPs were reliably elicited by the RDKs. The peak amplitude is always taken by the attended condition's driving feature, that is, for the 10-Hz RDK (blue horizontal bars as depicted on the very top), amplitudes are highest when blue or horizontal was cued, and likewise for all other cueing conditions.

Figure 2.

Scalp topographies and amplitude spectra for all SSVEPs. (A) Topographical maps for each RDK frequency averaged across conditions and pre- and postshift windows (data in B). The RDK-defining feature is depicted on top of each scalp map (e.g., the 10-Hz RDK consists of horizontal blue bars). Oz was the best electrode for all frequencies and conditions and was therefore used in all analyses. (B) Grand-averaged amplitude spectra of preshift (upper panel) and postshift (lower panel) windows obtained by fast Fourier transforms of the SSVEP waveforms per condition. Peaks are clearly visible at all four stimulation frequencies showing that SSVEPs were reliably elicited by the RDKs. The peak amplitude is always taken by the attended condition's driving feature, that is, for the 10-Hz RDK (blue horizontal bars as depicted on the very top), amplitudes are highest when blue or horizontal was cued, and likewise for all other cueing conditions.

Because we were interested in attentional shifts, we calculated an amplitude modulation index (AMI; Kastner, Pinsk, De Weerd, Desimone, & Ungerleider, 1999) between the absolute SSVEP amplitudes of the pre- (Apre) and postshift (Apost) time window per participant, condition, and RDK frequency based on the following formula:
$AMI=Apre−Apost/Apre+Apost$
(1)

The obtained values were then averaged across frequencies and single conditions to yield eight values of interest, that is, the four major shift conditions (C-C, C-O, O-O, O-C) per shift direction (away/toward). For a more detailed description, see our previous work (Müller et al., 2016, pp. 646–647). The resulting AMIs were then subjected to a repeated-measures ANOVA with the factors of Shift Origin (color/orientation), Shift Direction (away/toward), and Shift Destination (color/orientation).

Time–Frequency Analyses

To analyze the time course of each SSVEP amplitude, we applied Gabor filters (Gabor, 1947) to EEG epochs within the interval of −2400 to 2600 msec relative to the shifting cue, separately for each RDK frequency and the 12 single conditions (as a reminder, see Figure 1). The wider interval was used to attenuate edge artifacts of the Gabor filter, and we only used the frequency domain time course between −2000 and 2000 msec in all subsequent analyses. We employed a spectral bandwidth of ± 0.9-Hz FWHM (temporal resolution ± 245 msec) centered at the four RDK frequencies, respectively. Because absolute amplitudes differ between frequencies, we normalized the data by using entirely unattended time courses (EEGUA) as a baseline for frame-wise division with each time course of interest. For example, looking at the 10-Hz RDK (i.e., blue horizontal bars), there were two shift conditions where blue horizontal bars did not have to be attended in any part of the trial neither pre- nor postshift: When participants had to shift attention from red to vertical bars, or vice versa. The average of these two conditions (EEGUA) served as baseline for the 10-Hz RDK and was used for normalization of all other conditions where that RDK had to be attended in any one part of the trial (EEGA) according to the following formula:
$EEGnorm=EEGA/EEGUA−1$
(2)

This procedure was used analogously for all four RDKs. This way, we ensured that amplitudes were comparable across the different RDK frequencies and centered around zero when no attention was directed. Like the AMI, the resulting time courses were then averaged across frequencies and single conditions to obtain the four major shift conditions (C-C, C-O, O-O, O-C) per shift direction (away/toward). Because of the different shift directions, in half of the conditions, attention needed to be directed away from a given feature, and consequently, amplitudes were above zero preshift and returned to zero postshift. Therefore, we used the average of a 1000-msec preshift window (−1245 to −245 msec to exclude contamination from postshift effects based on Gabor filter size), and subtracted its average from the full time course first, so that we could test the postshift window against zero. Subsequently, we calculated running t tests against zero for postshift time courses (0 to 2000 msec) using a 95% confidence interval with undirected cluster corrections using 10000 permutations for multiple comparisons correction to identify shift onset times when amplitudes started to deviate from zero significantly.

RESULTS

Behavioral Analyses

Participants' average accuracy was 69% (± 11%), thereby approximating the difficulty level of 70% that was set during training. A one-way ANOVA revealed no significant differences between conditions, F(3, 116) = 0.997, p = .397; C-C: 71% ± 15%; O-O: 67% ± 9%; C-O: 67% ± 14%; O-C: 70% ± 12%. Thus, participants were compliant with the task (cue) and our training procedure guaranteed that all tasks were about equally difficult. Note that we focused on electrophysiological data and have therefore maximized the number of event free trials. Consequently, we did not perform any more advanced behavioral analyses as in the previous study (Müller et al., 2016) because of the limited amount of data.

Frequency Analyses

Grand mean topographical distributions for all four RDK frequencies across all trials are depicted in Figure 2A with Oz as the best electrode in all participants and all frequencies, which we consequently used in all further analyses. Grand mean amplitude spectra show that SSVEP amplitudes were present in all four RDK frequencies and were greater when the respective RDKs were attended compared to when they were not (see Figure 2B).

A three-way repeated-measures ANOVA on AMIs revealed that attentional modulation was differently affected by original and target features of the shifts as well as their irection (Shift Origin: F(1, 27) = 10.372, p = .003, η2 = 0.02; Shift Destination: F(1, 27) = 7.205, p = .012, η2 = 0.012; Shift Direction: F(1, 27) = 152.324, p < .001, η2 = 0.443; Shift Origin × Shift Direction: F(1, 27) = 26.162, p < .001, η2 = 0.048; Shift Destination × Shift Direction: F(1, 27) = 22.805, p < .001, η2 = 0.044). Specifically, attention to color resulted in the greatest modulations whereas attention to orientation only showed small modulations, and between-dimension shifts' modulations (C-O, O-C) were intermediate. Most strikingly, Figure 3 shows that when the shift origin was color (C-C and C-O conditions on the left), attentional modulation was considerably greater and less variable than when attention started at orientation (O-O and O-C on the right). As for the shift direction, attentional gain was higher for shifts away from features than for shifts toward them, all in line with our previous study (Müller et al., 2016). Unfortunately, attentional gain for shifts from orientation toward another orientation or color was so little that, again, it did not differ from zero as a result of one-sample t tests (toward O-O: t(27) = −1.134, p = .267; toward O-C: t(27) = −1.879, p = .071), but all other conditions did (|ts(27)| > 3, ps ≤ .005).

Figure 3.

Attentional gain by AMIs. Bars depict normalized changes in SSVEP amplitudes from preshift to postshift as calculated in Equation (1). Dots depict individual participants. Because of how the AMIs are calculated, facilitation is negative, so we flipped the y-axis for better visualization. AMIs were significantly different from zero for all conditions except for shifts from orientation toward another orientation or color (upper blue O-O and purple bars O-C). Note that shifts originating from color (left side: C-C and C-O) are considerably bigger and less variable than shifts from orientation (right side).

Figure 3.

Attentional gain by AMIs. Bars depict normalized changes in SSVEP amplitudes from preshift to postshift as calculated in Equation (1). Dots depict individual participants. Because of how the AMIs are calculated, facilitation is negative, so we flipped the y-axis for better visualization. AMIs were significantly different from zero for all conditions except for shifts from orientation toward another orientation or color (upper blue O-O and purple bars O-C). Note that shifts originating from color (left side: C-C and C-O) are considerably bigger and less variable than shifts from orientation (right side).

Time–Frequency Analyses

Figure 4 depicts the time courses for SSVEP amplitudes elicited by the four RDKs per condition, either shifting toward (top panel) or away from the respective RDK (bottom panel). Facilitation of the newly to-be-attended feature was fastest for shifts from color to color (C-C) starting 160 msec after the shifting cue, closely followed by O-C with 168 msec and then C-O with 309 msec. As for the AMIs, changes in SSVEP amplitude for O-O shifts over time were not substantial enough to become significant, and thus, we were not able to determine the time point at which SSVEP amplitudes changed significantly relative to the precue level. Conversely, inhibition/suppression of the then to-be-ignored feature commenced much later. As depicted in Figure 4 (lower panel), reduction of SSVEP amplitude below zero in C-C shifts was fastest with 426 msec, followed by O-C with 551 msec, and C-O with 582 msec. The significant reduction in SSVEP amplitude for O-O shifts at 660 msec needs to be treated with extreme caution, given the shallow slope of the entire time course. Nevertheless, it occurs that similar to our previous studies, shifting of attention from one feature to another is not a time invariant but again a biphasic process, with fast facilitation of the newly to-be-attended feature followed by inhibition/suppression of the then to-be-ignored feature. Furthermore, although the facilitation process did not exhibit a clear time difference between within compared to between dimensional shifts, the inhibition/suppression process was clearly much slower (by about 130 msec) for between dimensional shifts (note the limitation for O-O shifts).

Figure 4.

Time courses of SSVEP amplitudes. Amplitudes were normalized according to Equation (2). Shaded areas show 95% confidence intervals of consecutive t tests against zero. Bold lines indicate time points significantly different from zero (corrected for multiple comparisons with cluster corrections using 10000 permutations) with shift onsets denoted in corresponding colors. Note that, in the lower panel, amplitudes were originally above zero preshift and returned to zero postshift but were adjusted for better illustration and tests against zero (see Methods on Time–Frequency Analyses for more details).

Figure 4.

Time courses of SSVEP amplitudes. Amplitudes were normalized according to Equation (2). Shaded areas show 95% confidence intervals of consecutive t tests against zero. Bold lines indicate time points significantly different from zero (corrected for multiple comparisons with cluster corrections using 10000 permutations) with shift onsets denoted in corresponding colors. Note that, in the lower panel, amplitudes were originally above zero preshift and returned to zero postshift but were adjusted for better illustration and tests against zero (see Methods on Time–Frequency Analyses for more details).

DISCUSSION

In this study, we extended our precursor study (Müller et al., 2016), investigating temporal dynamics of neural facilitation after attentional selection shifts within or between different feature dimensions using frequency-tagged stimuli that elicit SSVEPs with cortical generators in early visual areas of the human brain (Andersen & Müller, 2010; Di Russo et al., 2007; Müller et al., 1998). With a full-factorial design, here, we were able to additionally measure the time course of neural inhibition/suppression of the to-be-ignored feature after the shifting cue. SSVEPs provided us with a unique tool given the ongoing oscillatory nature of the response. Furthermore, we intended to replicate our findings with regard to facilitatory dynamics for the newly to-be-attended feature. Our most important finding was a rapid facilitation and a delayed inhibition/suppression after the shifting cue. Thus, we replicated our previous study in which participants shifted attention to one of two colors in a red/blue RDK resulting in an early facilitation of the to-be-attended and delayed suppression (by about 130 msec) of the to-be-ignored color (Andersen et al., 2015). Rather than observing a time-invariant process of facilitation and inhibition/suppression as suggested by all of the attentional models, the observed biphasic time course suggests early sensory gain (facilitation) as proposed with conventional ERP studies or intracranial recordings in monkeys (see, e.g., Maunsell & Treue, 2006; Treue & Martínez Trujillo, 1999; Hillyard & Anllo-Vento, 1998; Hillyard et al., 1998) that is followed by competitive suppressive interactions as suggested by the “biased competition” account (Desimone & Duncan, 1995). Interestingly, the timing of that biphasic process depended on the featural quality of the shift. Shifting the attentional focus from one feature dimension to another (such as from color to orientation and vice versa) resulted in a much longer latency of the inhibitory/suppressive reduction in SSVEP amplitudes compared to when attention was shifted from color to another color (red to blue and vice versa). This difference in latency was in between roughly 130 and 150 msec. We also replicated our previous finding that facilitation of color-elicited SSVEP amplitudes was much faster compared to orientation regardless whether participants attended to color or orientation before the shifting cue. Interestingly, this temporal advantage also enumerated to roughly 140 msec.

Before we continue our discussion, we need to mention two limitations of our study. Unfortunately, we also replicated the null finding of attentional SSVEP amplitude time course modulation with regard to attentional shifts from orientation to orientation, although we changed the design to horizontal and vertical bars that previously resulted in significant amplitude effects (Andersen et al., 2008, 2015). This is insomuch hard to explain because we used the same frequencies as in the original study that showed an additive effect of color and orientation (Andersen et al., 2008). In our precursor study, we speculated that this might have been a consequence of receptive field size, the gradient of proportion of orientation-selective cells in V1 as a function of retinotopic eccentricity, and projection of electrical sources to the scalp surface in V1. We will not repeat the argumentation here and would like to refer the reader to the discussion in our 2016 paper (Müller et al., 2016). The only difference between the original and this study is the distribution of frequencies we used for our four RDKs. Whereas in the original study vertical and horizontal orientations were represented with a lower and a higher frequency (horizontal: 10 and 17.5 Hz; vertical: 12.5 and 15 Hz), in this study, both vertical bars were represented with the highest flicker frequencies (15 and 17.5 Hz). A well-known feature of SSVEPs is the reduction in absolute amplitude with increasing flicker frequency despite possible contributions of differences in cortical sources as in this study (see Figure 2). As a consequence, attention effects are tiny as well. Given the big variance between participants, our unfortunate assignment of 15 and 17.5 Hz to vertical bars might have resulted in that null effect and was not compensated by greater amplitudes for horizontal bars as in our first study that limited the interpretation of within-dimension shifting to color-to-color shifts. The 660-msec latency we found for inhibition/suppression of a previously attended orientation needs to be treated with extreme caution, given the shallow slope of the entire SSVEP amplitude time course and was, thus, not further considered in the interpretation of our data.

The second limitation is the fact that we were not able to test the different latencies statistically that were obtained by running t tests on the basis of averaged data. Just recently, we published a paper in which we tested different latencies of SSVEP amplitude amplification and alpha band modulation by means of Jackknifing-based single-subject estimates (Gundlach, Moratti, Forschack, & Müller, 2020). In addition, just recently, Antonov et al. (Antonov, Chakravarthi, & Andersen, 2020) used a resampling method for the same purpose and the same experimental question. We tried both methods, but results were neither reliable nor trustworthy. The two other studies had a spatial attention design with one frequency-tagged stimulus in the left/right visual hemifield, respectively. Thus, both SSVEP responses were clearly lateralized and separated compared to our situation here having all RDKs centrally superimposed upon each other. In addition, here, we needed to handle four rather than two frequencies. As a result, single-subject data are significantly more prone to noise compared to these two other studies and the time point estimate of a certain change in SSVEP amplitude (say 50% as in those two studies) is almost impossible or arbitrary at its best. Given these two limitations, we know that we need to consider our present results here as exploratory, although latency differences of about 130 msec and more are most certainly biologically significant, and it seems therefore implausible to us to argue with a time-invariant neural mechanism.

In the Introduction, we contrasted two possible mechanisms to explain different latencies of feature-based attentional shifts: the DWA (Müller et al., 2003) and top–down backward progression from higher (such as V4) to lower areas (V1, V2) in the visual processing stream (Buffalo et al., 2010). Contrary to the precursor study, here, we had color shifts to both colors and replicated faster facilitation of SSVEP amplitudes for color shifts substantiating our previous argumentation that those result were very unlikely a consequence of just having shifts from red to blue (see our discussion in Müller et al., 2016). Given the results of both studies, color facilitation follows more the proposed top–down backward progression given that shifts from color to orientation had a longer latency with regard to SSVEP amplitude augmentation. On the other hand, there is ample experimental evidence that color is a special feature having processing precedence over other features such as orientation and shape (Lee, Leonard, Luck, & Geng, 2018; Biderman, Biderman, Zivony, & Lamy, 2017; Geng, DiQuattro, & Helm, 2017; White, Lunau, & Carrasco, 2014; Krummenacher & Müller, 2012; Bartels & Zeki, 2006; Viviani & Aymoz, 2001; Moutoussis & Zeki, 1997; Found & Müller, 1996; Theeuwes, 1992). Given those findings, we cannot entirely exclude that the special standing of color in feature space supported rapid facilitation after attentional shifts toward color. Given the special role of color in feature space, future research should perhaps focus on other feature dimensions, such as motion and orientation for further understanding of neural dynamics in attentional shifts in feature space.

The most interesting result in this study, however, is that the long delay of inhibition/suppression of the then to-be-ignored feature rather follows the prediction of DWA with longer latencies for SSVEP amplitude inhibition/suppression for between- compared to within-feature dimensional shifts. Clearly, DWA offers no predictions regarding inhibition/suppression, but the observed neural temporal dynamics might have consequences on behavioral responses that would fit with behavioral DWA findings (see below).

It is striking that the respective latencies of the critical comparisons are very similar (facilitation of color at about 160–168 msec, regardless of the origin, inhibition/suppression between dimensional shifts for both dimensions at about 550–580 msec). Therefore, we are confident that we observed a general timing mechanism of neural competition in attentional selection shifts.

The observation of fast facilitation and slower inhibition/suppression has accumulating evidence from our previous studies. This biphasic process was first shown in a study where two overlapping red/blue RDKs were presented and after a neutral baseline a shifting cue indicated to what color participants needed to shift their attention (Andersen & Müller, 2010). The latency difference between facilitation and inhibition/suppression was at about 130 msec. Further support of such a biphasic process came from a recent study from our laboratory (Brummerloh & Müller, 2019) in which participants were cued to one of two features of a coherent object (rotating square that changes color). When participants either attended to color or rotation, again, we found fast facilitation of the SSVEP amplitudes of the to-be-attended feature followed by inhibition/suppression of the to-be-ignored one. In both studies, we speculated on the interaction between two well-known neural mechanisms of attentional prioritization.

While the fast enhancement of attended stimuli can be explained by a sensory gain mechanism (Maunsell & Treue, 2006; Treue & Martínez Trujillo, 1999; Hillyard et al., 1998) that boosts the signal strength of attended stimuli in early visual areas, later suppression of unattended stimuli can best be explained by an account that relies on competitive (suppressive) interactions as for example suggested in biased competition (Desimone & Duncan, 1995). Therein, stimuli contained in the same receptive field inhibit one another to gain neural representation and attention can bias that inhibition toward a given stimulus. Our display of four superimposed RDKs triggers strong competition. Whereas the sensory gain mechanism is believed to occur in lower visual areas, competitive interactions likely operate in higher visual areas (Maunsell & McAdams, 2001). Our data are in line with the idea that facilitation of attended stimuli in small receptive fields of lower visual areas is propagated to higher visual areas, where large receptive fields would lead to competition with and consequential inhibition/suppression of the unattended stimuli. As that information needs to travel back to lower visual areas, we can observe the reported lag. Such a feedback loop could also explain why the inhibition/suppression of a previously attended color after the shift to another color was about 130 msec faster compared to between-dimensional inhibition/suppression.

The time window, in which the to-be-attended feature is already enhanced but the previously attended feature is not yet entirely inhibited/suppressed, would be very interesting with regard to behavioral consequences. That is, is fast facilitation potent enough to allow participants to correctly identify changes in the newly attended RDK, or is inhibition/suppression the main player for correct responses and the avoidance of false alarms (i.e., reactions to the newly to-be-ignored stimulus)? Our original feature-based shifting study (Andersen & Müller, 2010) would hint that it is a mixture of both processes. We tested the time course of behavioral data and found a significant correlation with a selectivity measure of SSVEP amplitude time courses, that is, the time course of the to-be-attended minus the to-be-ignored color. Thus, it looks as if it is the “neural distance” between the two stimuli that is critical and, thus, similar to what would be predicted by signal detection theory (MacMillan, 2002). From that speculation, we would expect different time courses of behavioral data on the basis of the SSVEP amplitude selectivity measure, when comparing the respective conditions, what might lead to an answer for seemingly different findings from DWA (mostly based on behavioral data) and top–down feedback progression (from neurophysiological recordings). Given that our experimental design was trimmed for EEG data analyses, we do not have sufficient data to answer these questions but they form an intriguing starting point for new experiments.

Conclusion

In this study, we used SSVEPs to investigate temporal neural dynamics of attentional selection shifts in feature space. We found fast facilitation of SSVEP amplitudes after the cue for shifts toward color, regardless of the origin (color or orientation), what would be in line with the finding of top–down backward progression from higher to lower visual areas. Rapid facilitation of the newly to-be-attended feature was followed by slower inhibition/suppression of the then to-be-ignored feature. Shifts between dimensions had a longer latency for inhibition/suppression of SSVEP amplitudes compared to shifts from one color to the other. That pattern would be in line with DWA. Our results clearly indicate that shifting attention in feature space is not time invariant but exhibits a biphasic process of early sensory gain amplification followed by competitive interactions as proposed by biased competition. Future research should focus on the behavioral consequences of these neural temporal dynamics and might then be able to integrate the two seemingly opposing models.

Author Contributions

Paula Vieweg: Conceptualization; Data curation; Formal analysis; Investigation; Methodology; Project administration; Software; Visualization; Writing – original draft; Writing – review & editing. Matthias M. Müller: Conceptualization; Funding acquisition; Resources; Writing – original draft; Writing – review & editing.

Funding Information

Matthias M. Müller, Deutsche Forschungsgemeinschaft (http://dx.doi.org/10.13039/501100001659), grant number: MU 972/24-1.

Diversity in Citation Practices

A retrospective analysis of the citations in every article published in this journal from 2010 to 2020 has revealed a persistent pattern of gender imbalance: Although the proportions of authorship teams (categorized by estimated gender identification of first author/last author) publishing in the Journal of Cognitive Neuroscience (JoCN) during this period were M(an)/M = .408, W(oman)/M = .335, M/W = .108, and W/W = .149, the comparable proportions for the articles that these authorship teams cited were M/M = .579, W/M = .243, M/W = .102, and W/W = .076 (Fulvio et al., JoCN, 33:1, pp. 3–7). Consequently, JoCN encourages all authors to consider gender balance explicitly when selecting which articles to cite and gives them the opportunity to report their article's gender citation balance.

Acknowledgments

This work was supported by German Research Foundation (DFG) grant MU 972/24-1. The authors would like to thank Norman Forschack, Christopher Gundlach, and Christian Keitel for valuable discussions about experimental design and data analysis, and Renate Zahn for help with data recording. This experiment was realized using Cogent 2000 developed by the Cogent 2000 team at the Functional Imaging Laboratory and the Institute of Cognitive Neuroscience and Cogent Graphics developed by John Romaya at London’s Global University at the Wellcome Department of Imaging Neuroscience.

Reprint requests should be sent to Matthias M. Müller, Institute of Psychology, University of Leipzig, Neumarkt 9-19, 04109 Leipzig, Germany, or via e-mail: m.mueller@rz.uni-leipzig.de.

REFERENCES

REFERENCES
Allison
,
T.
,
Begleiter
,
A.
,
McCarthy
,
G.
,
Roessler
,
E.
,
Nobre
,
A. C.
, &
Spencer
,
D. D.
(
1993
).
Electrophysiological studies of color processing in human visual cortex
.
Electroencephalography and Clinical Neurophysiology
,
88
,
343
355
.
Andersen
,
S. K.
,
Hillyard
,
S. A.
, &
Müller
,
M. M.
(
2008
).
Attention facilitates multiple stimulus features in parallel in human visual cortex
.
Current Biology
,
18
,
1006
1009
.
Andersen
,
S. K.
, &
Müller
,
M. M.
(
2010
).
Behavioral performance follows the time course of neural facilitation and suppression during cued shifts of feature-selective attention
.
Proceedings of the National Academy of Sciences, U.S.A.
,
107
,
13878
13882
,
Andersen
,
S. K.
,
Müller
,
M. M.
, &
Hillyard
,
S. A.
(
2015
).
Attentional selection of feature conjunctions is accomplished by parallel and independent selection of single features
.
Journal of Neuroscience
,
35
,
9912
9919
.
Antonov
,
P. A.
,
Chakravarthi
,
R.
, &
Andersen
,
S. K.
(
2020
).
Too little, too late, and in the wrong place: Alpha band activity does not reflect an active mechanism of selective attention
.
Neuroimage
,
219
,
117006
.
Bartels
,
A.
, &
Zeki
,
S.
(
2006
).
The temporal order of binding visual attributes
.
Vision Research
,
46
,
2280
2286
.
Biderman
,
D.
,
Biderman
,
N.
,
Zivony
,
A.
, &
Lamy
,
D.
(
2017
).
Contingent capture is weakened in search for multiple features from different dimensions
.
Journal of Experimental Psychology: Human Perception and Performance
,
43
,
1974
1992
.
Brummerloh
,
B.
, &
Müller
,
M. M.
(
2019
).
Time matters: Feature-specific prioritization follows feature integration in visual object processing
.
Neuroimage
,
196
,
81
93
.
Buffalo
,
E. A.
,
Fries
,
P.
,
Landman
,
R.
,
Liang
,
H.
, &
Desimone
,
R.
(
2010
).
A backward progression of attentional effects in the ventral stream
.
Proceedings of the National Academy of Sciences, U.S.A.
,
107
,
361
365
.
Delorme
,
A.
, &
Makeig
,
S.
(
2004
).
EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis
.
Journal of Neuroscience Methods
,
134
,
9
21
.
Desimone
,
R.
, &
Duncan
,
J.
(
1995
).
Neural mechanisms of selective visual attention
.
Annual Review of Neuroscience
,
18
,
193
222
.
Di Russo
,
F.
,
Pitzalis
,
S.
,
Aprile
,
T.
,
Spitoni
,
G.
,
Patria
,
F.
,
Stella
,
A.
, et al
(
2007
).
Spatiotemporal analysis of the cortical sources of the steady-state visual evoked potential
.
Human Brain Mapping
,
28
,
323
334
.
Faul
,
F.
,
Erdfelder
,
E.
,
Buchner
,
A.
, &
Lang
,
A.-G.
(
2009
).
Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses
.
Behavior Research Methods
,
41
,
1149
1160
.
Forschack
,
N.
,
Andersen
,
S. K.
, &
Müller
,
M. M.
(
2017
).
Global enhancement but local suppression in feature-based attention
.
Journal of Cognitive Neuroscience
,
29
,
619
627
.
Found
,
A.
, &
Müller
,
H. J.
(
1996
).
Searching for unknown feature targets on more than one dimension: Investigating a “dimension-weighting” account
.
Perception & Psychophysics
,
58
,
88
101
.
Gabor
,
D.
(
1947
).
Theory of communication
.
Journal of the Institution of Electrical Engineers—Part I: General
,
94
,
58
,
Geng
,
J. J.
,
DiQuattro
,
N. E.
, &
Helm
,
J.
(
2017
).
Distractor probability changes the shape of the attentional template
.
Journal of Experimental Psychology: Human Perception and Performance
,
43
,
1993
2007
.
Gundlach
,
C.
,
Moratti
,
S.
,
Forschack
,
N.
, &
Müller
,
M. M.
(
2020
).
Spatial attentional selection modulates early visual stimulus processing independently of visual alpha modulations
.
Cerebral Cortex
,
30
,
3686
3703
.
,
N.
,
Liu
,
A. K.
,
Dale
,
A. M.
,
Cavanagh
,
P.
, &
Tootell
,
R. B. H.
(
1998
).
Retinotopy and color sensitivity in human visual cortical area V8
.
Nature Neuroscience
,
1
,
235
241
,
Hillyard
,
S. A.
, &
Anllo-Vento
,
L.
(
1998
).
Event-related brain potentials in the study of visual selective attention
.
Proceedings of the National Academy of Sciences, U.S.A.
,
95
,
781
787
,
Hillyard
,
S. A.
,
Vogel
,
E. K.
, &
Luck
,
S. J.
(
1998
).
Sensory gain control (amplification) as a mechanism of selective attention: Electrophysiological and neuroimaging evidence
.
Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences
,
353
,
1257
1270
.
Hubel
,
D. H.
, &
Wiesel
,
T. N.
(
1968
).
Receptive fields and functional architecture of monkey striate cortex
.
Journal of Physiology
,
195
,
215
243
.
Junghöfer
,
M.
,
Elbert
,
T.
,
Tucker
,
D. M.
, &
Rockstroh
,
B.
(
2000
).
Statistical control of artifacts in dense array EEG/MEG studies
.
Psychophysiology
,
37
,
523
532
.
Kastner
,
S.
,
Pinsk
,
M. A.
,
De Weerd
,
P.
,
Desimone
,
R.
, &
Ungerleider
,
L. G.
(
1999
).
Increased activity in human visual cortex during directed attention in the absence of visual stimulation
.
Neuron
,
22
,
751
761
.
Krummenacher
,
J.
, &
Müller
,
H. J.
(
2012
).
Dynamic weighting of feature dimensions in visual search: Behavioral and psychophysiological evidence
.
Frontiers in Psychology
,
3
,
221
.
Lee
,
J.
,
Leonard
,
C. J.
,
Luck
,
S. J.
, &
Geng
,
J. J.
(
2018
).
Dynamics of feature-based attentional selection during color–shape conjunction search
.
Journal of Cognitive Neuroscience
,
30
,
1773
1787
.
MacMillan
,
N. A.
(
2002
).
Signal detection theory
. In
H.
Pashler
&
J.
Wixted
(Eds.),
Stevens' handbook of experimental psychology
(pp.
43
90
).
Hoboken, NJ
:
John Wiley & Sons, Inc
.
Martínez
,
A.
,
Di Russo
,
F.
,
Anllo-Vento
,
L.
,
Sereno
,
M. I.
,
Buxton
,
R. B.
, &
Hillyard
,
S. A.
(
2001
).
Putting spatial attention on the map: Timing and localization of stimulus selection processes in striate and extrastriate visual areas
.
Vision Research
,
41
,
1437
1457
.
Maunsell
,
J. H. R.
, &
,
C. J.
(
2001
).
Effects of attention on the responsiveness and selectivity of individual neurons in visual cerebral cortex
. In
J.
Braun
,
C.
Koch
, &
J. L.
Davis
(Eds.),
Visual attention and cortical circuits
(pp.
103
119
).
Cambridge, MA
:
MIT Press
.
Maunsell
,
J. H. R.
, &
Treue
,
S.
(
2006
).
Feature-based attention in visual cortex
.
Trends in Neurosciences
,
29
,
317
322
.
Morgan
,
S. T.
,
Hansen
,
J. C.
, &
Hillyard
,
S. A.
(
1996
).
Selective attention to stimulus location modulates the steady-state visual evoked potential
.
Proceedings of the National Academy of Sciences, U.S.A.
,
93
,
4770
4774
.
Moutoussis
,
K.
, &
Zeki
,
S.
(
1997
).
A direct demonstration of perceptual asynchrony in vision
.
Proceedings of the Royal Society of London, Series B, Biological Sciences
,
264
,
393
399
,
Müller
,
H. J.
,
Heller
,
D.
, &
Ziegler
,
J.
(
1995
).
Visual search for singleton feature targets within and across feature dimensions
.
Perception & Psychophysics
,
57
,
1
17
.
Müller
,
H. J.
, &
Krummenacher
,
J.
(
2006a
).
Locus of dimension weighting: Preattentive or postselective?
Visual Cognition
,
14
,
490
513
.
Müller
,
H. J.
, &
Krummenacher
,
J.
(
2006b
).
Visual search and selective attention
.
Visual Cognition
,
14
,
389
410
.
Müller
,
H. J.
, &
,
R. B.
(
2000
).
Dimension-based visual attention modulates dual-judgment accuracy in Duncan's (1984) one- versus two-object report paradigm
.
Journal of Experimental Psychology: Human Perception and Performance
,
26
,
1332
1351
.
Müller
,
H. J.
,
Reimann
,
B.
, &
Krummenacher
,
J.
(
2003
).
Visual search for singleton feature targets across dimensions: Stimulus- and expectancy-driven effects in dimensional weighting
.
Journal of Experimental Psychology: Human Perception and Performance
,
29
,
1021
1035
.
Müller
,
M. M.
,
Picton
,
T. W.
,
Valdes-Sosa
,
P.
,
Riera
,
J.
,
Teder-Sälejärvi
,
W. A.
, &
Hillyard
,
S. A.
(
1998
).
Effects of spatial selective attention on the steady-state visual evoked potential in the 20–28 Hz range
.
Cognitive Brain Research
,
6
,
249
261
,
Müller
,
M. M.
,
Trautmann
,
M.
, &
Keitel
,
C.
(
2016
).
Early visual cortex dynamics during top–down modulated shifts of feature-selective attention
.
Journal of Cognitive Neuroscience
,
28
,
643
655
.
Roe
,
A. W.
,
Chelazzi
,
L.
,
Connor
,
C. E.
,
Conway
,
B. R.
,
Fujita
,
I.
,
Gallant
,
J. L.
, et al
(
2012
).
Toward a unified theory of visual area V4
.
Neuron
,
74
,
12
29
.
Theeuwes
,
J.
(
1992
).
Perceptual selectivity for color and form
.
Perception & Psychophysics
,
51
,
599
606
.
Treue
,
S.
, &
Martínez Trujillo
,
J. C.
(
1999
).
Feature-based attention influences motion processing gain in macaque visual cortex
.
Nature
,
399
,
575
579
.
Viviani
,
P.
, &
Aymoz
,
C.
(
2001
).
Colour, form, and movement are not perceived simultaneously
.
Vision Research
,
41
,
2909
2918
.
Wagner
,
G.
, &
Boynton
,
R. M.
(
1972
).
Comparison of four methods of heterochromatic photometry
.
Journal of the Optical Society of America
,
62
,
1508
1515
.
White
,
A. L.
,
Lunau
,
R.
, &
Carrasco
,
M.
(
2014
).
The attentional effects of single cues and color singletons on visual sensitivity
.
Journal of Experimental Psychology: Human Perception and Performance
,
40
,
639
652
.
Zeki
,
S.
(
1983
).
Colour coding in the cerebral cortex: The reaction of cells in monkey visual cortex to wavelengths and colours
.
Neuroscience
,
9
,
741
765
.