Endogenous attention modulates the amplitude and phase coherence of steady-state visual-evoked potentials (SSVEPs). In efforts to decipher the neural mechanisms of attentional modulation, we compared the time course of attentional modulation of SSVEP amplitude (thought to reflect the magnitude of neural population activity) and phase coherence (thought to reflect neural response synchronization). We presented two stimuli flickering at different frequencies in the left and right visual hemifields and asked observers to shift their attention to either stimulus. Our results demonstrated that attention increased SSVEP phase coherence earlier than it increased SSVEP amplitude, with a positive correlation between the attentional modulations of SSVEP phase coherence and amplitude. Furthermore, the behavioral dynamics of attention shifts were more closely associated with changes in phase coherence than with changes in amplitude. These results are consistent with the possibility that attention increases neural response synchronization, which in turn leads to increased neural population activity.
Visual attention is the function to select potentially important information from the vast amount of visual input in everyday life. Attention can be oriented to a region in space (Posner & Petersen, 1990; Posner, 1980; Posner, Snyder, & Davidson, 1980); to a visual feature such as color, orientation, and motion direction (Maunsell & Treue, 2006); or to a perceptual object (Moore, Yantis, & Vaughan, 1998; Egly, Driver, & Rafal, 1994; Duncan, 1984). In this study, we focus on visual spatial attention. Attentional selection in space is often analogized to a “zoom lens” or to a “spotlight” that lights up a part of our visual scene and facilitates information processing within the beam, although attention can be oriented to separate regions simultaneously (Jans, Peters, & De Weerd, 2010; Malinowski, Fuchs, & Müller, 2007; McMains & Somers, 2004, 2005; Gobell, Tseng, & Sperling, 2004; Müller, Malinowski, Gruber, & Hillyard, 2003). Many psychophysical studies have shown that spatially focused attention accelerates visual processing (Carrasco & McElree, 2001; Egeth & Yantis, 1997; Egly et al., 1994; Hikosaka, Miyauchi, & Shimojo, 1993a, 1993b; Eriksen & St James, 1986; Yantis & Jonides, 1984; LaBerge, 1983; Posner, 1980; Posner et al., 1980; Eriksen & Eriksen, 1974), increases sensitivity to the stimulus (Shioiri, Yamamoto, Oshida, Matsubara, & Yaguchi, 2010; Matsubara, Shioiri, & Yaguchi, 2007; Pestilli & Carrasco, 2005; Shioiri, Yamamoto, Kageyama, & Yaguchi, 2002; Carrasco, Penpeci-Talgar, & Eckstein, 2000; Lee, Itti, Koch, & Braun, 1999; Yeshurun & Carrasco, 1998, 1999; Luck et al., 1994; Hawkins et al., 1990), excludes external noise superimposed on the signal (Lu, Lesmes, & Dosher, 2002; Dosher & Lu, 2000), or integrates visual features into a perceptual object (Treisman & Schmidt, 1982; Treisman & Gelade, 1980). Visual spatial attention can be separated into two distinct modes: endogenous and exogenous (Chakravarthi & VanRullen, 2011; Carlson, Hogendoorn, & Verstraten, 2006; Ling & Carrasco, 2006; Corbetta & Shulman, 2002; Egeth & Yantis, 1997; Hikosaka et al., 1993b; Cheal & Lyon, 1991; Yantis & Jonides, 1990; Müller & Rabbitt, 1989; Nakayama & Mackeben, 1989; Weichselgartner & Sperling, 1987). Endogenous orienting of attention refers to the case where the observer voluntarily allocates attention, whereas exogenous orienting of attention refers to the case where attention is automatically captured by a salient stimulus such as a transient onset cue. In this study, we focus on the former, endogenous attention.
As the size of the attentional spotlight is spatially restricted (Shioiri et al., 2010; Matsubara et al., 2007; Müller, Bartelt, Donner, Villringer, & Brandt, 2003; Castiello & Umilta, 1990; Eriksen & St James, 1986), it is often assumed that the attentional spotlight moves around the visual space during scene perception. Understanding the neural mechanisms that control the dynamics of attention shifts is one of the important issues in the field of attention study.
However, investigating the dynamic neural processes associated with attention shifts has proven difficult because fMRI does not provide sufficient temporal resolution, and event-related EEG potentials occur primarily at stimulus onsets and offsets. Steady-state visual-evoked potentials (SSVEPs) provide a method to continuously track attention at a high temporal resolution. SSVEPs are oscillatory EEG potentials that occur in response to flickering visual stimuli. The magnitude of visual response to a flickered stimulus can be tracked by extracting the Fourier component corresponding to the flicker frequency in the EEG signal (Vialatte, Maurice, Dauwels, & Cichocki, 2010).
Endogenous attention to a flickering stimulus has been shown to increase SSVEP amplitude (Müller, Picton, et al., 1998; Morgan, Hansen, & Hillyard, 1996). Temporal changes in SSVEP amplitude can thus be used to continuously track the degree of attention allocated to a flickering stimulus (Andersen & Müller, 2010; Attar, Andersen, & Müller, 2010; Müller, 2008; Müller, Andersen, & Keil, 2008; Belmonte, 1998; Müller, Teder-Sälejärvi, & Hillyard, 1998). Attentional modulation of SSVEP amplitude may reflect a control mechanism for neural response gain that selectively enhances attended signals in early visual cortical areas (Müller, Picton, et al., 1998; Müller, Teder-Sälejärvi, et al., 1998; Hillyard et al., 1997).
Endogenous attention also increases phase coherence of SSVEPs (Kim, Grabowecky, Paller, Muthu, & Suzuki, 2007; Ding, Sperling, & Srinivasan, 2006). In other words, SSVEPs are more phase-locked to the luminance modulation of the flickering stimulus when the stimulus is attended than when it is ignored. This suggests that attention increases neural response synchronization as well as response amplitude. Some researchers have suggested that attentional enhancements of neural responses are mediated by the attentional boost of neural response synchronization (Kim et al., 2007; Tiesinga, Fellous, Salinas, Jos, & Sejnowski, 2004; Fries, Reynolds, Rorie, & Desimone, 2001; Steinmetz et al., 2000).
To understand how attention-induced increases in SSVEP amplitude and phase coherence are related, here we examine the time course of modulations of SSVEP amplitude and phase coherence associated with attention shifts. In particular, if attention directly modulates neural synchronization, which in turn modulates SSVEP amplitude, we would expect to see similar temporal dynamics for attentional modulations in phase coherence and in amplitude with possible earlier changes in phase coherence.
Another important area of interest is whether behavioral effects of attention are more closely associated with changes in phase coherence or changes in SSVEP amplitude. Although previous studies demonstrated temporal correlation between attentional modulation of SSVEP amplitude and behavioral performance (Andersen & Müller, 2010; Attar et al., 2010; Müller, 2008; Müller et al., 2008; Müller, Teder-Sälejärvi, et al., 1998), none considered the time course of attentional modulation of SSVEP phase coherence. Thus, whether the behavioral effects of attention are more closely associated with SSVEP amplitude or phase coherence remains unknown. To answer this question, we carefully compared the time courses of SSVEP phase coherence and amplitude with the time course of attentional modulation of behavioral performance.
Eight participants (one woman; 22–32 years) with normal or corrected-to-normal visual acuity gave informed consent to participate in this experiment. Data from one participant was excluded because of excessive artifacts in the EOG data (see Analysis). This study was approved by the ethics committee of the Tohoku University Research Institute of Electrical Communication.
Figure 1A shows a schematic view of our stimulus. Two circular square-wave gratings (5.2° diameter and 1.25 cycles/deg) were presented, centered at 5.5° eccentricity on the left and right sides of the central fixation marker. The gratings were presented against a dark background (<0.1 cd/m2). The maximum luminance of the rings was 143 cd/m2. To evoke SSVEPs, the luminance of the rings was modulated sinusoidally at 21.0 Hz on one side of the stimuli and at 28.0 Hz on the other side. The depth of the luminance modulation was 100% for both stimuli.
An arrow-shaped stimulus pointing either left or right was presented as a cue at the center of the display to indicate the circular grating to be attended. To avoid abrupt onset of an attention cue and maintain the participant's attention on the center of the display until the attention cue appeared, the cue was embedded in a rapid serial visual presentation (RSVP) stream. In the RSVP stream, meaningless symbols were presented at a rate of 5 times/sec (200 msec/symbol; Figure 1A). The symbols were composed of three lines randomly chosen from a set of nine line segments. All nine line segments were simultaneously shown at the center of the display, as illustrated in Figure 1A, with examples of the actual stimuli used shown in the inset. The color of the RSVP stimuli was dark blue (0.5 cd/m2), and line lengths were 0.6° for the horizontal and vertical lines and 0.4° for the oblique lines. The presentation of the attention cue was temporally aligned at 0° in the phase of the luminance modulation for both flickering gratings.
A randomly chosen ring on each grating changed color from white to yellow every 143 msec (7 times/sec) with the constraint that successive color changes never occurred on the same ring. The participant's task was to respond by pressing an assigned key as quickly as possible when the outermost ring of the attended grating changed color (Figure 1B).
Visual stimuli were presented in a dark room on a SONY GDM-F520 CRT display monitor at a refresh rate of 120 Hz using a ViSaGe graphics card (Cambridge Research System, Cambridge, UK) controlled using MATLAB software (The Mathworks, Natick, MA). The viewing distance was 50 cm.
Figure 1C shows the sequence of events within a trial. Each trial started with a key press by the participant. Initially, a static image of the two circular gratings and the nine line segments at the center were presented for 500 msec (pretrial period). Then, the flicker of the gratings and the RSVP stream started (precue period). Flicker frequencies were randomly assigned to the left and right gratings on each trial (21.0 Hz left/28.0 Hz right or 28.0 Hz left/21.0 Hz right). The duration of the precue period was chosen randomly on each trial and varied between values of 1200, 1400, 1600, 1800, and 2000 msec. At the end of the precue period, the central shape became an arrow-shaped cue for 200 msec. The participant was instructed to detect this arrow cue in the RSVP stream and to shift their attention toward the grating to which the arrow pointed. The two gratings remained flickering for 3000 msec after the cue presentation. During this period, the participant maintained attention on the cued grating (postcue period). Verification of their attention was enforced by a behavioral task. The participant was instructed to press an assigned key as soon as possible every time he or she detected a color change in the outermost ring of the attended grating (Figure 1B) while ignoring color changes on the other grating. The target event occurred randomly with the constraint that the minimum intertarget interval was 429 msec. The total number of target events in each time bin was the same for different conditions (see below) and the total number in each trial was 4.2 at the attended location on average. At the end of the trial, the participant reported the direction of the arrow-shaped cue to confirm that they had detected the cue.
There were four experimental conditions: two attention locations (left and right) and two flicker-frequency assignments (21.0 Hz on the left and 28.0 Hz on the right, and vice versa). Each condition consisted of 100 trials, including 20 repetitions of each precue period (see above). The experimental session for each participant consisted of 400 trials, which was divided into four blocks of 100 trials. All conditions, the attention locations, the flicker-frequency assignments, and the precue periods, were randomly intermixed across trials within each block. Each block lasted about 20 min.
We recorded brain electrical activity from 19 scalp electrodes mounted on an elastic cap connected to an EEG recording system (Neurofax EEG-9100, Nihon Koden, Tokyo, Japan). The electrode arrangement was based on the International 10–20 System (Figure 2). Reference electrodes were placed on both ear lobes (A1, A2). All EEG channels except for the midline channels (Fz, Cz, Pz) were referenced to the ear lobe ipsilateral to the hemisphere. The midline channels (Fz, Cz, Pz) were referenced to A1. EEG signals were recorded with a bandpass filter of 0.5–120 Hz and digitized at 1000 Hz. Recorded data were stored on a hard disk for off-line analyses. All electrode impedances were confirmed to be below 5 kΩ before each experimental block.
To avoid EEG artifacts generated by eye movements and blinks and ensure that any SSVEP modulation observed in the experiment are attributable to covert attention shifts, participants were asked to fixate on the center of the display (the RSVP stream) and try not to blink during each trial (up to 5 sec). Two additional electrodes were placed on 1 cm above and below the outer canthus of the left and right eyes in reference to the manual of EEG recording system. Lateral eye movements were recorded with a bipolar left-to-right outer canthus montage (horizontal EOG) for off-line analysis of the artifacts.
Trials were removed from data analyses when participants reported an incorrect cue direction or failed to detect the cue. We also excluded trials with EOG deflections of more than ±40 μV from the potential averaged over all data points through the trial, which was thought to correspond roughly to a 2.5° eye shift (Luck, 2005). EEG data captured during trials with target detection errors were included in the analyses regardless of the number of errors.
The hit (or FA) rate was defined as the ratio of the number of hits (or FAs) to the number of all targets (or distracters) presented in each time bin. To evaluate the attentional modulation of the behavioral performance, we computed d′ for each time bin with a conventional modification to the hit (or FA) rates of 0 and 1 (Macmillan & Kaplan, 1985).
SSVEP Amplitude and Phase Coherence
Although SSVEP amplitudes were calculated after averaging EEG data across trials in most previous studies, here we calculated SSVEP amplitudes before averaging EEG data across trials. This was necessary to dissociate the effect of attention on SSVEP amplitude from the effect of attention on SSVEP phase coherence. Even if attention only increased the phase coherence of SSVEP, the reduced phase variability of SSVEP across trials would necessarily increase the SSVEP amplitude in the averaged EEG waveform. By calculating the SSVEP amplitude separately on each trial and then averaging them across trials, we avoided this confound.
Time Course Analysis
The baseline of each SSVEP measure was defined as the mean between 500 and 250 msec before the cue onset and was set to zero in the analysis for both amplitude and ITPC.
Data Rejection Rate
The results for one participant were excluded from the analysis because of an excessive number of EOG artifacts (more than 40% of the trials) and the retroactive report of the failure of the experimental task. For the remaining participants, 5.4% of the trials were judged to be contaminated by eye movement artifacts on average (range = 1.5–10.8%, SE = 1.2%). The mean cue detection rate was 90.7% (range = 79.0–97.3%, SE = 2.9%). Total valid trials, which include neither artifacts nor misses of the cue, amounted to 86.5% (range = 74.5–95.8%, SE = 3.0%).
Figure 4A shows an RT distribution and the best-fit Gaussian function. The mean (μ) and the standard deviation (σ) of the function were estimated as 392.0 and 64.2 msec, respectively. We determined the responses between 199.5 msec (μ - 3σ) and 584.6 msec (μ + 3σ) after each target onset as hits, and the other responses (occurring earlier than 199.5 msec or later than 584.6 msec) as FAs.
The rates of hit and FA were calculated for each time bin after collapsing the data across the four experimental conditions (attended frequencies [21.0 and 28.0 Hz] and attended sides [left and right]). Figure 4B shows the d′ (top) as well as the hit and FA rates (bottom) as a function of the target presentation time relative to the onset of the attention cue. A one-tailed t test revealed that the d′ from the third time bin (t = 291.7 msec) up to the end of the trial was significantly greater than that in the first time bin (t = 0 msec) [t(6) > 2.00, p < .05 for all]. This result indicates that attention increased sensitivity to the target at least up to 2.5 sec, confirming that participants maintained attention on the cued location. To quantify the speed of the attention shift, we fitted the modified cumulative Gaussian function (Equation 12) to the d′ data using the value at t = 0 msec as the baseline of the function. The fitted curve is depicted by the solid line in Figure 4B. The parameters μ and σ were estimated as 386.3 and 139.3 msec, respectively.
It is worth noting that the behavioral performance gradually declined after the attention shift. This is unusual for endogenous attention. Several previous studies showed that the effect is stable even for several seconds after the attention shift (Herrington & Assad, 2009; Hikosaka et al., 1993b). The gradual decline might be partly because of repetitions of target detection. There might be an inhibitory influence of responses to the preceding target on the detection of the following target as in the case of attentional blink (Raymond, Shapiro, & Arnell, 1992).
Figure 5 (A–D) shows the mean amplitude spectra of EEG signals from occipital channels (O1/O2) when the 21.0 Hz grating was on the left and the 28.0 Hz grating was on the right (A and B), and vice versa (C and D). SSVEPs to the flickering stimuli (21.0 and 28.0 Hz, indicated by vertical lines) were apparent as the peaks at 21.0 and 28.0 Hz, especially in the channels contralateral to the stimuli. Furthermore, the SSVEPs were modulated by attention such that a peak became higher when attention was directed to the corresponding grating compared with when attention was directed to the other grating. We obtained a similar pattern of results for ITPC (Figure 5E–H). These results suggest that visual spatial attention increased both SSVEP amplitude and phase coherence.
To evaluate the attentional modulation of SSVEPs statistically, we extracted EEG components corresponding to the flicker frequencies (21.0 and 28.0 Hz). Figure 6 (A and B) shows the topographic distribution of attentional modulation of SSVEP amplitude (i.e., the attended minus the ignored condition).
The amplitude data were subjected to a repeated measures ANOVA with factors of Attentional State (attended vs. ignored), Stimulus Side (left vs. right), and Channel (19 sites). Main effects were found for Attentional State [F(1, 6) = 8.77, p < .05] and Channel [F(18, 108) = 2.86, p < .05] and interactions between Attentional State × Channel [F(18, 108) = 2.75, p < .05], Stimulus Side × Channel [F(18, 108) = 6.24, p < .05], and the three factors [F(18, 108) = 3.29, p < .05]. Analyses of simple main effects revealed that, for the left stimulus, significant attentional modulation of SSVEP amplitude occurred in Fp1, Fp2, P4, O2, and T6, and for the right stimulus, significant attentional modulation of SSVEP amplitude occurred in P3, O1, and T5 (all ps < .05; indicated by circles in Figure 6A and B).
We performed the same analyses on ITPC. A three-factor ANOVA revealed main effects of Attentional State [F(1, 6) = 50.75, p < .05] and Channel [F(18, 108) = 18.54, p < .05], the interactions between Stimulus Side × Channel [F(18, 108) = 2.24, p < .05], and the three factors [F(18, 108) = 2.33, p < .05]. Analyses of simple main effects revealed that, for the left stimulus, significant attentional modulation of ITPC occurred in C4, P4, O1, O2, T4, and T6, and for the right stimulus, significant attentional modulation of ITPC occurred in C3, P3, O1, T3, T5, Fz, and Cz (all ps < .05; indicated by circles in Figure 6C and D).
On the basis of the analyses above, we used the data from P3, O1, and T5 for the flickering stimulus in the right visual field and P4, O2, and T6 for the flickering stimulus in the left visual field in the following analyses because these channels showed significant SSVEP modulation in both amplitude and ITPC measures.
SSVEP Signal-to-Noise Ratio
Figure 7 (A and B) shows the SNRs of the SSVEP components in amplitude and ITPC, respectively. The SNRs are clearly larger than one and one-tailed t tests confirmed this for the amplitude data [t(6) > 1.96, all ps < .05] and for the ITPC data [t(6) > 3.17, all ps < .05].
Quantification of the time course of SSVEP modulation
To analyze the time course of attentional modulation of SSVEP amplitude and ITPC, we averaged the data across the selected channels (see above). Figure 8 (A and C) shows the averaged time course of SSVEP amplitude and ITPC, respectively, for the attended (red solid line) and ignored (blue dotted line) conditions. A black dotted line in Figure 8C shows the time course of ITPC related to the central RSVP stimulus (see below). The time course of attentional modulation, defined as the difference between the attended and ignored curves,2 is shown by the black solid lines in Figure 8B and D, respectively.
Attentional modulation of SSVEPs was sustained and was higher than the baseline for both measures. To evaluate the time course of attentional modulation of SSVEP amplitude and ITPC quantitatively, we fitted the modified cumulative Gaussian function to each time course data, as we did to evaluate the time course of behavioral data. For SSVEP amplitude, the parameters μ and σ were estimated as 800.8 and 175.1 msec, respectively. For ITPC, the parameters μ and σ were estimated as 549.3 and 123.5 msec, respectively.
Statistical test for the latency of SSVEP modulation
The time course analyses indicated that SSVEP latency differed by about 250 msec between the amplitude and ITPC. The time courses of attentional modulations of SSVEP amplitude and ITPC in the attended condition are directly compared in Figure 9A (replotted from the red lines in Figure 8A and C). To test whether this latency difference was statistically significant, we obtained the SSVEP latency data for each observer. Because the variation in individual data was too large to estimate the latency using Equation 12, we used a different definition of the latency (Busse, Katzner, & Treue, 2008). We calculated local slopes of the SSVEP function by fitting a linear regression line to a 301-msec interval between 0 and 1500 msec after the cue onset with a step size of 1 msec. Latency was defined as the time point at which the slope reached a maximum, which roughly corresponds to the parameter μ of Equation 12.
The average latency of SSVEP amplitude and ITPC is shown in Figure 9B (bar graphs). For both measures, the latency was within μ ± σ range obtained from the fitted function (Equation 12), which indicates the validity of this analysis (shown as vertical gray lines in Figure 9B). A two-tailed paired t test revealed that the attentional modulation latency of ITPC was significantly shorter than that of SSVEP amplitude [amplitude = 749.7 msec, ITPC = 618.6 msec; t(6) = 3.67, p < .05].
To investigate the effect of the central cue on the attentional states, we analyzed the frequency component corresponding to the central RSVP stimulus (5.0 Hz). As with the other SSVEP data, we averaged the RSVP-related SSVEP across the occipital channels contralateral to the attended visual field (left: P4, O2, T6; right: P3, O1, T5).
We also computed the SNR of the SSVEP data using Δf of 0.5 and N of 3, which corresponded to ±1.5 Hz in Equation 13. We analyzed only the ITPC data because one-tailed t tests revealed that the SNR of the ITPC was significantly higher than 1 [t(6) = 4.28, p < .05], whereas the SNR of the amplitude was not [t(6) = −4.94, p > .10].
The black dotted line in Figure 8C shows the temporal changes in the RSVP-related ITPC analyzed using the same procedure that we used for analyzing SSVEP responses to the gratings. A two-tailed t test revealed that the ITPC value averaged across 1000–2500 msec after the cue onset was significantly reduced compared with the precue baseline [t(6) = -2.71, p < .05]. The latency of the change was also computed in the same way as for the SSVEPs to the gratings (Figure 9B), except that the algorithm determined the time when the regression slope became maximally negative. The latency of the ITPC reduction was estimated as 527.4 msec (SE = 73.3 msec). It is about 90 msec faster than that of the peripheral ITPC increase (618.6 msec; see Figure 9B) although the difference is not statistically significant [t(6) = −1.38, p > .10]. The difference may be related to the dynamics of disengagement and reengagement of attention. We will discuss further in Discussion.
In this study, the temporal dynamics of attention shift were measured with both behavioral and SSVEP methods and attentional modulation was observed in both SSVEP amplitude and ITPC. Several studies have suggested that SSVEPs are generated by the phase alignment of EEG signals, rather than by the amplitude modulation of the EEG itself (Vialatte et al., 2010; Moratti, Clementz, Gao, Ortiz, & Keil, 2007). However, no direct comparison between attentional modulations in both SSVEP amplitude and phase coherence had been reported. Our analysis showed that both SSVEP phase and amplitude were modulated by attention in similar manner. This suggests the human attention system utilizes both neural gain control and neural response synchronization.
Both behavioral and SSVEP measures showed that endogenous attention rapidly shifted to the location indicated by the central cue and remained focused at that location for several seconds. The time courses of attention shifts were similar, but the latencies (parameter μ) differed for the behavioral and SSVEP measures.
The latency of behavioral performance (d′) was about 390 msec after the cue onset (μ), and the time required to shift attention fully to the stimulus was estimated to be about 520 msec (μ + σ). To be more precise, each time bin includes the duration for the target presentation (about 140 msec); thus, the time range for the latency of maximum attentional modulation was considered to be between 520 msec (μ + σ) and 660 msec (μ + σ + 140). This duration is longer than the latency reported in previous studies (about 300–400 msec; for reviews, see Wright & Ward, 2008; Egeth & Yantis, 1997). This discrepancy is perhaps due to some difference in cueing stimulus. The participants in this study had to detect an arrow-shaped cue embedded in the RSVP stream. This could cause a longer latency for detection and processing of the cue compared with a conventional arrow cue presented alone with a sudden onset. To estimate the time required for the cue processing, we averaged the raw EEG data time-locked to the cue onset and digitally filtered the averaged data between 0 and 15 Hz bands. Figure 10 shows the averaged EEG waveform from the same channels and conditions used in the SSVEP analysis. A negative deflection was first observed at around 310 msec after the cue onset, followed by a positive deflection at around 480 msec. Assuming the latency of the first deflection (about 310 msec) accurately reflects the time required for cue detection, the attention shifts could have been delayed by at least 310 msec after the presentation of the cue. Therefore, the net duration of time to shift attention could be 210–350 msec, which is consistent with the previous estimates (Wright & Ward, 2008; Egeth & Yantis, 1997).
The latency of attentional modulation of SSVEPs was around 800 msec in amplitude and 550 msec in ITPC and that for the behavioral data was around 390 msec. To compare the physiological results with the behavioral ones, we must take into account the time delay between physical stimulus input and the response onset in the brain (Müller, Teder-Sälejärvi, et al., 1998). That is, attentional modulation at a time (T0) in the SSVEP measures should reflect the brain's attentional state at the same moment. The temporal axis of the behavioral performance, on the other hand, is determined by the timings of target presentation. In other words, the behavioral performance at time T0 does not reflect the attentional state at T0 in the brain, unlike the SSVEP measures. To directly compare the time courses, we must align the temporal axis of the different measures to the visual stimulus onset. Assuming that ΔT represents the time required to convey the flicker signal to the brain area with attentional modulation, the SSVEP at T0 should reflect the neural activity to the flickered stimulus presented ΔT earlier (T0 − ΔT). The neural generators of SSVEPs have been reported in early visual areas including V1, V2, and V3 (Andersen & Müller, 2010; Andersen, Hillyard, & Müller, 2008; Di Russo et al., 2007; Müller et al., 2006; Müller, Picton, et al., 1998; Hillyard et al., 1997), and the latency for the response facilitation is estimated to be delayed by 100–150 msec after the stimulus input (Hillyard & Anllo-Vento, 1998; Müller, Teder-Sälejärvi, et al., 1998). Here, we defined the delay (ΔT) as 150 msec based on the previous study (Müller, Teder-Sälejärvi, et al., 1998) and corrected the SSVEP latencies by subtracting ΔT (150 msec) from the latencies. Consequently, the latency of the amplitude modulation would be estimated to be 650 msec, whereas that of the ITPC modulation would be around 400 msec. The latency of the behavioral performance of 390–530 msec (considering the duration of a target presentation: 140 msec) became consistent with that of ITPC. Our results, therefore, show that the time course of behavioral performance more closely follows that of neural response synchronization than that of neural gain increase.
The time course of the RSVP-related ITPC shows two interesting aspects of the attention shifts (Figure 8C). First, the ITPC to the central RSVP stimulus was decreased approximately 530 msec after the cue onset, which can be attributed to the time of the attentional disengagement from the central fixation. One could estimate the time for switching attention from the center to peripheral as 90 msec, considering the latency of ITPC enhancement to the peripheral flickering stimuli (about 620 msec estimated by the local maximum slope method, which is different from 550 msec estimated by the curve-fitting method). These results are consistent with a serial execution model of disengagement, movement, and reengagement for shifting attention (Posner & Raichle, 1994). Second, we found a transient increase in the ITPC of the RSVP at around 300 msec (maximum ITPC at 288 msec). This transient increase appears to mirror the transient decrease in the SSVEPs to the peripheral flickering stimuli (Figure 8A and C). Attention may be captured transiently to the central RSVP when the cue is detected. This could enhance neural responses to the central RSVP stimulus while inhibiting responses to the peripheral flickering stimulus.
The estimated latency was faster in ITPC than in amplitude despite the fact that these two measures were derived from identical EEG signals (Figures 8 and 9). This result can be interpreted in two ways. One interpretation is that, although they have a common mechanism in the attention system, some time must elapse before neural synchronization leads to an enhanced neural response amplitude. Another interpretation is that they reflect different neural mechanisms. To investigate these possibilities, we analyzed the correlations between SSVEP amplitude and ITPC. Figure 11A shows a scatterplot of SSVEP amplitude as a function of ITPC for the 21.0 Hz stimulus (dark gray dots) and the 28.0 Hz stimulus (light gray dots). Pearson's correlation analyses revealed clear positive correlations between the two measures for 21.0 Hz SSVEPs (r = 0.57, p < .05) and for 28.0 Hz SSVEPs (r = 0.65, p < .05). We also examined the relationship between attentional modulations of ITPC and amplitude (each defined as the attended minus the ignored condition). Figure 11B shows a scatterplot of the amplitude modulation as a function of the ITPC modulation. The result also showed a significant positive correlation between the two measures (r = 0.43, p < .05).
Although the correlation analyses above may suggest common neural substrates for the enhancement of SSVEP amplitude and ITPC, the causal relationship between the two measures remains unclear. Although this is consistent with the suggestion that neural synchronization increases neural response amplitude (Kim et al., 2007), the SSVEP amplitude could also influence the ITPC. For example, when internal/external noise (e.g., spontaneous activity) is superimposed on the SSVEP signals, the noise will produce greater intertrial variability in the SSVEP phase (i.e., smaller ITPC) when the SSVEP amplitude is smaller. Therefore, an increase in the ITPC could be explained by an increase in the SSVEP amplitude. However, our time course analysis revealed that the ITPC modulation precedes the amplitude modulation. This rules out the possible interpretation that attention directly increases SSVEP amplitude with the increased amplitude necessarily elevating the ITPC measure and supports the notion that attention increases neural response synchronization that mediates attentional gain in SSVEP amplitudes (Kim et al., 2007).
The correlation analyses above may have oversimplified the relationship between ITPC and amplitude. There are at least two important differences between attentional modulations of SSVEP amplitude and ITPC. First, the latency of attentional modulation is clearly different between amplitude and ITPC (Figures 8 and 9). Second, whereas the scalp region of attentional modulation was narrowly focused on the occipital channels contralateral to the visual stimulation for SSVEP amplitude, the region of modulation was broader for ITPC (Figure 6). If the response synchronization directly induced the amplitude modulation, the latency and the topography should be similar in the two measures.
A question is whether there is an amplitude modulation mechanism that is independent of neural synchronization. To answer this question, we examined whether the amplitude modulation is solely caused by the ITPC modulation. We addressed this question with two approaches. First, we analyzed the data assuming a linear correlation between ITPC and amplitude (see Figure 11). If the amplitude modulation is completely predicted by ITPC, the slope of the regression line of the attended condition should be the same as that of the ignored condition (Figure 12A). On the other hand, if there is an additional attentional modulation of amplitude that is independent of an attentional modulation of ITPC, the amplitude data should be larger than that predicted from the regression line for the ignored condition (Figure 12B). Figure 12C shows the regression lines in the attended (black) and ignored (gray) conditions for each participant. A paired t test showed that the regression slope was marginally steeper in the attended condition than in the ignored condition (Figure 12D) [t(6) = 1.93, p = .051; one-tailed].
Second, we computed the latency of the SSVEP amplitude from averaged (not single-trial) EEG data (gray dotted line in Figure 8B) and compared it with the ITPC latency. Because the trial-averaged SSVEP amplitude is influenced by ITPC, it should be more similar to the ITPC than the single-trial one. However, the difference between the two measures should not disappear if there is an amplitude modulation mechanism that is independent of neural synchronization. We evaluated the latency of the trial-averaged SSVEP amplitude by both the curve fitting (Equation 12) and the local maximum slope. The results showed that, in both cases, the latencies of the amplitude were still longer than those of the ITPC (curve fitting [parameter μ]: amplitude = 633.7 msec, ITPC = 549.3 msec; local maximum slope: amplitude = 687.3 msec, ITPC = 618.6 msec) although the difference is smaller than the original comparison and is not statistically significant [t(6) = 0.73, p > .10; two-tailed t test]. The fact that latencies between the two measures become similar but does not disappear is exactly what we predict.
These results from the two additional analyses support the assumption that there are both synchronization-dependent and -independent neural amplification mechanisms. Future research should investigate the relationship between the amplitude and the ITPC in further detail.
In summary, our results showed that endogenous attention modulates both the amplitude and phase coherence of SSVEPs. Time course analyses revealed that the attentional modulation of ITPC occurred earlier than the attentional modulation of SSVEP amplitude and was more closely associated with the dynamics of behavioral performance. Taken together with the relatively strong correlation between ITPC and amplitude, our results support the notion that attention increases neural response synchronization, which in turn boosts neural response strength at the population level (Kim et al., 2007).
We thank the two anonymous reviewers for helpful comments and discussions. This work was supported by Grants-in-Aid for Scientific Research (A18203036 and B18330153) and by a Grant-in-Aid for JSPS Fellows to Y. K. by the Japan Society for the Promotion of Science.
Reprint requests should be sent to Yoshiyuki Kashiwase, Research Institute of Electrical Communication, Tohoku University, 2-1-1 Katahira, Aoba-ku, Sendai 980-8577, Japan, or via e-mail: email@example.com or firstname.lastname@example.org.
Because of the limited temporal precision of the stimulus presentation system, there could be some time lags up to 7 msec (5.2 msec on average) between the configured and actual stimulus presentation times. Note, however, that these lags are small compared with the time window for detecting a hit (about 385 msec).
Although the analysis can be performed using the data from only the attended condition, we analyzed the difference between the attended and ignored conditions to isolate the effect of endogenous attention on the time course of SSVEP amplitude and phase coherence. We confirmed that the results are similar even if we analyzed only the attended condition.