Abstract

We used event-related potentials (ERPs) and gamma band oscillatory responses (GBRs) to examine whether intermodal attention operates early in the auditory, visual, and tactile modalities. To control for the effects of spatial attention, we spatially coregistered all stimuli and varied the attended modality across counterbalanced blocks in an intermodal selection task. In each block, participants selectively responded to either auditory, visual, or vibrotactile stimuli from the stream of intermodal events. Auditory and visual ERPs were modulated at the latencies of early cortical processing, but attention manifested later for tactile ERPs. For ERPs, auditory processing was modulated at the latency of the Na (29 msec), which indexes early cortical or thalamocortical processing and the subsequent P1 (90 msec) ERP components. Visual processing was modulated at the latency of the early phase of the C1 (62–72 msec) thought to be generated in the primary visual cortex and the subsequent P1 and N1 (176 msec). Tactile processing was modulated at the latency of the N160 (165 msec) likely generated in the secondary association cortex. Intermodal attention enhanced early sensory GBRs for all three modalities: auditory (onset 57 msec), visual (onset 47 msec), and tactile (onset 27 msec). Together, these results suggest that intermodal attention enhances neural processing relatively early in the sensory stream independent from differential effects of spatial and intramodal selective attention.

INTRODUCTION

Attention is often discussed as if it were a unitary process where some stimuli are simply selected for further or more in-depth processing. However, this is an incomplete view of the multifaceted nature of attention. Attentional processes in the environment often involve complex interactions of selection and enhancement across and within multiple sensory modalities. In navigating this dizzying array of sensory stimuli, humans and many other animals must use short- and long-term goals to select a relevant sensory stream while suppressing distracters. In fact, human participants often report that they employ both enhancement and suppression to tune their attention intermodally, focusing within and across sensory modalities.

An important guiding principle in characterizing the neural bases of attentional phenomena is to investigate where and when attention modulates sensory processing. One model of attention is a series of hierarchical filters attenuating and enhancing processing along a sequential processing stream according to task demands (Hillyard & Anllo-Vento, 1998), which may reflect both feedforward and feedback mechanisms of attention (Schroeder, Mehta, & Foxe, 2001). According to a hierarchical model, spatial attention operates early, enhancing the processing of both target and nontarget stimuli in the attended location, whereas further selection based on stimulus features occurs later in the processing stream. These concepts have typically been investigated within a single sensory modality. An important theoretical consideration is where does intermodal attention fit in the attentional hierarchy?

In the present study, we investigated the latency at which intermodal attention influences sensory processing using well-studied evoked potentials and oscillatory responses in the gamma band. Oscillatory activity is less commonly reported than ERPs but gamma band oscillatory responses (GBRs; generally 20 to 80 Hz) are observed in multiple sensory modalities. For example, gamma oscillations occur in the olfactory bulb (Eeckman & Freeman, 1990; Freeman, 1975), retina (Ogden, 1973), visual cortex (Narici, Carozzo, Lopez, Ogliastro, & Sannita, 2003; Whittaker & Siegfried, 1983), auditory cortex (Pantev et al., 1991; Galambos, Makeig, & Talmachoff, 1981), and somatosensory system (Arnfred, Hansen, Parnas, & Mørup, 2007). These gamma band responses are typically phase locked to stimulation and the early latencies of these oscillations support a source early in the sweep of sensory information from the peripheral receptors to the cortex. For example, in the visual modality, evoked gamma localized to the visual cortex is immediately preceded by retinal oscillations (Lopez & Sannita, 1997). GBRs are also elicited with focal stimulation of the thalamic reticular nucleus (Macdonald, Fifkova, Jones, & Barth, 1998) but also occur spontaneously and appear to be an emergent property of laminar interconnected structures such as the cerebral cortex, hippocampus, and retina (Sukov & Barth, 2001).

If oscillations phase lock with stimulation, the result of an average across trials is present in the ERP. However, whether ERPs are entirely due to phase locking of oscillations is controversial (Klimesch, Sauseng, Hanslmayr, Gruber, & Freunberger, 2007; Min et al., 2007; Makeig et al., 2002). There is evidence that lower-frequency oscillations are coupled to high-frequency oscillations (Demiralp et al., 2007; Canolty et al., 2006; Lakatos et al., 2005) independent of ERP dynamics. The precise relationship between ERPs and oscillatory responses is uncertain. The purpose of the present article is not to resolve this question but to use the GBR as an additional tool to elucidate the mechanisms of intermodal attention. We follow (Sannita, 2000) and use the term “gamma band oscillatory responses” rather than “evoked” gamma which implies a new oscillatory response that did not exist in the prestimulus period.

Although early GBRs are a sensory response, like other sensory responses they are modulated with attention. In many respects, the effects of selective and spatial attention on GBRs echo what has been reported with ERPs. In the auditory modality, spatial attention in a fast dichotic listening task may operate early, modulating the magnetic 20–50 msec event-related average (Woldorff et al., 1993) and the scalp-recorded GBR earlier than 50 msec (Tiitinen et al., 1993). In the visual modality, the C1 response, which is the main index of V1 processing, may not be modulated with spatial attention (Di Russo, Martínez, & Hillyard, 2003; Foxe & Simpson, 2002; Noesselt et al., 2002; Hillyard & Anllo-Vento, 1998; Mangun, Buonocore, Girelli, & Jha, 1998; Clark & Hillyard, 1996), but may be modulated with other types of attention (Proverbio, Del Zotto, & Zani, 2007; Khoe, Mitchell, Reynolds, & Hillyard, 2005). However, recently, Kelly, Gomez-Ramirez, and Foxe (2008) report spatial attention modulation of the C1. Visual GBRs are also reported to be modulated with selective attention (Busch, Schadow, Fründ, & Herrmann, 2006). Tactile spatial attention also enhances GBRs in regions consistent with contralateral S1 (Bauer, Oostenveld, Peeters, & Fries, 2006). Intermodal attention modulates spontaneous gamma activity in the auditory cortex of the cat (Lakatos et al., 2004) and induced gamma increases over the sensory cortex of attended sensory modalities and decreases over the cortex of unattended sensory modalities (Sokolov, Pavlova, Lutzenberger, & Birbaumer, 2004). Here we present evidence that intermodal attention affects GBRs in the auditory, visual, and tactile sensory modality independent of the effects of spatial and intramodal attention.

Intermodal attention is typically assessed with ERPs in tasks where participants detect targets in one sensory modality while nontargets appear in the same modality as the target and also in other sensory modalities. For example, the subject is asked to detect a dim light in a series of slightly brighter lights, while sounds are also presented (Hötting, Rösler, & Röder, 2003; Talsma & Kok, 2001, 2002; Mehta, Ulbert, & Schroeder, 2000a, 2000b). Important for addressing intermodal attention, one can compare auditory nontargets when they are in the attended sensory channel to when another modality is the attended channel. In human vision, landmark studies using ERP methodology have demonstrated that spatial attention is a prerequisite for observation of attention to features and that spatial attention benefits extend to nontarget stimuli (Hillyard & Anllo-Vento, 1998). A further rationale for this selective attention design stems from the fact that ERPs are quite sensitive to arousal. Through requiring an active selective attention task in each modality, arousal can be equated across blocks, and focusing the analysis on nontargets eliminates any motor response confounds.

The intramodal design for investigating intermodal attention has two potential mechanisms of attention at work. The first is intermodal enhancement and the second is intramodal selective attention within the attended channel. For example, if a dim target must be selected from a series of brighter targets, target and nontarget visual stimuli are both in the attended sensory channel, but the selective attention demands of the task may require that each visual stimulus be further evaluated to determine if it matches the target template. In this circumstance, attention might operate to delay gating until a slightly later stage of processing to reduce false alarms. In the present study, we use a different approach and required selection only between sensory modalities rather than both within and between modalities. To compensate for response-related contamination of the ERPs and GBRs, we limit our perceptual analysis to 200 msec and earlier, a time range less likely to be contaminated with response-related potentials.

The influence of space is often overlooked in intermodal attention studies (reviewed by Eimer, 2001). For example, in previous studies, researchers presented auditory stimuli through headphones and the visual stimuli on a computer monitor (Alho, Woods, & Algazi, 1994; Woods, Alho, & Algazi, 1993). Spatial selective attention can operate early (25 msec) in a fast-paced dichotic listening task (Woldorff et al., 1993). Thus, if stimuli are not spatially coregistered, the effects of intermodal attention could be influenced by spatial attention. The few studies controlling for the effects of spatial attention by implementing spatial coregistration of stimuli across multiple modalities report intermodal attention effects as early as 70 msec for auditory stimuli (Talsma & Kok, 2001, 2002; Eimer & Schroger, 1998), but these tasks involve intramodal selective attention as well. Here we address the open question of how early does intermodal attention operate with spatially coregistered stimuli in an intermodal selection task without intramodal attention demands?

Another way intermodal attention has been examined is through cueing studies where participants are cued to which modality they should expect a target and after some delay, stimuli in the expected or unexpected modality are presented. This design is an elegant way to examine preparatory and subsequent redirection of attention (Foxe & Simpson, 2005; Foxe, Simpson, Ahlfors, & Saron, 2005). However, findings relating to the latency at which attention operates in such a cued task may not directly generalize to tasks where sustained attention is required. For example, in a tactile attention task, the effects of attention were observed 60 msec earlier than when sustained attention was required relative to when subjects were asked to switch attention trial to trial (Eimer & Forster, 2003). In the present study, we used a block design to address the question of whether sustained intermodal attention affects early sensory processing.

In summary, there is evidence that some aspects of attention, primarily spatial, influence early sensory processing, but the effects of intermodal attention without the differential effects of spatial and intramodal selective attention are unknown. We propose that sustained intermodal attention may tune early sensory cortices, allowing attentional modulation to operate at latencies similar to or earlier than has been reported for spatial attention. In the present study, we tested whether intermodal attention elicits enhanced processing in three sensory modalities—auditory, visual, and tactile—as indexed by sensory ERP components and GBRs at the latencies and scalp topographies corresponding to the earliest stages of sensory processing.

METHODS

Participants

Experimental participants with normal or corrected-to-normal visual acuity; normal hearing; no history of significant medical, neurological, or psychiatric illness; and ages 18 to 35 years were recruited from the university community. Twenty-five participants participated in the experiment. Four participants were immediately excluded due to excessive physiological artifacts (blinks, muscle). Additional participants were excluded when there were insufficient trials for adequate signal to noise in the ERP for a particular analysis, as determined by a subjective assessment of each participant's data quality separately validated by ranking participants by the number of trials per average. The mean age of the 21 participants was 23 years; all were right handed and 10 were women. The University of California Berkeley Institutional Review Board approved the study and written informed consent was obtained from all participants.

Electrophysiological Recording Procedures

The EEG was collected in a sound-attenuated chamber at a 1024-Hz digitization rate with a 72-channel Biosemi system. Signals were recorded relative to the Common Mode Sense (CMS) active electrode and then digitally re-referenced to the nose and down-sampled to 512 Hz when analyzed for ERPs and to 256 Hz for GBRs. The Biosemi system uses a CMS active electrode and a passive electrode (DRL) to form a feedback loop that drives the average potential of the system as close as possible to the reference voltage in the analog–digital box (see www.biosemi.com). The 64 head channels were positioned according to the extended 10–20 system, plus external electrodes (two mastoid, three ocular, one nose).

Stimulus Presentation and Experimental Apparatus

Stimuli were presented with E-prime software operating on a PC (Windows 98) with a SoundblasterLive Soundcard. Digital stimulus markers were recorded with the EEG and timing accuracy was verified via a photodiode and oscilloscope. Instructions and visual stimuli were presented via a DLP projector (Plus U5-132 Digital Projector) at a 60-Hz refresh rate onto an acoustically permeable screen (Dazian Fabrics). Auditory, visual, and vibrotactile stimuli were 300 msec in duration. Participants used a forehead rest to maintain head position across blocks with jaw relaxed. Visual stimuli were a flashed blue square (4° visual angle) 10°–15° to the left or right of a central white fixation cross on a black background. The ambient light from the DLP projector was the only light source. Participants were instructed to keep their eyes at a fixation cross and eye movements were monitored with closed-circuit television and the EOG. Contaminated trials were removed from analysis. Auditory stimuli were a 60-dB SPL broadband sound with 5 msec on- and offset attenuation played through a speaker (Orb Audio) placed behind the screen out of view directly behind the location of the visual stimuli (see Figure 1). The sound, a duck quack, had a fundamental frequency of 300 Hz with 10 harmonics; the maximum power was at 600 Hz. Tactile stimuli were presented with a modified mini speaker (Philmore), with paper cone removed to reduce audibility. A 200-Hz tone with 5 msec on- and offset attenuation was delivered to the fingertips. The speaker was mounted in a sound-insulated foam mitten placed on the subject's hand, which was also placed behind the screen, arm outstretched, directly behind the visual stimulus (see Figure 1). To eliminate pressure differences across the session, fingertips within the mitten were attached to the vibration device with elastic. The hand was supine with fingertips at 90° to the palm of the hand. Vibrations were undetectable by the sound–pressure meter. At the beginning of each session, participants were asked to report vibrations with and without their fingertips on the vibration device; the amplitude was then adjusted within a small range so each participant was unable to hear the vibration.

Figure 1. 

Schematic diagram of experimental apparatus. (A) Auditory, vibrotactile, and visual stimuli were spatially coregistered to the left or right of fixation and participants responded to targets with the opposite hand. The stimulated hand was placed behind a sound-permeable screen onto which visual stimuli were flashed via a DLP projector. (B) The visual stimuli, speaker, and vibration device were aligned in azimuth and elevation. Stimulus waveforms for the auditory and tactile modalities are shown in the legend.

Figure 1. 

Schematic diagram of experimental apparatus. (A) Auditory, vibrotactile, and visual stimuli were spatially coregistered to the left or right of fixation and participants responded to targets with the opposite hand. The stimulated hand was placed behind a sound-permeable screen onto which visual stimuli were flashed via a DLP projector. (B) The visual stimuli, speaker, and vibration device were aligned in azimuth and elevation. Stimulus waveforms for the auditory and tactile modalities are shown in the legend.

Task Design

Auditory, visual, and tactile stimuli were presented in an intermodal attention task. Simultaneous auditory–visual stimuli were also presented, but because both intermodal attention and multisensory interactions are expected to affect these bimodal stimuli, findings related to these stimuli are beyond the scope of the present article and are not described here. Participants were instructed to respond to stimuli in one sensory modality in each of three blocks as quickly and accurately as possible, regardless of whether that stimulus was paired with another stimulus in a different modality. The order of blocks was counterbalanced across subjects. Reaction times were collected with a serial response box (Psychology Software Tools). Stimuli were presented either to the left or right of fixation, counterbalanced by subject, and subjects responded with the opposite hand. We presented stimuli in pseudorandom order with the following proportions to equalize the number of responses to and the number of each type of stimulus across blocks. In Equalize Stimuli blocks (EqStim), each stimulus type (auditory, visual, auditory–visual, and tactile) was 25% of the total. In the Equalize Responses (EqResp) blocks, the proportion of tactile stimuli was increased to equalize the total number of responses across blocks (auditory = 20%, visual = 20%, auditory–visual = 20%, and tactile = 40%). A total of 1860 stimuli were presented in pseudorandom order with the ISI jittered at random intervals between 1400 and 1900 msec. To maintain the alertness of our participants, blocks were broken into 4- to 5-min sessions. With short breaks following each session, recording sessions were about 90 min in duration.

Data Analysis

Behavioral Data

Reaction times, hit rates, and false alarm rates were analyzed with a Proportion (EqStim, EqResp) × Stimulus (auditory, visual, tactile) ANOVA. Main effects and interactions were followed up with post hoc t tests Bonferroni corrected for the number of post hoc comparisons within each dependent variable. Hit rate is the number of correct responses divided by the total number of trials where a response was required. False alarm rate is the number of responses when a target was not present divided by the total number of trials where a response should be withheld. To evaluate performance in the context of signal detection theory, d′ and centered criterion (c) were also estimated and analyzed. Values for d′ were estimated for each subject by subtracting Z(false alarms) from Z(hits). Values for c were estimated by subtracting d′/2 from Z(false alarms). If no errors were present, a value of 0.5 was assigned to the empty category. Values for c were also subjected to a one-sample t test to test the null hypothesis that the bias was zero.

ERP Data

Epochs time-locked to stimulus onset were extracted from −130 to 770 msec, averaged, and band-pass filtered off-line from 1 to 50 Hz. Epochs containing horizontal or vertical eye movements or blinks, excessive muscle activity, or isolated electrode malfunction were excluded from the average. For each subject, the ERPs were digitally re-referenced to the nose. The baseline for each channel (defined relative to the stimulus onset as −100 to 0 msec) was subtracted for each average for statistical comparisons. Our dependent variable (peak amplitude) was calculated with a two-step method. First, peaks of established ERP components were detected in the grand average across participants (GAV) in a priori electrodes and latencies of interest determined from pilot data and ERP literature. We used a semiautomatic peak detection algorithm (Brain Vision Analyzer software; Gilching, Germany). If a peak was not apparent in the grand average, the latency range was not analyzed further. Electrode locations for participants receiving right-sided stimulation were digitally flipped across the midline to simulate all stimuli appearing to the left of fixation to allow comparison across subjects receiving stimulation on different sides. Electrodes contralateral and ipsilateral to the side of stimulation are indicated by a “c” or “i,” respectively. A priori electrodes and latencies of interest where we expected a peak in the ERP were: Auditory: Na (10–30 msec, Fz), P50 (30–50 msec; FCz), P1 (60–130 msec; FCz), N1 (115–170 msec; Cz); Visual: C1 (50–90 msec, CPz), P1 (80–130 msec, P10-c), N1 (120–200, P10-c); and Vibrotactile: N40 (10–50 msec, Pz), P80 (30–90, FC6-c), N80 (60–100, C4-c), P100 (80–130, FCz), N160 (120–200 msec, FC6-c). To increase signal relative to noise, amplitudes were then averaged across neighboring electrodes to form channel groups of interest (Auditory: F, FC, C; Visual: CP, P-c; and Vibrotactile: P, FC-c, C-c, FC) as indicated in Figure 2. Second, a time window was centered on the detected peak and the mean voltage within the time window was calculated for each subject. Window sizes were adjusted from 10 to 60 msec as latencies increased as specified in Table 2.

Figure 2. 

Electrode groups. Electrodes groups were defined surrounding electrodes of interest for each component. Electrode groups on the midline are labeled according to the first letter of the center electrode (e.g., Fz → F). Electrode groups contralateral to the side of stimulation are indicated with a “c.”

Figure 2. 

Electrode groups. Electrodes groups were defined surrounding electrodes of interest for each component. Electrode groups on the midline are labeled according to the first letter of the center electrode (e.g., Fz → F). Electrode groups contralateral to the side of stimulation are indicated with a “c.”

We hypothesized that intermodal attention would increase the amplitude of ERP components prior to 200 msec at specific electrode sites. We first tested whether other independent variables affected the peak amplitudes with a Stimulus proportion (EqStim, EqResp) × Attention modality (auditory, visual, tactile) ANOVA with hemifield (left, right) as a between-subjects factor. Subjects with insufficient numbers of trials per cell were excluded from this analysis. We followed with tests of whether ERP amplitude differed between the two unattended modalities and performed planned one-tailed t tests to test the hypothesis that attended stimuli elicited larger ERP components than the average of the ERPs following these stimuli when attention was directed to either of the other two sensory modalities.

GBR Data

Our prediction was that intermodal attention directed to a stimulus increases power in the GBR in the initial response to the stimulus (<150 msec) in each of the three sensory modalities. Because we are recording from the scalp, we defined a narrow frequency range a priori (30–55 Hz) to reduce the influence of muscle artifacts or the potential influence of 60 Hz line noise. We directed our analysis to the electrodes of interest from the ERP analysis. We first extracted epochs 200 msec prior to the stimulus to 800 msec following the stimulus. For each subject, for each channel of interest, the event-related spectral perturbation (ERSP) was calculated from the ERP (which preserves oscillations phase-locked to the stimulus) using a wavelet-based analysis implemented with the TIMEF function of EEGLAB software (Delorme & Makeig, 2004; Makeig, Debener, Onton, & Delorme, 2004; Makeig, Delorme et al., 2004). We used a three-cycle wavelet (0.5 expansion factor, window size 64 samples, pad ratio 8) to examine the frequency composition of 1-sec epochs down-sampled from 1024 to 256 Hz. The precise time and frequency resolution varies across frequencies. At each frequency point, the power (dB) for that frequency in the 50-msec prestimulus baseline period was subtracted and the resulting power was then averaged across channels of interest. Planned comparisons of attended to unattended stimuli were calculated by performing a pointwise one-way t test on the difference between power in attended and unattended GBR images from 50 msec before the stimulus to 150 msec following the stimulus. Areas of significance within the time–frequency image can be interpreted as average increases in power relative to baseline. As an average signal, the ERP preserves only oscillations time- and phase-locked to the stimulus, a property of sensory evoked potentials, while reducing the amplitude of oscillations which are not time- and phase-locked to the stimulus, such as noise in the prestimulus period or “induced” oscillations. Thus, a GBR power increase calculated from the ERP indicates either increased phase-locking or increased power after stimulus onset. There is evidence that these oscillatory power increases can be due to a combination of phase-resetting of existing oscillations and increased power (Min et al., 2007), and we chose this ERP-based method to capitalize on this property. The average power with the significant time and frequency range was then plotted at each electrode to yield scalp topographies of the power. To confirm that the electrodes of interest did not unduly influence the topographies, we also generated topographies of the average GBR across all electrodes.

RESULTS

Behavioral Data

There was a main effect of stimulus modality for response time [F(2, 40) = 30.2, p < .05], hit rate [F(2, 40) = 6.5, p < .05], d′ [F(2, 40) = 10.7, p < .05], and criterion (c) [F(2, 40) = 6.2, p < .05], but not false alarm rate (p > .05). Corrected post hoc t tests reveal that visual stimuli were faster than either auditory (p < .05) or tactile stimuli (p < .05). Visual d′ was higher than auditory (p < .05) and tactile stimuli (p < .05). The c for visual stimuli was smaller than for auditory stimuli (p < .05) and visual hit rate was higher than auditory hit rate (p < .05). Across conditions, c values were larger than zero, indicating a tendency to respond stimulus absent (p < .05). Across all behavioral measures, there was no main effect of proportion (p > .05), but proportion interacted with stimulus for three behavioral measures: false alarm rate [F(2, 40) = 7.72, p < .05], response time [F(2, 40) = 6.1, p < .05], and c [F(2, 40) = 3.6, p < .05]. However, proportion was only significantly different within a stimulus modality for tactile false alarm rate (p < .05). False alarms were rare overall and the increase from 0.6% to 0.9% with a larger proportion of tactile stimuli corresponds to an average increase of 0.5 false alarms (Table 1).

Table 1. 

Behavioral Differences for Auditory, Vibrotactile, and Visual Intermodal Attention


Auditory (A)
Visual (V)
Vibrotactile (T)
Mean
SEM
Mean
SEM
Mean
SEM
Response time (msec) EqResp 503 18 437 17 490 15 
EqStim 484 18 426 16 506 17 
Hit rate (%) EqResp 86.9 3.1 95.4 1.4 91.6 2.8 
EqStim 87.8 3.2 96.7 0.7 91.4 2.8 
False alarm rate (%) EqResp 0.8 0.2 0.6 0.1 0.9 0.2 
EqStm 1.2 0.2 0.7 0.1 0.6 0.2 
d′ EqResp 4.0 0.2 4.6 0.1 4.1 0.1 
EqStim 3.9 0.2 4.5 0.1 4.4 0.2 
Criterion c EqResp 0.53 0.09 0.32 0.06 0.41 0.08 
EqStim 0.44 0.08 0.26 0.05 0.50 0.09 

Auditory (A)
Visual (V)
Vibrotactile (T)
Mean
SEM
Mean
SEM
Mean
SEM
Response time (msec) EqResp 503 18 437 17 490 15 
EqStim 484 18 426 16 506 17 
Hit rate (%) EqResp 86.9 3.1 95.4 1.4 91.6 2.8 
EqStim 87.8 3.2 96.7 0.7 91.4 2.8 
False alarm rate (%) EqResp 0.8 0.2 0.6 0.1 0.9 0.2 
EqStm 1.2 0.2 0.7 0.1 0.6 0.2 
d′ EqResp 4.0 0.2 4.6 0.1 4.1 0.1 
EqStim 3.9 0.2 4.5 0.1 4.4 0.2 
Criterion c EqResp 0.53 0.09 0.32 0.06 0.41 0.08 
EqStim 0.44 0.08 0.26 0.05 0.50 0.09 

SEM = standard error of the mean.

ERP Data

Visual Stimuli

Early visual components apparent in the grand average were the C1 (72 msec), P1 (120 msec), and N1 (176 msec) (Figure 3). In the Stimulus proportion × Attention × Hemifield ANOVA, five subjects were excluded due to insufficient numbers of trials for a reliable ERP for each cell. At the CP electrode group, we did not find evidence that the visual components were affected by stimulus proportion or hemifield (p > 0.10), but did find the predicted main effects of attention for the C1 [F(2, 28) = 5.93, p < .05] and N1 [F(2, 28) = 6.1, p < .05] components, but not the P1 (p > .10). When we divide the C1 into an early (62–72 msec) and late (72–82 msec) phase, there was a main effect of attention only for the early phase [F(2, 28) = 3.55, p < .05]. The late phase of the C1 tended to be larger [F(2, 28) = 2.85, p < .10]. We performed our planned follow-up t tests on the ERPs generated by averaging across hemifield and stimulus proportion, and after collapsing across these cells, all 21 subjects had sufficient trials to be included in the analysis. We found no evidence for a difference between the two unattended conditions (attend tactile and attend auditory) (p > .10) so we tested whether the ERP to the attended stimulus was larger than the average of the two unattended conditions (see Table 2). At the latency of the C1, which had a negative polarity, amplitudes within a 10-msec window were more negative at electrode group CP when the visual modality was attended rather than when the auditory or tactile modality was attended (p < .05). We further tested the early and late phases of the C1 for the effects of visual attention. The average voltage within the C1-early window was more negative with visual attention (p < .05), and although the ANOVA revealed no main effect of attention in the late phase of the C1, t tests reveal a significant difference in the C1-late window (p < .05) for electrode group CP, but not for group P-c that reflects the topography of the P1 over posterior scalp contralateral to the stimulus. P1 amplitudes were also not different at group P-c but amplitudes within the N1 latency window were larger with visual attention (p < .05). We considered the possibility that alpha noise in the prestimulus baseline may have shifted the waveform negatively with a 100-msec baseline. When the analysis was repeated with a 50-msec baseline, which more effectively centers the attended and unattended waveforms, attention effects were statistically significant for all three observed early components: the C1 [t(20) = −2.13, p < .05], early [t(20) = −2.45, p < .05], but not late [t(20) = −1.94, p > .10], P1 [t(20) = 1.70, p < .05], and N1 [t(20) = −3.9, p < .05]. We further verified the C1 effect with follow-up analyses. We split the subjects into two groups, odd and even, and found that the attention effect was significant at C1 for both odd [t(10) = −2.13, p < .05] and even [t(9) = −2.35, p < .05] subject groups. The scalp topographies of the difference waves (attended minus unattended; Figure 3) are consistent with modulations of the C1, P1, and N1. The topography of the C1 attention effect does not show evidence of an overlapping contribution from the posterior–lateral P1.

Figure 3. 

Intermodal attention effect on visual ERPs. (A) ERPs in response to visual stimuli when the visual modality is attended (red) or not attended (blue) are shown in two midline electrode groups (C and P) and two posterior–lateral electrode groups (P-i, ipsilateral to stimulation and P10-c, contralateral). The difference wave between attended and unattended visual ERPs is overlaid (black). The C1 and N1 were modulated with visual attention (p < .05) with a 100-msec baseline; the C1, P1, and N1 modulation were significant with a 50-msec baseline. (B) A posterior view of the scalp topographies of the attention effects (Attend − Unattend) is shown at the latencies of the C1, P1, and N1. Smaller insets show the topography of each component of the average ERP across all visual stimuli (attended and unattended).

Figure 3. 

Intermodal attention effect on visual ERPs. (A) ERPs in response to visual stimuli when the visual modality is attended (red) or not attended (blue) are shown in two midline electrode groups (C and P) and two posterior–lateral electrode groups (P-i, ipsilateral to stimulation and P10-c, contralateral). The difference wave between attended and unattended visual ERPs is overlaid (black). The C1 and N1 were modulated with visual attention (p < .05) with a 100-msec baseline; the C1, P1, and N1 modulation were significant with a 50-msec baseline. (B) A posterior view of the scalp topographies of the attention effects (Attend − Unattend) is shown at the latencies of the C1, P1, and N1. Smaller insets show the topography of each component of the average ERP across all visual stimuli (attended and unattended).

Table 2. 

Intermodal Attention Effects on ERP Components

Component
Electrode
Latency (msec)
Mean Difference (>μV)
SEM
t(20)
p Value (One-tailed)
(A) Visual 
C1 CP 67–77 −0.96 0.37 −3.08 .003** 
C1-early CP 62–72 −0.86 0.39 −2.45 .012* 
C1-late CP 72–82 −0.57 0.34 −1.94 .033* 
 P-c 72–82 −0.25 0.41 −0.85 ns 
P1 P-c 110–130 0.23 0.31 0.41 ns 
N1 P-c 166–186 −2.1 0.57 −3.90 .002** 
 
(B) Auditory 
Na 24–34 −0.70 0.26 −3.10 .003** 
P1 FC 85–95 0.73 0.31 −2.01 .024* 
N1 138–158 0.16 0.50 0.11 ns 
 
(C) Vibrotactile 
N40 40–50 −0.33 −0.33 −1.18 ns 
P80 FC-c 75–95 0.15 0.15 0.25 ns 
P100 FC 80–120 0.30 0.30 0.73 ns 
N160 FC-c 135–195 −0.45 0.32 −1.73 .049* 
Component
Electrode
Latency (msec)
Mean Difference (>μV)
SEM
t(20)
p Value (One-tailed)
(A) Visual 
C1 CP 67–77 −0.96 0.37 −3.08 .003** 
C1-early CP 62–72 −0.86 0.39 −2.45 .012* 
C1-late CP 72–82 −0.57 0.34 −1.94 .033* 
 P-c 72–82 −0.25 0.41 −0.85 ns 
P1 P-c 110–130 0.23 0.31 0.41 ns 
N1 P-c 166–186 −2.1 0.57 −3.90 .002** 
 
(B) Auditory 
Na 24–34 −0.70 0.26 −3.10 .003** 
P1 FC 85–95 0.73 0.31 −2.01 .024* 
N1 138–158 0.16 0.50 0.11 ns 
 
(C) Vibrotactile 
N40 40–50 −0.33 −0.33 −1.18 ns 
P80 FC-c 75–95 0.15 0.15 0.25 ns 
P100 FC 80–120 0.30 0.30 0.73 ns 
N160 FC-c 135–195 −0.45 0.32 −1.73 .049* 

The mean difference between attended and unattended amplitudes within a window surrounding the grand-average peaks was compared to zero.

SEM = standard error of the mean.

*

p < .05.

**

p < .01.

ns

= p > .10.

Auditory Stimuli

Early peaks detected in the auditory grand average were the Na (29 msec), P1 (90 msec), and N1 (148 msec) (see Figure 4). Results of statistical comparisons at these components are summarized in Table 2. In the Stimulus proportion × Attention × Hemifield ANOVA, four subjects were excluded due to insufficient numbers of trials for a reliable ERP for each cell. We did not find evidence that auditory components were affected by stimulus proportion or hemifield (p > .10), but did find the predicted main effects of attention for the Na [F(2, 30) = 5.13, p < .05] and P1 [F(2, 30) = 3.44, p < .05] components, but not the later N1 (p > .10). By averaging across stimulus proportion and hemifield cells, all 21 subjects had sufficient trials to be included in the planned t-test analysis. There was no evidence for a difference between the two unattended condition (attend tactile and attend visual) (p > .10). Planned comparisons between attended and the average of the two unattended conditions revealed early attention effects (see Table 2). At the earliest latency examined (Na), mean amplitudes at electrode group F were more negative when the auditory modality was attended than when attention was deployed to the visual or tactile modality (p < .05). Mean amplitudes at the P1 latency window were more positive at group FC when the auditory modality was attended (p < .05). Mean amplitudes within the N1 latency window were not different at group C with attention to the auditory modality (p > .10). Scalp topographies of the difference wave (attended minus unattended; Figure 4) are consistent with the topography of the P1 considered to be in the superior temporal plane. The Na difference wave is more widespread.

Figure 4. 

Intermodal attention effect on auditory ERPs. (A) ERPs in response to auditory stimuli when the auditory modality is attended (red) or not attended (blue) are shown in two midline electrode groups at frontal (F) and central (C) sites. The difference wave between attended and unattended auditory ERPs is overlaid (black). The inset of group F emphasizes the modulation of the Na and P1 with attention (p < .05). (B) Scalp topographies of the attention effect (Attend − Unattend) are shown at the latency of the Na and P1. The top row depicts the scalp contralateral to the side of stimulation, the bottom, ipsilateral. Smaller insets show the topography for the average ERP across all auditory stimuli (attended and unattended).

Figure 4. 

Intermodal attention effect on auditory ERPs. (A) ERPs in response to auditory stimuli when the auditory modality is attended (red) or not attended (blue) are shown in two midline electrode groups at frontal (F) and central (C) sites. The difference wave between attended and unattended auditory ERPs is overlaid (black). The inset of group F emphasizes the modulation of the Na and P1 with attention (p < .05). (B) Scalp topographies of the attention effect (Attend − Unattend) are shown at the latency of the Na and P1. The top row depicts the scalp contralateral to the side of stimulation, the bottom, ipsilateral. Smaller insets show the topography for the average ERP across all auditory stimuli (attended and unattended).

Tactile Stimuli

Tactile components apparent in the grand average were the N40 (45 msec), P80 (85 msec), P100 (100 msec), and N160 (165 msec) (Figure 5). Results of statistical comparisons at these components are summarized in Table 2C. In the Stimulus proportion × Attention × Hemifield ANOVA, four subjects were excluded due to insufficient numbers of trials for a reliable ERP for each cell. In contrast to the results for auditory and visual stimuli, we did not find a main effect of attention for any of the vibrotactile components (p > .10). Furthermore, we did not find evidence that the earliest two components (N45 and P80) were affected by stimulus proportion or hemifield. However, we did find evidence that the later tactile components were affected by the change in stimulus proportion. The P1 [F(1, 15) = 5.1, p < .05] and N160 [F(1, 15) = 5.5, p < .05] components were each larger in the EqStim blocks (where there was a smaller proportion of tactile trials, 25% of total) than the EqResp blocks (where there was a larger proportion of tactile trials, 40%). For symmetry with the analysis of the auditory and visual modalities, we still performed our planned follow-up t tests on the ERPs generated by averaging across these cells and with all 21 subjects included. The effects of tactile attention relative to the average of auditory and visual stimuli were not significant for the N40, P80, or P100. There was an effect of attention at the N160 component (p < .05).There were no differences between the two unattended conditions for any component (p > .10).

Figure 5. 

Intermodal attention effect on vibrotactile ERPs. (A) ERPs in response to vibrotactile stimuli when the tactile modality is attended (red) or not attended (blue) are shown in two midline electrodes, fronto-central (FC) and posterior (P) and two frontal–lateral electrodes (FC-i, ipsilateral to stimulation and FC-c, contralateral). The difference wave between attended and unattended is overlaid (black). The first significant effect with attention was at the latency of the N160 (p < .05) in electrode group FC-c. (B) Scalp topographies of the attention effect (Attend − Unattend) are shown at the latency of the N160. The smaller insets show the topography of the N160 across all vibrotactile stimuli (attended and unattended).

Figure 5. 

Intermodal attention effect on vibrotactile ERPs. (A) ERPs in response to vibrotactile stimuli when the tactile modality is attended (red) or not attended (blue) are shown in two midline electrodes, fronto-central (FC) and posterior (P) and two frontal–lateral electrodes (FC-i, ipsilateral to stimulation and FC-c, contralateral). The difference wave between attended and unattended is overlaid (black). The first significant effect with attention was at the latency of the N160 (p < .05) in electrode group FC-c. (B) Scalp topographies of the attention effect (Attend − Unattend) are shown at the latency of the N160. The smaller insets show the topography of the N160 across all vibrotactile stimuli (attended and unattended).

Gamma Band Oscillatory Responses

We performed planned t tests comparing attended and unattended stimuli with the full cohort of 21 subjects. As shown in the left panel of Figure 6, for each modality we observed a significant power increase [t(18)> 1.73, p < .05, one-tailed] in the gamma range for evoked oscillations in response to attended relative to unattended stimuli. For attended visual stimuli relative to unattended, there was a significant increase in power in the 32- to 44-Hz frequency range the with an average onset of 47 msec. For attended auditory stimuli relative to unattended, power increased in the 45- to 50-Hz frequency range with an average onset of 57 msec. For attended vibrotactile stimuli relative to unattended, power increased in the 42- to 53-Hz frequency range with an average onset of 27 msec. The right panel of Figure 6 demonstrates the distribution of the significant power increase across the scalp. For the visual modality, power was distributed posteriorly. For the auditory modality, power was distributed centrally. For the vibrotactile modality, power was distributed over the contralateral fronto-central scalp. We validated that the initial selection of the channels of interest did not bias the resulting topographies by comparing them to the evoked ERSP image averaged across all channels.

Figure 6. 

Intermodal attention effects on gamma band oscillatory responses (GBRs). The left panel shows the ERSP images of the difference between attended and unattended ERSPs in the gamma band for visual (top), auditory (middle), and vibrotactile (bottom) stimuli averaged for channels indicated in white (right). Outermost contours indicate a significant difference between attended and unattended (p < .05, one-tailed; inner contours are drawn at intervals decreasing by .01). The plot below each ERSP image shows the average power within the frequency range of the attention effect over time. Green lines indicate the on- and offset of significance. Attention effects in the gamma band onset at 47 msec for the visual modality, 57 msec for the auditory modality, and 27 msec for the vibrotactile modality. The right panel shows the scalp topographies of the significant gamma band attention effect.

Figure 6. 

Intermodal attention effects on gamma band oscillatory responses (GBRs). The left panel shows the ERSP images of the difference between attended and unattended ERSPs in the gamma band for visual (top), auditory (middle), and vibrotactile (bottom) stimuli averaged for channels indicated in white (right). Outermost contours indicate a significant difference between attended and unattended (p < .05, one-tailed; inner contours are drawn at intervals decreasing by .01). The plot below each ERSP image shows the average power within the frequency range of the attention effect over time. Green lines indicate the on- and offset of significance. Attention effects in the gamma band onset at 47 msec for the visual modality, 57 msec for the auditory modality, and 27 msec for the vibrotactile modality. The right panel shows the scalp topographies of the significant gamma band attention effect.

DISCUSSION

We assessed whether intermodal attention operates early in neural processing, controlling for spatial attention and intramodal selective attention. We spatially coregistered auditory, tactile, and visual stimuli and required stimulus selection across sensory modalities (intermodal selection) rather than within a modality (intramodal selection). We found some evidence for early modulation of intermodal attention in each of the three sensory modalities. There were some differences between the results generated from ERP and GBR methods, as well as differences across sensory modalities. Although auditory and visual modalities were each modulated early with intermodal attention, tactile intermodal attention only manifested at the earliest latencies in the GBR. Although we do not directly compare nonspatial intermodal attention and spatial intramodal attention in the same study, relating our findings to previous studies provides some insights into how intermodal attention may fit into a hierarchical attention scheme.

In the ERP domain, visual processing was modulated at early stages of sensory processing as indexed by the C1 (72 msec), earlier than has generally been reported for selective spatial attention. Only the early phase of the C1 was affected by our attention manipulation, whereas the later phase—which may overlap with the P1 (Foxe & Simpson, 2002) extrastriate sources (Di Russo et al., 2003; Clark & Hillyard, 1996)—only tended to be affected by attention. The C1 is well accounted for by a source in the calcarine sulcus, inverting when stimuli are presented in the upper or lower hemifield (Di Russo et al., 2003). Despite careful attempts (Di Russo et al., 2003; Foxe & Simpson, 2002; Noesselt et al., 2002; Hillyard & Anllo-Vento, 1998; Mangun et al., 1998; Clark & Hillyard, 1996), the early phase of the C1 has not been shown to be modified with spatial attention with an intramodal selection task, with the exception of one recent report (Kelly et al., 2008), but there are other reports of attentional modulation of the C1 in different circumstances (Proverbio et al., 2007; Khoe et al., 2005). Here we report intermodal attention modulation at a latency and topography consistent with the C1. We cannot be certain that our early attention effect (62 msec) is the previously reported C1 that has been localized to the calcarine sulcus because our stimuli were not designed to show polarity inversion across the calcarine sulcus. Examination of the waveforms suggests that the C1 is not apparent in the unattended conditions, but we do not conclude that the C1 is absent as it may be simply attenuated in amplitude.

Further attention processing was indicated by increased amplitudes of the later components, generally considered to be in the extrastriate cortex, which are modified with spatial attention as early as 80 msec (Mangun et al., 1998). Intermodal attention increased amplitudes of the P1 (120 msec) and N1 (176 msec), but the P1 modulation was sensitive to manipulations of the prestimulus baseline. Due to alpha noise in the prestimulus period, the P1 effect was not significant with a 100-msec baseline, but when a shorter 50-msec baseline was chosen, the data revealed significant C1, P1, and N1 enhancement with visual attention. Prestimulus alpha activity may simply be noise, but could also be related to sustained intermodal attention (Min & Herrmann, 2007). Power in the gamma band was also increased with onset of visual intermodal attention at 47 msec with a posterior topography. The onset time of the GBR attention effect, combined with the modulation of the early phase of the C1, supports the interpretation that visual intermodal attention may operate as early, or earlier than, visual spatial selective attention.

Auditory processing was also modulated at relatively early latencies in the ERP and GBR domains. The latencies of the intermodal attention enhancements we report are similar to modulation with spatial selective attention. Spatial attention effects have been reported as early as the Pa (20–50 msec) component of the MLR when attending to one ear while ignoring another in a fast-paced dichotic listening task with high attention load (Woldorff et al., 1993) and is thought to index primary cortical processing (Woods, Alain, Covarrubias, & Zaidel, 1995; Liégeois-Chauvel, Musolino, Badier, Marquis, & Chauvel, 1994; Deiber, Ibañez, Fischer, Perrin, & Mauguière, 1988). GBR attention modulations were also elicited in a similar task (Tiitinen et al., 1993). The auditory ERP intermodal attention effect we report here (29 msec) is relatively late for the classic Na evoked by clicks (19 msec) and may reflect primarily a cortical source or a mixture of cortical and subcortical sources given the relatively broad topography of the attention effect at this latency. Data from patients with frontal cortex lesions support the notion that top–down influences from elements of distributed cortical attention networks can modify early sensory processing at the primary cortical level (Knight, Scabini, & Woods, 1989), suggesting a possible frontal involvement in this early modulation. Auditory attention effects on the GBR power onset at 57 msec with a centrally distributed topography. By analogy to ERPs, the topography may be consistent with a source in the superior temporal plane. It is notable that these early intermodal attention modulations were elicited without the constraints of a fast ISI and high load (Woldorff et al., 1993). Another point of interest is that the attention effect in the GBR domain is not significant until 57 msec compared to 29 msec in the ERP domain. The onset profile of the auditory attention effect on the GBR does appear to start earlier than the visual or tactile attention effect, but this was not significant until 57 msec; it may be that the GBR reflects a different neural mechanism than the mid-latency ERP response. As noted, the precise relationship between ERPs and GBRs is not known. Overall, our early modulations in the ERP and GBR domains suggest that auditory intermodal attention operates at similar latencies to auditory spatial selective attention.

In the tactile modality we found no evidence of attentional modulation of the ERPs at the earliest latencies. However, tactile attention modulated the GBR power relatively early, at 27 msec. This attention effect was distributed over the fronto-central scalp consistent with contralateral scalp distributions of early tactile ERP components (Hämäläinen, Kekoni, Sams, Reinikainen, & Näätänen, 1990). In the ERP domain, we found some evidence of tactile attention effects at the N160 latency with a bilateral topography consistent with a source in the somatosensory association cortex and similar to N140 attention effects reported elsewhere (Eimer & Forster, 2003; García-Larrea, Lukaszewicz, & Mauguière, 1995; Hämäläinen et al., 1990; Michie, Bearpark, Crawford, & Glue, 1987; Desmedt & Robertson, 1977). One explanation for why our results for the tactile modality may be different than the auditory and visual modality is that our choice of vibrotactile stimulus may not be optimal to show robust early attention effect in the ERP. However, behavioral results seem to indicate that the strength of the stimulation was comparable to the auditory stimuli. Responses to the tactile modality were only 4 msec slower, on average, than responses to the auditory stimuli, did not have the smallest d′, and were not the least accurate. Thus, it seems unlikely that the stimuli were simply too faint to elicit robust attention effects. Furthermore, in contrast to the ERP domain, in the spectral domain, we found that tactile attention increased GBR power at 27 msec. The early latency and contralateral fronto-central scalp topography of this oscillatory attention effect may indicate a modulation of the primary somatosensory cortex. Contralateral tactile ERPs have been attributed to the primary somatosensory cortex (Hämäläinen et al., 1990). It may be that we observed an effect in the GBRs but not the ERPs because of signal-to-noise advantages when honing in on one frequency range. Another option is that ERPs and GBRs reflect different brain processes that are differentially affected by intermodal attention.

The tactile modality was the only modality where we observed an effect of our stimulus proportion manipulation in the ERPs. We observed larger P1 and N1 components with smaller proportions of tactile stimuli but the proportion manipulation did not affect auditory and visual stimuli. This may be because the proportion of tactile stimuli was increased from 25% in EqStim blocks to 40% in EqResp blocks, whereas the proportion of auditory and visual stimuli were similar in these two blocks (25% and 20%, respectively). Thus, it may be that tactile stimuli habituated in the EqResp blocks, reducing their amplitudes. This is one explanation for our inability to detect tactile attention changes in the ERP. Perhaps the higher proportion of tactile ERPs reduced the amplitudes of the early components so we were unable to detect a robust signal. However, it is important to note that we did not find evidence of an interaction between the stimulus proportion and intermodal attention. In the behavioral domain, only false alarm rates differed with proportion within any stimulus modality and only in the tactile modality. The false alarm rate was small and the reported increase in false alarms with increasing proportion of tactile stimuli corresponded to an average increase of ½ a false alarm, which suggests that the influence of stimulus proportion on false alarm rate for the tactile modality was minimal.

Recall that the stimulus proportion manipulation was included to balance the stimulus–response requirements of the different stimulus types as our task included auditory–visual bimodal stimuli. The analysis of these bimodal stimuli is outside the scope of the present article because attentional modulations may include both intermodal attention effects as well as multisensory interactions which merit separate consideration. However, we should note that the inclusion of the bimodal stimuli might mean that auditory and visual attention blocks would be categorically different. For example, in an auditory block, subjects cannot always assign a no-go response to the visual stimulus because sometimes it is paired with an auditory stimulus. However, in a tactile block, all auditory and visual stimuli require a no-go response. For this reason, we directly compared the two unattended blocks for each stimulus types. We found no evidence of a difference between the two unattended conditions for each sensory modality. Perhaps these go and no-go response contingencies do not operate at the earliest latencies.

We also designed our experiment to equate the task demands and arousal across blocks, namely, selecting a stimulus in one modality at the expense of stimuli in other sensory modalities. In our experiment, stimulus detection in all modalities was well above threshold (d′ > 3.9) and the task was performed with high accuracy (>85% hits) in all conditions. We took care to present stimuli at a detectable but not overwhelming level and subjectively equated the stimuli in terms of how dim, quiet, or soft the stimulation. However, because we did not perform an extensive psychophysical procedure to equate stimuli, there were still behavioral differences across modalities. Therefore, we should be cautious when directly comparing the relative latencies of attention modulation in the three sensory modalities. For example, visual stimuli were fastest and had the highest d′. This may explain why visual stimuli were modulated relatively earlier in the processing stream relative to spatial attention than the auditory modality, but this empirical question awaits further experiments.

An important theoretical issue to consider is that stimuli were all presented in an attended spatial location and we do not directly compare the influence of intermodal attention at attended and unattended locations. It is interesting that in another study reporting C1 attention effects (Khoe et al., 2005), competing stimuli were also presented in an attended spatial location. A reasonable speculation is that spatial attention interacts with or is a prerequisite for intermodal attention to elicit the early modulations we report.

There are methodological issues to consider. First, the method in which we coregister stimuli has spatial limitations. With our apparatus, stimulation in the three sensory modalities was coregistered precisely in azimuth and altitude and participants could not see the speaker or vibration device, however, the devices were separated by 2 to 4 cm in the depth plane due to equipment limitations. It seems unlikely that such a small separation in the depth plane would elicit earlier spatial attention effects than have been reported for large spatial separations across the midline. A further limitation to the present study is that when selection takes place across sensory modalities, the comparison between attended and not-attended stimuli is also a comparison between target and nontarget stimuli. We accept this limitation as a necessary tradeoff in investigating intermodal selection without the influence of intramodal selection and only compare attended (target) and not-attended (nontarget) stimuli in the first 200 msec. Furthermore, a claim that our amplitude differences with intermodal attention were due to contamination by response-related ERPs would have to account for the fact that although responses to attended visual stimuli were 30 msec faster than responses to auditory stimuli, attention modulation of the ERP is 30 msec later than for the auditory modality. Another issue to consider is that a blocked experimental design is susceptible to differences across blocks that are not attention related, but we counterbalanced blocks across participants to control for order effects across participants. Further controls were applied to the apparatus across blocks to prevent drift in the position of the participants relative to the peripheral stimulators. We did not find evidence for a hemispheric difference in the present study and collapsed across presentation hemifield in our analysis.

Because many attention studies, to date, have found attention modulations on ERPs, it is not surprising that the influences of attention would be evident in the spectrotemporal domain as well. Indeed, there is increasing evidence that attention effects can modulate both oscillations phase-locked to the stimulus such as the ones utilized in the present article and induced oscillations. In the auditory and visual modality, the GBR analysis confirms and extends the ERP results. In the tactile modality, the GBRs demonstrated earlier effects of attention than the ERPs. Examining GBRs may be prudent when looking for effects of attention on small-amplitude, early-evoked potentials.

The current data support the hypothesis that intermodal attention may operate early in sensory processing in the visual, auditory, and possibly vibrotactile modalities. The view that intermodal attention operates early is consistent with imaging studies reporting modulation of sensory cortices with intermodal attention (Tournev et al., 1999; Woodruff et al., 1996), but offers further information about the latency at which such modulations occur. The early latencies observed in the current study support the hypothesis that intermodal attention operates early in the sweep of sensory information to the primary and secondary association cortices at latencies as early, or in some cases earlier, than has been reported with spatial attention manipulations.

Acknowledgments

We thank the following research assistants who provided diligent and excellent technical assistance: Milky Kohno, Elaine Wu, Genessa Giorgi, David Horton, and Sara LaHue for their valuable assistance with building the experimental apparatus (G. G., D. H.), data collection (M. K., E. W.), preprocessing (M. K., E. W., S. L.), and manuscript preparation (S. L.). We are grateful for the constructive criticisms of the anonymous reviewers and financial support contributed by NINDS NS21135 and PO 40813 (R. T. K.) and NRSA F31 MH74342 (C. M. K.).

Reprint requests should be sent to Christina M. Karns, 1227 University of Oregon, Eugene, OR 97403-1227, or via e-mail: ckarns@uoregon.edu.

REFERENCES

REFERENCES
Alho
,
K.
,
Woods
,
D. L.
, &
Algazi
,
A.
(
1994
).
Processing of auditory stimuli during auditory and visual attention as revealed by event-related potentials.
Psychophysiology
,
31
,
469
479
.
Arnfred
,
S. M.
,
Hansen
,
L. K.
,
Parnas
,
J.
, &
Mørup
,
M.
(
2007
).
Proprioceptive evoked gamma oscillations.
Brain Research
,
1147
,
167
174
.
Bauer
,
M.
,
Oostenveld
,
R.
,
Peeters
,
M.
, &
Fries
,
P.
(
2006
).
Tactile spatial attention enhances gamma-band activity in somatosensory cortex and reduces low-frequency activity in parieto-occipital areas.
Journal of Neuroscience
,
26
,
490
501
.
Busch
,
N. A.
,
Schadow
,
J.
,
Fründ
,
I.
, &
Herrmann
,
C. S.
(
2006
).
Time–frequency analysis of target detection reveals an early interface between bottom–up and top–down processes in the gamma-band.
Neuroimage
,
29
,
1106
1116
.
Canolty
,
R. T.
,
Edwards
,
E.
,
Dalal
,
S. S.
,
Soltani
,
M.
,
Nagarajan
,
S. S.
,
Kirsch
,
H. E.
,
et al
(
2006
).
High gamma power is phase-locked to theta oscillations in human neocortex.
Science
,
313
,
1626
1628
.
Clark
,
V. P.
, &
Hillyard
,
S. A.
(
1996
).
Spatial selective attention affects early extrastriate but not striate components of the visual evoked potential.
Journal of Cognitive Neuroscience
,
8
,
387
402
.
Deiber
,
M. P.
,
Ibañez
,
V.
,
Fischer
,
C.
,
Perrin
,
F.
, &
Mauguière
,
F.
(
1988
).
Sequential mapping favours the hypothesis of distinct generators for Na and Pa middle latency auditory evoked potentials.
Electroencephalography and Clinical Neurophysiology
,
71
,
187
197
.
Delorme
,
A.
, &
Makeig
,
S.
(
2004
).
EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis.
Journal of Neuroscience Methods
,
134
,
9
21
.
Demiralp
,
T.
,
Bayraktaroglu
,
Z.
,
Lenz
,
D.
,
Junge
,
S.
,
Busch
,
N. A.
,
Maess
,
B.
,
et al
(
2007
).
Gamma amplitudes are coupled to theta phase in human EEG during visual perception.
International Journal of Psychophysiology
,
64
,
24
30
.
Desmedt
,
J. E.
, &
Robertson
,
D.
(
1977
).
Differential enhancement of early and late components of the cerebral somatosensory evoked potentials during forced-paced cognitive tasks in man.
Journal of Physiology
,
271
,
761
782
.
Di Russo
,
F.
,
Martínez
,
A.
, &
Hillyard
,
S. A.
(
2003
).
Source analysis of event-related cortical activity during visuo-spatial attention.
Cerebral Cortex
,
13
,
486
499
.
Eeckman
,
F. H.
, &
Freeman
,
W. J.
(
1990
).
Correlations between unit firing and EEG in the rat olfactory system.
Brain Research
,
528
,
238
244
.
Eimer
,
M.
(
2001
).
Crossmodal links in spatial attention between vision, audition, and touch: Evidence from event-related brain potentials.
Neuropsychologia
,
39
,
1292
1303
.
Eimer
,
M.
, &
Forster
,
B.
(
2003
).
Modulations of early somatosensory ERP components by transient and sustained spatial attention.
Experimental Brain Research
,
151
,
24
31
.
Eimer
,
M.
, &
Schroger
,
E.
(
1998
).
ERP effects of intermodal attention and cross-modal links in spatial attention.
Psychophysiology
,
35
,
313
327
.
Foxe
,
J. J.
, &
Simpson
,
G. V.
(
2002
).
Flow of activation from V1 to frontal cortex in humans. A framework for defining “early” visual processing.
Experimental Brain Research
,
142
,
139
150
.
Foxe
,
J. J.
, &
Simpson
,
G. V.
(
2005
).
Biasing the brain's attentional set: II. Effects of selective intersensory attentional deployments on subsequent sensory processing.
Experimental Brain Research
,
166
,
393
401
.
Foxe
,
J. J.
,
Simpson
,
G. V.
,
Ahlfors
,
S. P.
, &
Saron
,
C. D.
(
2005
).
Biasing the brain's attentional set: I. Cue driven deployments of intersensory selective attention.
Experimental Brain Research
,
166
,
370
392
.
Freeman
,
W. J.
(
1975
).
Mass action in the nervous system.
New York
:
Academic Press
.
Galambos
,
R.
,
Makeig
,
S.
, &
Talmachoff
,
P. J.
(
1981
).
A 40-Hz auditory potential recorded from the human scalp.
Proceedings of the National Academy of Sciences, U.S.A.
,
78
,
2643
2647
.
García-Larrea
,
L.
,
Lukaszewicz
,
A. C.
, &
Mauguière
,
F.
(
1995
).
Somatosensory responses during selective spatial attention: The N120-to-N140 transition.
Psychophysiology
,
32
,
526
537
.
Hämäläinen
,
H.
,
Kekoni
,
J.
,
Sams
,
M.
,
Reinikainen
,
K.
, &
Näätänen
,
R.
(
1990
).
Human somatosensory evoked potentials to mechanical pulses and vibration: Contributions of SI and SII somatosensory cortices to P50 and P100 components.
Electroencephalography and Clinical Neurophysiology
,
75
,
13
21
.
Hillyard
,
S. A.
, &
Anllo-Vento
,
L.
(
1998
).
Event-related brain potentials in the study of visual selective attention.
Proceedings of the National Academy of Sciences, U.S.A.
,
95
,
781
787
.
Hötting
,
K.
,
Rösler
,
F.
, &
Röder
,
B.
(
2003
).
Crossmodal and intermodal attention modulate event-related brain potentials to tactile and auditory stimuli.
Experimental Brain Research
,
148
,
26
37
.
Kelly
,
S. P.
,
Gomez-Ramirez
,
M.
, &
Foxe
,
J. J.
(
2008
).
Spatial attention modulates initial afferent activity in human primary visual cortex.
Cerebral Cortex.
Epub ahead of print.
Khoe
,
W.
,
Mitchell
,
J. F.
,
Reynolds
,
J. H.
, &
Hillyard
,
S. A.
(
2005
).
Exogenous attentional selection of transparent superimposed surfaces modulates early event-related potentials.
Vision Research
,
45
,
3004
3014
.
Klimesch
,
W.
,
Sauseng
,
P.
,
Hanslmayr
,
S.
,
Gruber
,
W.
, &
Freunberger
,
R.
(
2007
).
Event-related phase reorganization may explain evoked neural dynamics.
Neuroscience and Biobehavioral Reviews
,
31
,
1003
1016
.
Knight
,
R. T.
,
Scabini
,
D.
, &
Woods
,
D. L.
(
1989
).
Prefrontal cortex gating of auditory transmission in humans.
Brain Research
,
504
,
338
342
.
Lakatos
,
P.
,
Shah
,
A. S.
,
Knuth
,
K. H.
,
Ulbert
,
I.
,
Karmos
,
G.
, &
Schroeder
,
C. E.
(
2005
).
An oscillatory hierarchy controlling neuronal excitability and stimulus processing in the auditory cortex.
Journal of Neurophysiology
,
94
,
1904
1911
.
Lakatos
,
P.
,
Szilágyi
,
N.
,
Pincze
,
Z.
,
Rajkai
,
C.
,
Ulbert
,
I.
, &
Karmos
,
G.
(
2004
).
Attention and arousal related modulation of spontaneous gamma-activity in the auditory cortex of the cat.
Brain Research, Cognitive Brain Research
,
19
,
1
9
.
Liégeois-Chauvel
,
C.
,
Musolino
,
A.
,
Badier
,
J. M.
,
Marquis
,
P.
, &
Chauvel
,
P.
(
1994
).
Evoked potentials recorded from the auditory cortex in man: Evaluation and topography of the middle latency components.
Electroencephalography and Clinical Neurophysiology
,
92
,
204
214
.
Lopez
,
L.
, &
Sannita
,
W. G.
(
1997
).
Magnetically recorded oscillatory responses to luminance stimulation in man.
Electroencephalography and Clinical Neurophysiology
,
104
,
91
95
.
Macdonald
,
K. D.
,
Fifkova
,
E.
,
Jones
,
M. S.
, &
Barth
,
D. S.
(
1998
).
Focal stimulation of the thalamic reticular nucleus induces focal gamma waves in cortex.
Journal of Neurophysiology
,
79
,
474
477
.
Makeig
,
S.
,
Debener
,
S.
,
Onton
,
J.
, &
Delorme
,
A.
(
2004
).
Mining event-related brain dynamics.
Trends in Cognitive Sciences
,
8
,
204
210
.
Makeig
,
S.
,
Delorme
,
A.
,
Westerfield
,
M.
,
Jung
,
T. P.
,
Townsend
,
J.
,
Courchesne
,
E.
,
et al
(
2004
).
Electroencephalographic brain dynamics following manually responded visual targets.
PLoS Biology
,
2
,
e176
.
Makeig
,
S.
,
Westerfield
,
M.
,
Jung
,
T. P.
,
Enghoff
,
S.
,
Townsend
,
J.
,
Courchesne
,
E.
,
et al
(
2002
).
Dynamic brain sources of visual evoked responses.
Science
,
295
,
690
694
.
Mangun
,
G. R.
,
Buonocore
,
M. H.
,
Girelli
,
M.
, &
Jha
,
A. P.
(
1998
).
ERP and fMRI measures of visual spatial selective attention.
Human Brain Mapping
,
6
,
383
389
.
Mehta
,
A. D.
,
Ulbert
,
I.
, &
Schroeder
,
C. E.
(
2000a
).
Intermodal selective attention in monkeys: I. Distribution and timing of effects across visual areas.
Cerebral Cortex
,
10
,
343
358
.
Mehta
,
A. D.
,
Ulbert
,
I.
, &
Schroeder
,
C. E.
(
2000b
).
Intermodal selective attention in monkeys: II. Physiological mechanisms of modulation.
Cerebral Cortex
,
10
,
359
370
.
Michie
,
P. T.
,
Bearpark
,
H. M.
,
Crawford
,
J. M.
, &
Glue
,
L. C.
(
1987
).
The effects of spatial selective attention on the somatosensory event-related potential.
Psychophysiology
,
24
,
449
463
.
Min
,
B. K.
,
Busch
,
N. A.
,
Debener
,
S.
,
Kranczioch
,
C.
,
Hanslmayr
,
S.
,
Engel
,
A. K.
,
et al
(
2007
).
The best of both worlds: Phase-reset of human EEG alpha activity and additive power contribute to ERP generation.
International Journal of Psychophysiology
,
65
,
58
68
.
Min
,
B. K.
, &
Herrmann
,
C. S.
(
2007
).
Prestimulus EEG alpha activity reflects prestimulus top–down processing.
Neuroscience Letters
,
422
,
131
135
.
Narici
,
L.
,
Carozzo
,
S.
,
Lopez
,
L.
,
Ogliastro
,
C.
, &
Sannita
,
W. G.
(
2003
).
Phase-locked oscillatory approximately 15- to 30-Hz response to transient visual contrast stimulation: Neuromagnetic evidence for cortical origin in humans.
Neuroimage
,
19
,
950
958
.
Noesselt
,
T.
,
Hillyard
,
S. A.
,
Woldorff
,
M. G.
,
Schoenfeld
,
A.
,
Hagner
,
T.
,
Jäncke
,
L.
,
et al
(
2002
).
Delayed striate cortical activation during spatial attention.
Neuron
,
35
,
575
587
.
Ogden
,
T. E.
(
1973
).
The oscillatory waves of the primate electroretinogram.
Vision Research
,
13
,
1059
1074
.
Pantev
,
C.
,
Makeig
,
S.
,
Hoke
,
M.
,
Galambos
,
R.
,
Hampson
,
S.
, &
Gallen
,
C.
(
1991
).
Human auditory evoked gamma-band magnetic fields.
Proceedings of the National Academy of Sciences, U.S.A.
,
88
,
8996
9000
.
Proverbio
,
A. M.
,
Del Zotto
,
M.
, &
Zani
,
A.
(
2007
).
Inter-individual differences in the polarity of early visual responses and attention effects.
Neuroscience Letters
,
419
,
131
136
.
Sannita
,
W. G.
(
2000
).
Stimulus-specific oscillatory responses of the brain: A time/frequency-related coding process.
Clinical Neurophysiology
,
111
,
565
583
.
Schroeder
,
C. E.
,
Mehta
,
A. D.
, &
Foxe
,
J. J.
(
2001
).
Determinants and mechanisms of attentional modulation of neural processing.
Frontiers in Bioscience
,
6
,
D672
D684
.
Sokolov
,
A.
,
Pavlova
,
M.
,
Lutzenberger
,
W.
, &
Birbaumer
,
N.
(
2004
).
Reciprocal modulation of neuromagnetic induced gamma activity by attention in the human visual and auditory cortex.
Neuroimage
,
22
,
521
529
.
Sukov
,
W.
, &
Barth
,
D. S.
(
2001
).
Cellular mechanisms of thalamically evoked gamma oscillations in auditory cortex.
Journal of Neurophysiology
,
85
,
1235
1245
.
Talsma
,
D.
, &
Kok
,
A.
(
2001
).
Nonspatial intermodal selective attention is mediated by sensory brain areas: Evidence from event-related potentials.
Psychophysiology
,
38
,
736
751
.
Talsma
,
D.
, &
Kok
,
A.
(
2002
).
Intermodal spatial attention differs between vision and audition: An event-related potential analysis.
Psychophysiology
,
39
,
689
706
.
Tiitinen
,
H.
,
Sinkkonen
,
J.
,
Reinikainen
,
K.
,
Alho
,
K.
,
Lavikainen
,
J.
, &
Näätänen
,
R.
(
1993
).
Selective attention enhances the auditory 40-Hz transient response in humans.
Nature
,
364
,
59
60
.
Tournev
,
I.
,
Kalaydjieva
,
L.
,
Youl
,
B.
,
Ishpekova
,
B.
,
Guergueltcheva
,
V.
,
Kamenov
,
O.
,
et al
(
1999
).
Congenital cataracts facial dysmorphism neuropathy syndrome, a novel complex genetic disease in Balkan Gypsies: Clinical and electrophysiological observations.
Annals of Neurology
,
45
,
742
750
.
Whittaker
,
S. G.
, &
Siegfried
,
J. B.
(
1983
).
Origin of wavelets in the visual evoked potential.
Electroencephalography and Clinical Neurophysiology
,
55
,
91
101
.
Woldorff
,
M. G.
,
Gallen
,
C. C.
,
Hampson
,
S. A.
,
Hillyard
,
S. A.
,
Pantev
,
C.
,
Sobel
,
D.
,
et al
(
1993
).
Modulation of early sensory processing in human auditory cortex during auditory selective attention.
Proceedings of the National Academy of Sciences, U.S.A.
,
90
,
8722
8726
.
Woodruff
,
P. W.
,
Benson
,
R. R.
,
Bandettini
,
P. A.
,
Kwong
,
K. K.
,
Howard
,
R. J.
,
Talavage
,
T.
,
et al
(
1996
).
Modulation of auditory and visual cortex by selective attention is modality-dependent.
NeuroReport
,
7
,
1909
1913
.
Woods
,
D. L.
,
Alain
,
C.
,
Covarrubias
,
D.
, &
Zaidel
,
O.
(
1995
).
Middle latency auditory evoked potentials to tones of different frequency.
Hearing Research
,
85
,
69
75
.
Woods
,
D. L.
,
Alho
,
K.
, &
Algazi
,
A.
(
1993
).
Intermodal selective attention: Evidence for processing in tonotopic auditory fields.
Psychophysiology
,
30
,
287
295
.