Abstract

The spatial rule of multisensory integration holds that cross-modal stimuli presented from the same spatial location result in enhanced multisensory integration. The present study investigated whether processing within the somatosensory cortex reflects the strength of cross-modal visuotactile interactions depending on the spatial relationship between visual and tactile stimuli. Visual stimuli were task-irrelevant and were presented simultaneously with touch in peripersonal and extrapersonal space, in the same or opposite hemispace with respect to the tactile stimuli. Participants directed their attention to one of their hands to detect infrequent tactile target stimuli at that hand while ignoring tactile targets at the unattended hand, all tactile nontarget stimuli, and any visual stimuli. Enhancement of ERPs recorded over and close to the somatosensory cortex was present as early as 100 msec after onset of stimuli (i.e., overlapping with the P100 component) when visual stimuli were presented next to the site of tactile stimulation (i.e., perihand space) compared to when these were presented at different locations in peripersonal or extrapersonal space. Therefore, this study provides electrophysiological support for the spatial rule of visual–tactile interaction in human participants. Importantly, these early cross-modal spatial effects occurred regardless of the locus of attention. In addition, and in line with previous research, we found attentional modulations of somatosensory processing only to be present in the time range of the N140 component and for longer latencies with an enhanced negativity for tactile stimuli at attended compared to unattended locations. Taken together, the pattern of the results from this study suggests that visuotactile spatial effects on somatosensory processing occur prior and independent of tactile–spatial attention.

INTRODUCTION

Research in the last two decades has provided evidence that spatial representations of tactile and visual events occurring in the space immediately surrounding the body (i.e., peripersonal space; e.g., Rizzolatti, Fadiga, Fogassi, & Gallese, 1997) are coded by an integrated visuotactile system centered on body parts (i.e., hand-centered) (Bremmer, Schlack, Duhamel, Graf, & Fink, 2001; Duhamel, Golby, & Goldberg, 1991, 1998; Graziano, Tian Hu, & Gross, 1997; Fogassi et al., 1996; Graziano & Gross, 1993, 1995; Rizzolatti, Scandolara, Matelli, & Gentilucci, 1981). Single-cell recordings from several brain structures of the macaque monkey, such as the putamen and some parietal and premotor cortical areas, have revealed the existence of bimodal neurons that respond to both somatosensory and visual inputs. The visual receptive fields of these neurons are located on body parts (e.g., the hand), extending a few centimeters into the surrounding area and are in spatial register with the location of the neurons' tactile receptive fields. Furthermore, the neuronal discharge of these neurons has been found to decrease as the distance between visual stimuli and the body part touched increases, according to the spatial rule of multisensory integration (Stein & Stanford, 2008; Stein & Meredith, 1993).

It has been suggested that a neuronal system representing visuotactile peripersonal space, and perihand space in particular, operates in humans with similar properties as those shown in macaque, such as the constraint of spatial proximity between visual and tactile signals (Macaluso, 2006; Macaluso & Driver, 2005; Làdavas & Farnè, 2004a; Làdavas, 2002; Bremmer, Schlack, Duhamel, et al., 2001; Bremmer, Schlack, Shah, et al., 2001; Làdavas, di Pellegrino, Farnè, & Zeloni, 1998). Studies on brain-damaged people have shown modulatory effects of visual stimuli on tactile perception that are dependent on spatial proximity between visual and tactile inputs (Làdavas & Farnè, 2004b; Làdavas, 2002; Làdavas et al., 1998). These studies have reported that tactile extinction, that is, the decrease of contralesional tactile detection by the simultaneous presentation of an ipsilesional touch (Moscovitch & Behrmann, 1994; Heilman, Bowers, Valenstein, & Watson, 1993), can be significantly reduced if a task-irrelevant visual stimulus is presented concurrently next to the contralesional hand (i.e., perihand space) (Làdavas et al., 1998; di Pellegrino, Làdavas, & Farnè, 1997; Bender, 1952). In contrast, if visual stimuli are presented in a region of space beyond the space immediately surrounding the body (i.e., ∼35 cm), tactile extinction is only weakly reduced (Làdavas et al., 1998).

The neural basis of visuotactile interactions in humans has been studied in recent years using both hemodynamic and electrophysiological measures. For instance, recent fMRI studies have shown that activity in heteromodal (e.g., intraparietal sulcus) as well as in modality-specific (i.e., occipital and somatosensory) brain areas is enhanced for spatially congruent compared to incongruent visuotactile bimodal stimulation (Macaluso, Frith, & Driver, 2000, 2002, 2005). In these studies, visual stimuli were always presented in peripersonal space, either close to the site of tactile stimulation (i.e., hand) or at a distance from it. However, unlike neuropsychological studies, fMRI studies have not compared brain responses to visuotactile bimodal stimulation under near and far space conditions.

To date, two ERP studies have investigated spatial constraints of visuotactile interactions in the human brain using simultaneous presentation of visual and tactile stimuli (Piesco, Molholm, Sehatpour, Ritter, & Foxe, 2005; Schürmann, Kolev, Menzel, & Yordanova, 2002). In these studies, ERPs obtained in response to simultaneous bimodal stimulation were compared with the algebraic sum of ERP responses to unimodal single stimuli (see Stanford & Stein, 2007; Gondan & Röder, 2006; Macaluso & Driver, 2005 for a critical discussion of this method). In one of these studies (Piesco et al., 2005), bimodal interaction effects (i.e., differences between ERP responses to bimodal stimulation compared to the sum of unimodal responses) were found to be present regardless of whether visual and tactile stimuli were presented at the same or different locations in peripersonal space. In the other study (Schürmann et al., 2002), visual stimuli were presented in extrapersonal space either in the same or in the opposite hemispace to tactile stimuli. In this study, bimodal interaction effects were found to occur regardless of the location of the visual stimuli at some electrode sites, whereas at other sites, only bimodal stimuli presented in the same hemispace showed a different pattern of ERP responses compared to the sum of the single inputs. From these studies, it is not clear whether and to what extent spatial congruence plays a role in visuotactile interactions. However, it has been argued that spatial effects in multisensory paradigms are less likely to be present in low complexity tasks (Gondan, Niederhaus, Rosler, & Röder, 2005; Forster, Cavina-Pratesi, Aglioti, & Berlucchi, 2002; Posner, 1978). Likewise, in the previous ERP studies, participants were either performing a simple detection task (Piesco et al., 2005) or were not engaged in any task (Schürmann et al., 2002).

The aim of the present study was to investigate whether ERPs elicited by tactile stimuli coupled with task-irrelevant visual stimuli are modulated by the spatial relationship between tactile and visual stimuli in accordance with the spatial rule of multisensory integration; the rationale being that ERP responses should reflect the extent of visual–tactile interactions under different spatial configurations. In particular, and differently from previous studies on multisensory integration that focused on nonlinear responses (e.g., Piesco et al., 2005; Schürmann et al., 2002), the present study looked specifically at whether the location of task-irrelevant visual stimuli modulates behavioral and electrophysiological responses associated with processing within the somatosensory cortex (i.e., ERPs recorded over and close to the somatosensory cortex). To this aim, the visual stimuli were always presented concurrently with tactile stimuli in peripersonal (“near space”) or extrapersonal space (“far space”), either in the same or opposite hemispace as tactile stimuli. Participants were required to direct their attention to one hand in order to detect tactile target stimuli delivered on that hand. According to previous studies (Macaluso et al., 2005; Làdavas & Farnè, 2004a; Làdavas et al., 1998; Duhamel et al., 1991), we expected ERPs to be modulated by the distance of visual stimuli from the site of tactile stimulation, with enhancement of ERPs under conditions when visual stimuli were presented at the same location as tactile stimuli compared to conditions when the former were presented at a different location in peripersonal or extrapersonal space. In addition, we expected tactile–spatial attention to modulate somatosensory processing with enhanced ERPs to tactile stimuli delivered on the currently attended compared to unattended hand (e.g., Forster & Eimer, 2005; Michie, Bearpark, Crawford, & Glue, 1987; Michie, 1984; Desmedt & Robertson, 1977).

METHODS

Participants

Fifteen paid volunteers took part in the experiment. Three had to be excluded due to an excess of alpha waves. Thus, 12 participants (3 men and 9 women), aged between 23 and 36 years (average age = 26.8 years), remained in the sample. All participants were right-handed and all had normal or corrected-to-normal vision by self-report. The experiment was approved by the Ethics Committee, City University, London, and all participants gave their written informed consent.

Stimuli and Apparatus

Participants sat in a dimly lit sound-attenuated experimental chamber resting their arms on a table in front of them. Two sets of two small boxes (3 × 5 × 3 cm) were each placed equidistant to the left and right of the participant's midline at a distance of 40 cm and 110 cm from the participant's body, respectively (see Figure 1). Each box had one tactile stimulator and one LED (red light-emitting diode) embedded in its surface, the LED being 1.5 cm distant from the tactile stimulator. Participants' index fingers were placed on top of the set of boxes closest to them, covering the tactile stimulators. The distance between participants' index fingers was 40 cm.

Figure 1. 

Schematic representation of the experimental setup showing the sites of tactile stimulation and the four possible locations of visual stimuli in “near space” and “far space,” and in the two hemispaces. On the left side, the actual distance of the visual stimuli from the participant's body is given.

Figure 1. 

Schematic representation of the experimental setup showing the sites of tactile stimulation and the four possible locations of visual stimuli in “near space” and “far space,” and in the two hemispaces. On the left side, the actual distance of the visual stimuli from the participant's body is given.

On each trial, one tactile and one visual stimulus were presented simultaneously. Tactile stimulation was provided using 12-V solenoids driving a metal rod with a blunt conical tip to the top segment of the index finger making contact with the fingers whenever a current was passed through the solenoid. Tactile nontarget stimuli consisted of one rod tip contacting a participant's index finger for 200 msec. Tactile target stimuli were infrequent and had a gap of 4 msec in the continuous contact after a duration of 98 msec. Task-irrelevant visual stimuli were provided by 5 mm LEDs that were illuminated for 200 msec.

Each participant completed two experiments, one “near space” and one “far space” experiment. The two experiments differed in the locations of visual stimuli; in the experiment “near space,” the visual stimuli were always presented close to the participants' hands, that is, on the set of boxes where the fingers were placed; in the experiment “far space,” the visual stimuli were always presented on the set of boxes that were located at a distance of 70 cm from the participants' hands (see Figure 1).

Each participant was instructed to keep his or her gaze on a small white fixation square (0.8 cm2) placed half-way between the two set of boxes at a distance of 75 cm from the participant's body. White noise (50 dB, measured from the position of participant's head) was presented from two loudspeakers placed at 110 cm from the participant's head and equidistant to the right and left of the midline, to mask any sounds made by the tactile stimulators.

Participants responded by pressing a button with either foot. Half of the participants used their left foot and the other half used their right foot to respond to targets. The foot they had to use to give their responses was assigned at the beginning of the experimental session and was kept constant throughout the two experiments.

Procedure

At the start of the experimental session, the participants carried out two pre-experimental blocks of 48 trials each to ensure they could detect the visual stimuli and discriminate the tactile stimuli that they would receive during the experiment. In the first block, only visual stimuli were presented and participants had to respond to all of them. The visual stimuli were presented randomly and with equal probability at one of the four possible locations (Figure 1). In the second block, only tactile stimuli were delivered and participants had to respond to all tactile target stimuli (“gap” stimulation) while ignoring tactile nontarget stimuli (continuous stimulation). Tactile targets were delivered randomly on half of the trials (i.e., 24 trials) with equal probability to the right and the left hand. Participants started the experimental session only when their accuracy in the two pretests was 75% or above. The data of the pre-experimental blocks were not analyzed further. Following the pretest, each participant completed one “near space” and one “far space” experiment. Half of the participants performed the “near space” experiment first followed by the “far space” experiment; for the other half, this order was reversed. The two experiments differed only in the location of the task-irrelevant visual stimuli (see above; see also Figure 1). Each experiment consisted of 12 experimental blocks of 96 trials each. Before the start of each block, participants were instructed to attend either to their right or left hand throughout the block in order to respond to infrequent targets (“gap” stimulation) at the attended hand. For half of the participants, the order of which hand they attended to was right–left–right, and so forth; the other half of participants started with their left hand. In each block, eight valid tactile targets (i.e., tactile target stimuli delivered to the attended hand), which required a foot response, and eight invalid tactile targets (i.e., target stimuli on the unattended hand), which had to be ignored and required no response, were delivered. Targets were presented with equal probability to the right or left hand. The remaining 80 trials were nontarget trials (continuous stimulation) and were randomly presented with equal probability to the right and left hand; these also required no response. Visual stimuli were always presented simultaneously with tactile stimuli, with equal probability either in the same or opposite hemispace with respect to the tactile stimulus. Participants were instructed to ignore all visual stimuli throughout the experiment. The intertrial interval between successive stimuli was randomly set between 1200 and 1600 msec. Participants were instructed to maintain fixation at the fixation point throughout each block and this was monitored throughout the experiment via a camera.

Recording and Data Analysis

Electroencephalogram (EEG) was recorded with Ag–AgCl electrodes and linked-earlobe reference from 28 scalp electrodes (midline electrodes: Fz, FCz, Cz, Pz; electrodes over the right hemisphere: Fp2, F4, F8, FC2, FC6, C4, T8, CP2, CP6, P4, P8, O2 and the homologous electrode sites over the left hemisphere). Horizontal electrooculogram (HEOG) was recorded bipolarly from the outer canthi of both eyes. Electrode impedance was kept below 5 kΩ. EEG and EOG were sampled with a 500-Hz digitization rate. EEG and EOG were epoched off-line into 700-msec periods, starting 100 msec before and ending 600 msec after the onset of tactile stimuli. ERPs for tactile nontarget stimuli were averaged relative to a 100-msec prestimulus baseline. Trials with eye blinks (Fp1 or Fp2 exceeding ±60 μV relative to baseline), horizontal movements (HEOG exceeding ±30 μV relative to baseline, approximately equal to ±2.5° of visual angle; see Mangun & Hillyard, 1991) or other artifacts (a voltage exceeding ±60 μV relative to baseline at electrodes FC6, C4, CP2, CP6, P4, P8, O2 and at homologous electrode sites over the left hemisphere) measured within 600 msec after stimulus onset, were excluded from analysis. ERP analysis was restricted to nontarget trials only and trials immediately following a response were excluded from analysis in order to avoid contamination of averaged ERPs by movement-related artifacts.

As the main aim of this study was to investigate cross-modal spatial effects on processing within the somatosensory cortex, statistical analyses (repeated measures ANOVAs) were conducted for recording sites over and close to somatosensory areas (FC5, FC6, C3, C4, CP1, CP2, CP5, CP6, P3, P4, P7, and P8), where somatosensory ERP components are largest.1 ERP mean amplitudes were computed within successive measurement windows centered on the latencies of early somatosensory ERP components: P100 (80–125 msec after stimuli onset) and N140 (125–175 msec after stimuli onset). Mean amplitudes were also computed for the time interval of 180 and 295 msec poststimuli in order to investigate longer-latency effects.

To investigate cross-modal effects and effects of tactile spatial attention on ERPs, overall statistical analysis (repeated measures ANOVA) included the factors light location (“near space–same,” “near space–opposite,” vs. “far space”—collapsed across “same” and “opposite” trials), attention (attended vs. unattended), electrode site (see above), and hemisphere (contralateral vs. ipsilateral to the stimulated hand). “Far space–same” and “far space–opposite” trials were averaged together in the statistical analyses for simplicity reasons, as no significant main effects or interactions involving the factor light location or attention were obtained in a preanalysis comparing “far space–same” to “far space–opposite” trials (with factors: light location, attention, electrode site, and hemisphere) for any of the analysis time intervals stated above (all F < 2.7; all p > .13).2

To further explore cross-modal effects, pairwise follow-up analyses were carried out to compare ERPs for all combinations of the three light location conditions. To investigate cross-modal effects on response speed to tactile stimuli, repeated measures ANOVA was performed on mean RTs to tactile target stimuli delivered at the attended hand, with the factors light location (“near space–same,” “near space–opposite” vs. “far space”—averaged across “same” and “opposite” trials) and hand (left vs. right hand). For all analyses, Greenhouse–Geisser adjustments to the degrees of freedom were applied when appropriate.

RESULTS

Behavioral Data

Participants' task was to direct their attention to one hand to detect infrequent tactile target stimuli on that hand while ignoring tactile target stimuli at the other hand and any tactile nontarget stimuli. On each trial, task-irrelevant visual stimuli were presented either in the near or far space in the same or opposite hemispace as tactile stimuli (i.e., “near space–same,” “near space–opposite,” “far space–same,” and “far space–opposite” trials). There was no significant difference between the number of missed tactile targets between the conditions “near space–same,” “near space–opposite,” and “far space” (averaged across “same” and “opposite” presentations) (Table 1) and the overall rate of false alarms to nontarget stimuli was below 1%. Participants were faster in responding to tactile targets when task-irrelevant visual stimuli were presented at the same location as tactile stimuli (494.3 msec; “near space–same” trials) than when these were presented at a distance from the stimulated hand either in peripersonal (510.6 msec; “near space–opposite” trials) or extrapersonal space (516.8 msec; “far space” trials) (Table 1). Overall analysis comparing mean RTs to tactile targets on “near space–same” and“–opposite” and “far space” trials showed a main effect of light location [F(1, 11) = 5.51, p < .02]. Pairwise planned comparisons showed that participants responded significantly faster to tactile targets delivered on “near space–same” trials compared to “near space–opposite” trials [F(1, 11) = 8.31, p < .02], and compared to trials in which visual stimuli were presented at a distance from the hands in extrapersonal space (“far space” trials) [F(1, 11) = 7.35, p < .03]. In addition, mean RTs to tactile targets delivered on “near space–opposite” trials did not differ significantly from “far space” trials [F(1, 11) = 0.81, ns].

Table 1. 

Percentage of Missed Responses and Mean RTs to Tactile Target Stimuli under Conditions when Task-irrelevant Visual Stimuli were Presented in Either “Near Space” (Separate for Same and Opposite Hemispace) or “Far Space”


Near Space–Same
Near Space–Opposite
Far Space
Missed responses (%) 2.3 3.8 2.9 
Mean RTs (msec) 494.3 510.6 516.8 

Near Space–Same
Near Space–Opposite
Far Space
Missed responses (%) 2.3 3.8 2.9 
Mean RTs (msec) 494.3 510.6 516.8 

ERP Results

Cross-modal spatial effects and effects of tactile–spatial attention on ERPs recorded over and close to the somatosensory cortex were determined by comparing ERPs obtained for tactile nontarget stimuli at the attended and unattended hand for three possible locations of concurrently presented task-irrelevant visual stimuli; that is, in near space, in the same and the opposite hemispace with respect to the tactile stimuli (i.e., “near space–same” and “near space–opposite” trials), and in far space (i.e., “far space” trials, averaged across “same” and “opposite” trials). Cross-modal effects and effects of tactile–spatial attention on ERPs are shown in separate figures.

Figure 2 shows cross-modal effects on ERPs at electrode sites over and near the somatosensory cortex: Grand-averaged ERPs (collapsed across attended and unattended conditions) are shown separately for “near space–same” trials (solid lines), “near space–opposite” trials (dashed lines), and “far space” trials (dotted lines) at electrodes contralateral and ipsilateral (left and right half of the figure, respectively) to the site of tactile stimulation. For the time range of the P100 component, ERPs for tactile stimuli coupled with spatially coincident visual stimuli (“near space–same” trials) appear to be enhanced as compared to ERPs obtained on “far space” trials, and in addition, ERPs elicited on “near space–same” trials appear also to be enhanced in comparison to ERPs elicited on “near space–opposite” trials, however, this latter difference appears to be only present at electrode sites contralateral to the site of tactile stimulation. For the following N140 component and for later latencies, no such enhancement of ERPs elicited on “near space–same” trials appears to be present.

Figure 2. 

Grand-averaged ERP waveforms (collapsed across attended and unattended conditions) elicited by tactile nontarget stimuli presented with visual stimuli on “near space–same” (solid lines), “–opposite” (dashed lines), and “far space” (averaged across “–same” and “–opposite” trials; dotted lines) trials in the 500-msec interval after stimulus onset. ERPs are displayed separately for electrodes contralateral (left) and ipsilateral (right) to the site of tactile stimulation.

Figure 2. 

Grand-averaged ERP waveforms (collapsed across attended and unattended conditions) elicited by tactile nontarget stimuli presented with visual stimuli on “near space–same” (solid lines), “–opposite” (dashed lines), and “far space” (averaged across “–same” and “–opposite” trials; dotted lines) trials in the 500-msec interval after stimulus onset. ERPs are displayed separately for electrodes contralateral (left) and ipsilateral (right) to the site of tactile stimulation.

Figure 3 shows attentional modulations of ERPs recorded over and near the somatosensory cortex. The figure displays grand-averaged ERPs in response to tactile nontarget stimuli delivered to the currently attended (solid lines) and unattended hand (dashed lines) collapsed across all light location conditions at electrode sites contralateral (left) and ipsilateral (right half of the figure) to the stimulated hand. As can be seen from the figure, sustained tactile spatial attention affected ERP responses with enhanced amplitudes for tactile stimuli at the currently attended compared to the unattended hand in the time range of the N140 component followed by a sustained attentional negativity.

Figure 3. 

Grand-averaged ERP waveforms (collapsed across light location conditions) elicited by tactile nontarget stimuli delivered to the attended (solid lines) and unattended hand (dashed lines) in the 500-msec interval after stimulus onset. ERPs are displayed separately for electrodes contralateral (left) and ipsilateral (right) to the site of tactile stimulation.

Figure 3. 

Grand-averaged ERP waveforms (collapsed across light location conditions) elicited by tactile nontarget stimuli delivered to the attended (solid lines) and unattended hand (dashed lines) in the 500-msec interval after stimulus onset. ERPs are displayed separately for electrodes contralateral (left) and ipsilateral (right) to the site of tactile stimulation.

Statistical analyses confirmed these observations. In the overall analysis comparing ERPs to tactile stimuli under attended and unattended conditions for three spatial locations of visual stimuli, a significant main effect of light location was present in the P100 time range (80–125 msec poststimuli onset) [F(1, 11) = 5.17, p < .03]. Pairwise comparisons were carried out between ERPs for all combinations of the three trial types (see above). For the “near space–same” to “near space–opposite” comparison, a Light location × Hemisphere interaction [F(1, 11) = 13.97, p < .004] was present and follow-up analyses separate for electrodes over the two hemispheres showed a significant main effect of light location only at electrodes contralateral to the site of tactile stimulation [F(1, 11) = 8.23, p < .02]. In addition, a significant main effect of light location was also present for the “near space–same” to “far space” comparison [F(1, 11) = 8.84, p < .02]. Taken together, these comparisons confirm enhancement of P100 amplitudes on trials when tactile and visual stimuli are presented at the same location.2 Furthermore, no significant main effect or interactions involving the factor light location were present in the “near space–opposite” to “far space” comparison. This comparison indicates that ERPs obtained under conditions when visual stimuli are presented at a different location as tactile stimuli either in near or far space are statistically the same. Importantly, no Light location × Attention interactions were obtained in any of the above analyses, and further, additional follow-up analysis comparing ERPs only on “near space–same” trials under attended and unattended conditions revealed no significant main effect or interactions involving the factor attention, suggesting that cross-modal interaction effects on “near space–same” trials occurred regardless of whether or not attention was directed to the site of tactile stimulation. A significant main effect of hemisphere was also present in the P100 time range [F(1, 11) = 40.61, p < .001], with overall enhanced amplitudes over the hemisphere ipsilateral compared to contralateral to the site of tactile stimulation. For the subsequent analysis window of the N140 component (125–175 msec poststimuli), initial overall analysis showed a significant main effect of attention [F(1, 11) = 9.47, p < .02] and an Attention × Hemisphere interaction [F(1, 11) = 18.30, p < .01], confirming the presence of larger N140 amplitudes for tactile stimuli at the currently attended compared to the unattended hand (see Figure 3). In addition, a Light location × Attention interaction [F(1, 11) = 4.26, p < .04] was obtained and follow-up analyses revealed that the main effect of attention only reached statistical significance on “near space–same” and “far space” trials [both F(1, 11) > 7.31, p < .03]. For later latencies (180–295 msec poststimulus), a significant main effect of attention [F(1, 11) = 28.83, p < .0001] was present, reflecting an enhanced negativity for tactile stimuli at attended versus unattended locations (Figure 3). In addition, a significant main effect of hemisphere was present [F(1, 11) = 25.25, p < .001], indicating that, in this time interval, ERP amplitudes were more pronounced over the hemisphere contralateral to the site of tactile stimulation. No other main effects or interactions involving the factors light location or attention were present, indicating that attentional modulations of ERPs occurred irrespective of the spatial location of task-irrelevant visual stimuli.3

DISCUSSION

In the present study, we investigated whether processing within the somatosensory cortex reflects cross-modal interactions between tactile stimuli and task-irrelevant visual stimuli according to the spatial rule of visual–tactile integration, which predicts stronger cross-modal interactions between spatially coincident visual and tactile stimuli. Visual stimuli were presented simultaneously with touch close to the hands in peripersonal space (“near space”) or 70 cm from the hands in extrapersonal space (“far space”), and either in the same or opposite hemispace with respect to the tactile stimuli. Participants had to direct their attention to one of their hands in order to detect infrequent “gap” tactile targets delivered to the attended hand while ignoring tactile targets at the unattended hand, all tactile nontarget stimuli and any visual stimuli.

We found that response speed to tactile target stimuli was modulated by the relative spatial location of task-irrelevant visual stimuli. That is, participants' responses to tactile targets were faster when visual stimuli were presented at the same location as tactile stimuli compared to responses obtained under conditions when visual stimuli were presented at a different location in peripersonal and extrapersonal space. To our knowledge, this is the first time that a near–far cross-modal (visuotactile) modulation has been shown in healthy people. This result is in agreement with previous studies in right-damaged patients that showed that task-irrelevant visual stimuli can enhance tactile perception when they are presented in the area surrounding the body part touched (i.e., perihand space), while these cross-modal effects are reduced or no longer present when visual stimuli are placed outside this area in far space (Làdavas et al., 1998; di Pellegrino et al., 1997).

In line with the behavioral results, early ERPs recorded over and close to the somatosensory cortex were found to be modulated by the spatial relationship between visual and tactile stimuli. In particular, the P100 component was enhanced for ERPs in response to tactile stimuli coupled with spatially congruent visual stimuli compared to ERPs obtained under conditions when visual stimuli were presented at a distance from the site of tactile stimulation in near and far space. Under conditions when visual stimuli were presented in near space, this enhancement was only present for ERP responses contralateral to the site of tactile stimulation, in agreement with previous imaging studies on multisensory interactions (Macaluso et al., 2005; Macaluso & Driver, 2001; Foxe et al., 2000). In addition, and crucially, no reliable difference was present in the P100 time interval between conditions in which visual stimuli were presented at a distance from the site of tactile stimulation in peripersonal and extrapersonal space; although on visual inspection of the graphs (Figure 2), a gradient of cross-modal modulation can be observed between these conditions, with greater ERP amplitudes for the condition when visual stimuli were presented in near space. Taken together, these results show that the spatial relationship between visual and tactile stimuli modulate early ERPs, with enhanced amplitudes for tactile stimuli coupled with visual stimuli delivered near the site of tactile stimulation (i.e., perihand space) compared to ERPs obtained when visual stimuli are presented at a different location in peripersonal or extrapersonal space, as one would predict according to the spatial rule of multisensory integration (see Stein & Stanford, 2008; Stein & Meredith, 1993).

As noted above, these multisensory effects were present at relatively early stages of somatosensory processing. That is, a main effect of light location was only observed in the P100 time interval and was not present at later stages of somatosensory processing. As the somatosensory P100 component is assumed to be generated in the secondary somatosensory cortex (S2, i.e., a modality-specific area; Frot & Mauguière, 1999; Hari et al., 1984), the cross-modal modulation of this component suggests that sensory-specific areas can be modulated by spatially congruent visual–tactile stimulation. This result is in line with recent fMRI studies that have shown that activity in modality-specific brain regions (i.e., parietal operculum, corresponding to S2; and occipital cortex) can be modulated by cross-modal interactions between visual and tactile stimuli at congruent locations (Macaluso et al., 2000, 2002, 2005).

Our finding that the modality-specific P100 component was modulated by visuotactile interactions is compatible, in principle, with hierarchical models of multisensory integration that involve feedback projections from multimodal regions of convergence to unimodal somatosensory areas (Felleman & Van Essen, 1991; Jones & Powell, 1970), as well as with the proposal, based on recent neurophysiological evidence, that direct anatomical connections between sensory-specific brain areas are involved in multisensory integration (Ghazanfar & Schroeder, 2006; Macaluso, 2006; Cappe & Barone, 2005; Schroeder & Foxe, 2005; Falchier, Clavagnier, Barone, & Kennedy, 2002). Although these two models are not mutually exclusive, it has been suggested that unlike cross-modal modulations of very early ERPs (i.e., ∼40–50 msec poststimulus; Giard & Peronnet, 1999 for auditory–visual; and Murray et al., 2004; Foxe et al., 2002 for auditory–tactile) that occur regardless of spatial congruence of bimodal stimuli and that may rely on direct influences between modality-specific areas, later cross-modal spatial effects on modality-specific ERPs (see Eimer, 2004; Kennett, Eimer, Spence, & Driver, 2001; Eimer & Driver, 2000) could be accounted for by top–down modulations from heteromodal cortical regions (see Macaluso & Driver, 2005). These spatially specific cross-modal effects have been mostly reported for cross-modal spatial attention, and these have been shown to arise at around 100 msec after stimulus onset (see Eimer, 2004) similarly to the effects found in this study.

Importantly, however, cross-modal spatial modulations of the somatosensory P100 in the current study were present regardless of whether or not attention was directed to the site of tactile stimulation. That is, no Light location × Attention interaction was obtained in the P100 range, and more specifically, no significant difference was observed between ERPs in response to spatially coincident visual–tactile stimuli under attended and unattended conditions for the P100 time range. In contrast, attentional modulations were present at subsequent stages of processing; that is, attended tactile stimuli elicited an enhanced somatosensory N140 component followed by a sustained negativity compared to unattended tactile inputs, in agreement with previous ERP studies on tactile–spatial attention (e.g., Forster & Eimer, 2005; Garcia-Larrea, Lukaszewicz, & Mauguière, 1995; Michie et al., 1987; Michie, 1984; Desmedt & Robertson, 1977).

Our findings that cross-modal ERP effects occurred under both attended and unattended conditions, and that such cross-modal modulations occurred earlier than attentional modulations may suggest that, at least under the present experimental circumstances, integration of visual–tactile stimuli may take place at a preattentive stage of processing. Likewise, previous behavioral as well as ERP studies looking at multisensory integration in other modalities than touch and vision have also suggested that multisensory integration may occur preattentively (e.g., Fort, Delpuech, Pernier, & Giard, 2002; McDonald, Teder-Salejarvi, & Ward, 2001; Bertelson, Vroomen, De Gelder, & Driver, 2000; Giard & Peronnet, 1999). On the other hand, recent evidence has suggested that multisensory integration and attention may interact. For instance, Talsma and Woldorff (2005) showed that audiovisual integration effects on ERPs (i.e., superadditive responses) were larger in amplitude and occurred earlier when attention was directed to the multisensory combination of visual and auditory stimuli; and in a later study, Talsma, Doty, and Woldorff (2007) showed that superadditive effects depend on both visual and auditory modalities being attended. The heterogeneity of the results from the abovementioned studies suggests that the interplay between cross-modal integration and attention may be flexible and depend on experimental factors.

In the present study, an interaction between light location and attention, although absent for the P100 time interval, was found in the later N140 time range, indicating that attentional modulations (i.e., enhanced amplitudes for attended relative to unattended trials) were present for “near space–same” and “far space” trials but failed to reach significance for “near space–opposite” trials. Two considerations may be drawn from these results. First, we can speculate that in the “near space–opposite” attended trials, the visual stimuli, which are delivered on the (tactile-) unattended side, might act as exogenous cues (see Macaluso, Frith, & Driver, 2001; Macaluso et al., 2000) and draw attention away from the side of tactile stimulation. As a result, no attentional enhancement on ERPs would be present for the “near space–opposite” attended trials, which therefore would not reliably differ from the “near space–opposite” unattended trials. Second, the result that attentional ERP modulations were present when visual stimuli were delivered in far as well as in close space (at least for near space–same trials) indicates that effects of tactile spatial attention are not influenced by whether task-irrelevant visual stimuli were presented near or far from the body.

In summary, the findings from the current study provide ERP evidence in support of the spatial rule of multisensory integration between vision and touch in humans. That is, cross-modal visual–tactile interactions modulate somatosensory processing depending on the spatial relationship between visual and tactile stimuli, with enhancement of ERPs under conditions when visual stimuli are presented at the same location as tactile stimuli compared to conditions when visual stimuli are presented at a different location in peripersonal or extrapersonal space. Importantly, cross-modal spatial effects have been found to occur irrespective of whether or not attention is directed to the site of tactile stimulation, and to precede attentional modulations.

Acknowledgments

This research was funded by City University, London.

Reprint requests should be sent to Chiara F. Sambo, Psychology Department, City University, Northampton Square, London EC1V 0HB, or via e-mail: Chiara.Sambo.1@city.ac.uk.

Notes

1. 

Throughout this article, we refer to ERPs measured over and close to the somatosensory cortex. As expected, ERP waveforms at the recording sites included in the statistical analysis show the typical pattern of somatosensory ERP components (P45, N80, P100, and N140) in response to tactile stimuli. Although, as task-irrelevant visual stimuli were always presented concurrently with tactile stimuli, contributions from visual ERP responses as well as multisensory superadditive and subadditive effects may also be present in the ERP waveforms.

2. 

A preliminary analysis (repeated measures ANOVA) including factors space (near vs. far), hemispace (same vs. opposite), electrode site, and hemisphere (see Methods), revealed a Space × Hemispace × Hemisphere interaction [F(1, 11) = 5.94, p < .04] in the time range of the P100 (80–125 msec after stimuli onset). This result, together with the pairwise comparisons shown above, further demonstrates that cross-modal effects were specific for the “near space–same” trials. No other main effects or interactions involving the factors space and hemispace were present in any other time interval investigated.

3. 

We further tested whether tactile–spatial attention modulates ERPs recorded over the visual cortex (i.e., at O1 and O2 electrodes) for the same time intervals as used in the other analyses. We compared ERPs obtained under conditions when visual stimuli were presented at (tactile-) attended versus unattended locations collapsed across near and far space and congruence conditions. In the time interval between 125 and 175 msec poststimuli, a main effect of attention approached significance (F = 3.82, p = .077), indicating that attending to tactile stimulus locations may result in weak attentional modulations over the visual cortex. No other main effects of attention or hemisphere, or interactions involving these factors were found in any of the analysis time intervals.

REFERENCES

Bender
,
M. B.
(
1952
).
Disorders in perception
,
Springfield, IL
:
Thomas
.
Bertelson
,
P.
,
Vroomen
,
J.
,
De Gelder
,
B.
, &
Driver
,
J.
(
2000
).
The ventriloquist effect does not depend on the direction of deliberate visual attention.
Perception & Psychophysics
,
62
,
321
332
.
Bremmer
,
F.
,
Schlack
,
A.
,
Duhamel
,
J. R.
,
Graf
,
W.
, &
Fink
,
G. R.
(
2001
).
Space coding in primate posterior parietal cortex.
Neuroimage
,
14
,
S46
S51
.
Bremmer
,
F.
,
Schlack
,
A.
,
Shah
,
N. J.
,
Zafiris
,
O.
,
Kubischik
,
M.
,
Hoffmann
,
K. P.
,
et al
(
2001
).
Polymodal motion processing in posterior parietal and premotor cortex: A human fMRI study strongly implies equivalencies between humans and monkeys.
Neuron
,
29
,
287
296
.
Cappe
,
C.
, &
Barone
,
P.
(
2005
).
Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey.
European Journal of Neuroscience
,
22
,
2886
2902
.
Desmedt
,
J. E.
, &
Robertson
,
D.
(
1977
).
Differential enhancements of early and late components of the cerebral somatosensory evoked potentials during forced-pace cognitive tasks in man.
Journal of Physiology
,
271
,
761
782
.
di Pellegrino
,
G.
,
Làdavas
,
E.
, &
Farnè
,
A.
(
1997
).
Seeing where your hands are.
Nature
,
338
,
730
.
Duhamel
,
J. R.
,
Golby
,
C. L.
, &
Goldberg
,
M. E.
(
1991
).
Congruent representation of visual and somatosensory space in single neurons of monkey ventral intraparietal cortex area (area VIP).
In J. Paillard (Ed.),
Brain and space
(pp.
223
236
).
Oxford
:
Oxford University Press
.
Duhamel
,
J. R.
,
Golby
,
C. L.
, &
Goldberg
,
M. E.
(
1998
).
Ventral intraparietal area of the macaque: Congruent visual and somatic response properties.
Journal of Neurophysiology
,
79
,
126
136
.
Eimer
,
M.
(
2004
).
Electrophysiology of human crossmodal spatial attention.
In C. Spence & J. Driver (Eds.),
Crossmodal space and crossmodal attention
(pp.
221
245
).
Oxford
:
Oxford University Press
.
Eimer
,
M.
, &
Driver
,
J.
(
2000
).
An event-related brain potential study of cross-modal links in spatial attention between vision and touch.
Psychophysiology
,
37
,
697
705
.
Falchier
,
A.
,
Clavagnier
,
S.
,
Barone
,
P.
, &
Kennedy
,
H.
(
2002
).
Anatomical evidence of multimodal integration in primate striate cortex.
Journal of Neuroscience
,
22
,
5749
5759
.
Felleman
,
D. J.
, &
Van Essen
,
D. C.
(
1991
).
Distributed hierarchical processing in the primate cerebral cortex.
Cerebral Cortex
,
1
,
1
47
.
Fogassi
,
L.
,
Gallese
,
V.
,
Fadiga
,
L.
,
Luppino
,
G.
,
Matelli
,
M.
, &
Rizzolatti
,
G.
(
1996
).
Coding of peripersonal space in inferior premotor cortex (area F4).
Journal of Neurophysiology
,
76
,
141
157
.
Forster
,
B.
,
Cavina-Pratesi
,
C.
,
Aglioti
,
S. M.
, &
Berlucchi
,
G.
(
2002
).
Redundant target effect and intersensory facilitation from visual–tactile interactions in simple reaction time.
Experimental Brain Research
,
143
,
480
487
.
Forster
,
B.
, &
Eimer
,
M.
(
2005
).
Covert attention in touch: Behavioural and electrophysiological evidence for cost and benefits.
Psychophysiology
,
42
,
171
179
.
Fort
,
A.
,
Delpuech
,
C.
,
Pernier
,
J.
, &
Giard
,
M. H.
(
2002
).
Early auditory–visual interactions in human cortex during nonredundant target identification.
Cognitive Brain Research
,
14
,
20
30
.
Foxe
,
J. J.
,
Morocz
,
I. A.
,
Murray
,
M. M.
,
Higgings
,
B. A.
,
Javitt
,
D. C.
, &
Schroeder
,
C. E.
(
2000
).
Multisensory auditory–somatosensory interactions in early cortical processing revealed by high-density electrical mapping.
Cognitive Brain Research
,
10
,
77
83
.
Foxe
,
J. J.
,
Wylie
,
G. R.
,
Martinez
,
A.
,
Schroeder
,
C. E.
,
Javitt
,
D. C.
,
Guilfoyle
,
D.
,
et al
(
2002
).
Auditory–somatosensory multisensory processing in auditory association cortex: An fMRI study.
Journal of Neurophysiology
,
88
,
540
543
.
Frot
,
M.
, &
Mauguière
,
F.
(
1999
).
Timing and spatial distribution of somatosensory responses recorded in the upper bank of the sylvian fissure (SII area) in humans.
Cerebral Cortex
,
8
,
854
863
.
Garcia-Larrea
,
L.
,
Lukaszewicz
,
A. C.
, &
Mauguière
,
F.
(
1995
).
Somatosensory responses during selective spatial attention: The N120-to-N140 transition.
Psychophysiology
,
32
,
526
537
.
Ghazanfar
,
A. A.
, &
Schroeder
,
C. E.
(
2006
).
Is neocortex essentially multisensory?
Trends in Cognitive Sciences
,
10
,
278
285
.
Giard
,
M. H.
, &
Peronnet
,
F.
(
1999
).
Auditory–visual integration during multimodal object recognition in humans: A behavioural and electrophysiological study.
Journal of Cognitive Neuroscience
,
11
,
473
490
.
Gondan
,
M.
,
Niederhaus
,
B.
,
Rosler
,
F.
, &
Röder
,
B.
(
2005
).
Multisensory processing in the redundant-target effect: A behavioral and event-related potential study.
Perception & Psychophysics
,
67
,
713
726
.
Gondan
,
M.
, &
Röder
,
B.
(
2006
).
New method for detecting interactions between the senses in event-related potentials.
Brain Research
,
16
,
389
397
.
Graziano
,
M. S.
, &
Gross
,
C. G.
(
1993
).
A bimodal map of space: Somatosensory receptive fields in the macaque putamen with corresponding visual receptive fields.
Experimental Brain Research
,
97
,
96
109
.
Graziano
,
M. S.
, &
Gross
,
C. G.
(
1995
).
The representation of extrapersonal space: A possible role for bimodal, visual–tactile neurons.
In M. S. Gazzaniga (Ed.),
The cognitive neurosciences
(pp.
1021
1034
).
Cambridge, MA
:
MIT Press
.
Graziano
,
M. S.
,
Tian Hu
,
X.
, &
Gross
,
C. G.
(
1997
).
Visuospatial properties of ventral premotor cortex.
Journal of Neurophysiology
,
77
,
2268
2292
.
Hari
,
R.
,
Reinikainen
,
K.
,
Kaukoranta
,
E.
,
Hämäläinen
,
M.
,
Ilmoniemi
,
R.
,
Penttinen
,
A.
,
et al
(
1984
).
Somatosensory evoked cerebral magnetic fields from SI and SII in man.
Electroencephalography and Clinical Neurophysiology
,
57
,
254
263
.
Heilman
,
K. M.
,
Bowers
,
D.
,
Valenstein
,
E.
, &
Watson
,
R. T.
(
1993
).
Disorders of visual attention.
Baillieres Clinical Neurology
,
2
,
389
413
.
Jones
,
E. G.
, &
Powell
,
T. P.
(
1970
).
An anatomical study of converging sensory pathways within the cerebral cortex of the monkey.
Brain
,
93
,
793
820
.
Kennett
,
S.
,
Eimer
,
M.
,
Spence
,
C.
, &
Driver
,
J.
(
2001
).
Tactile–visual links in exogenous spatial attention under different postures: Convergent evidence from psychophysics and ERPs.
Journal of Cognitive Neuroscience
,
13
,
462
478
.
Làdavas
,
E.
(
2002
).
Functional and dynamic properties of visual peripersonal space.
Trends in Cognitive Sciences
,
6
,
17
22
.
Làdavas
,
E.
,
di Pellegrino
,
G.
,
Farnè
,
A.
, &
Zeloni
,
G.
(
1998
).
Neuropsychological evidence of an integrated visuotactile representation of peripersonal space in humans.
Journal of Cognitive Neuroscience
,
10
,
581
589
.
Làdavas
,
E.
, &
Farnè
,
A.
(
2004a
).
Neuropsychological evidence of integrated multisensory representation of space in humans.
In G. A. Calvert, C. Spence, & B. E. Stein (Eds.),
The handbook of multisensory processes
(pp.
799
818
).
Cambridge, MA
:
MIT Press
.
Làdavas
,
E.
, &
Farnè
,
A.
(
2004b
).
Visuo-tactile representation of near-the-body space.
Journal of Physiology (Paris)
,
98
,
161
170
.
Macaluso
,
E.
(
2006
).
Multisensory processing in sensory-specific cortical areas.
Neuroscientist
,
12
,
327
338
.
Macaluso
,
E.
, &
Driver
,
J.
(
2001
).
Spatial attention and crossmodal interactions between vision and touch.
Neuropsychologia
,
39
,
1304
1316
.
Macaluso
,
E.
, &
Driver
,
J.
(
2005
).
Multisensory spatial interactions: A window onto functional integration in the human brain.
Trends in Neurosciences
,
28
,
264
271
.
Macaluso
,
E.
,
Frith
,
C. D.
, &
Driver
,
J.
(
2000
).
Modulation of human visual cortex by crossmodal spatial attention.
Science
,
289
,
1206
1208
.
Macaluso
,
E.
,
Frith
,
C. D.
, &
Driver
,
J.
(
2001
).
Multisensory integration and crossmodal attention effects in the human brain.
Science
,
292
,
1791a
.
Macaluso
,
E.
,
Frith
,
C. D.
, &
Driver
,
J.
(
2002
).
Crossmodal spatial influences of touch on extrastriate visual areas take current gaze-direction into account.
Neuron
,
34
,
647
658
.
Macaluso
,
E.
,
Frith
,
C. D.
, &
Driver
,
J.
(
2005
).
Multisensory stimulation with or without saccades: fMRI evidence for crossmodal effects on sensory-specific cortices that reflect multisensory location-congruence rather than task-relevance.
Neuroimage
,
26
,
414
425
.
Mangun
,
G. R.
, &
Hillyard
,
S. A.
(
1991
).
Modulations of sensory-evoked brain potentials indicate changes in perceptual processing during visual–spatial priming.
Journal of Experimental Psychology: Human Perception and Performance
,
17
,
1057
1074
.
McDonald
,
J. J.
,
Teder-Salejarvi
,
W. A.
, &
Ward
,
L. M.
(
2001
).
Multisensory integration and crossmodal attention effects in the human brain.
Science
,
292
,
1791a
.
Michie
,
P. T.
(
1984
).
Selective attention effects on somatosensory event-related potentials.
Annals of the New York Academy of Sciences
,
425
,
250
255
.
Michie
,
P. T.
,
Bearpark
,
H. M.
,
Crawford
,
J. M.
, &
Glue
,
L. C. T.
(
1987
).
The effects of spatial selective attention on the somatosensory event-related potentials.
Psychophysiology
,
24
,
449
463
.
Moscovitch
,
M.
, &
Behrmann
,
M.
(
1994
).
Coding of spatial information in the somatosensory system: Evidence from patients with neglect following parietal lobe damage.
Journal of Cognitive Neuroscience
,
6
,
151
155
.
Murray
,
M. M.
,
Molholm
,
S.
,
Michel
,
C. M.
,
Heslenfeld
,
D. J.
,
Ritter
,
W.
,
Javitt
,
D. C.
,
et al
(
2004
).
Grabbing your ear: Rapid auditory–somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment.
Cerebral Cortex
,
15
,
963
974
.
Piesco
,
J.
,
Molholm
,
S.
,
Sehatpour
,
P.
,
Ritter
,
W.
, &
Foxe
,
J. J.
(
2005
).
Dissociating the multiple phases of somatosensory–visual integrations in the human event-related potential: A high-density electrical mapping study
,
Paper presented at the 6th International Multisensory Research Forum
(
June 5–8
),
Rovereto, Italy
.
Posner
,
M. I.
(
1978
).
Chronometric exploration of mind
,
Hillsdale, NJ
:
Erlbaum
.
Rizzolatti
,
G.
,
Fadiga
,
L.
,
Fogassi
,
L.
, &
Gallese
,
V.
(
1997
).
The space around us.
Science
,
277
,
190
191
.
Rizzolatti
,
G.
,
Scandolara
,
C.
,
Matelli
,
M.
, &
Gentilucci
,
M.
(
1981
).
Afferent properties of periarcuate neurons in macaque monkeys: II. Visual responses.
Behavioral Brain Research
,
2
,
147
163
.
Schroeder
,
C. E.
, &
Foxe
,
J. J.
(
2005
).
Multisensory contribution to low-level, “unisensory” processing.
Current Opinion in Neurobiology
,
15
,
454
458
.
Schürmann
,
M.
,
Kolev
,
V.
,
Menzel
,
K.
, &
Yordanova
,
J.
(
2002
).
Spatial coincidence modulates interaction between visual and somatosensory evoked potentials.
NeuroReport
,
13
,
779
783
.
Stanford
,
T. R.
, &
Stein
,
B. E.
(
2007
).
Superadditivity in multisensory integration: Putting the computation in context.
NeuroReport
,
18
,
787
792
.
Stein
,
B. E.
, &
Meredith
,
M. A.
(
1993
).
Merging of the senses
,
Cambridge
:
MIT Press
.
Stein
,
B. E.
, &
Stanford
,
T. R.
(
2008
).
Multisensory integration: Current issues from the perspective of the single neuron.
Nature Reviews Neuroscience
,
9
,
255
266
.
Talsma
,
D.
,
Doty
,
T. J.
, &
Woldorff
,
M. G.
(
2007
).
Selective attention and audiovisual integration: Is attending to both modalities a prerequisite for early integration?
Cerebral Cortex
,
17
,
679
690
.
Talsma
,
D.
, &
Woldorff
,
M. G.
(
2005
).
Attention and multisensory integration: Multiple phases of effects on the evoked brain activity.
Journal of Cognitive Neuroscience
,
17
,
1098
1114
.