Previous studies have suggested that the putative human homologue of the ventral intraparietal area (hVIP) is crucially involved in the remapping of tactile information into external spatial coordinates and in the realignment of tactile and visual maps. It is unclear, however, whether hVIP is critical for the remapping process during audio-tactile cross-modal spatial interactions. The audio-tactile ventriloquism effect, where the perceived location of a sound is shifted toward the location of a synchronous but spatially disparate tactile stimulus, was used to probe spatial interactions in audio-tactile processing. Eighteen healthy volunteers were asked to report the perceived location of brief auditory stimuli presented from three different locations (left, center, and right). Auditory stimuli were presented either alone (unimodal stimuli) or concurrently to a spatially discrepant tactile stimulus applied to the left or right index finger (bimodal stimuli), with the hands adopting either an uncrossed or a crossed posture. Single pulses of TMS were delivered over the hVIP or a control site (primary somatosensory cortex, SI) 80 msec after trial onset. TMS to the hVIP, compared with the control SI-TMS, interfered with the remapping of touch into external space, suggesting that hVIP is crucially involved in transforming spatial reference frames across audition and touch.
Behavioral and neuroscientific studies have observed cross-modal interactions at the first cortical processing stages (see, e.g., Kayser & Logothetis, 2007; Macaluso, 2006; Shimojo & Shams, 2001). One example of spatial cross-modal interaction is the so-called “ventriloquism effect” (VE), where the perceived location of a sound is shifted toward the location of a synchronous but spatially disparate visual stimulus (Bertelson & Aschersleben, 1998; Howard & Templeton, 1966). Recent ERP and fMRI studies have provided evidence that this effect is associated with a modulation of activity in the planum temporale (i.e., secondary auditory cortex). Because this effect was found between 260 and 400 msec after stimulus onset, it was suggested that auditory cortex may be modulated via feedback projections from multisensory, for example, posterior parietal, brain areas (Bruns & Röder, 2010a; Bonath et al., 2007).
Recent studies have demonstrated a VE with audio-tactile stimuli as well (Bruns & Röder, 2010a, 2010b; Caclin, Soto-Faraco, Kingstone, & Spence, 2002). For instance, Caclin and coworkers (2002) reported that vibrotactile stimuli applied to the two index fingers aligned with the body midline biased the perceived location of concurrently presented lateralized sounds toward a central position. That is, the ability to discriminate the hemispace (left or right) in which the tones were presented was impaired by the concurrent tactile input. This finding suggests that auditory localization is modulated by simultaneously presented tactile input, similar to the classical audio-visual VE (Bertelson & Aschersleben, 1998).
It has been hypothesized that the integration of spatial information across modalities requires a remapping into a common external reference frame (Shore, Spry, & Spence, 2002; Yamamoto & Kitazawa, 2001). Bruns and Röder (2010a, 2010b) have further addressed in which coordinate system the audio-tactile VE takes place. By manipulating body posture (hands uncrossed vs. crossed), they found that auditory localization was biased toward the external location of the concurrent tactile stimuli. Importantly, however, the VE was reduced, though still significant when participants adopted a crossed posture. The reduction of the VE with crossed hands suggests that both anatomical and external coordinates of the tactile input were active at the same time (see also Heed & Röder, 2010), but that the external coordinates predominantly defined the ventriloquist shift. Thus, these findings imply that the audio-tactile VE arises after tactile stimuli were remapped into external coordinates. Both behavioral and ERP studies have provided evidence that this remapping starts as early as 60 msec after stimulus onset (Azañón, Longo, Soto-Faraco, & Haggard, 2010) and both anatomical and external coordinates are available between 100 and 190 msec (Overvliet, Azañón, & Soto-Faraco, 2011; Heed & Röder, 2010).
Primate studies have suggested that the ventral part of the intraparietal sulcus (VIP) is a candidate area for (re)coding stimuli into external coordinates. Neurons in this region respond to visual, vestibular, tactile, and auditory stimulation (Schlack, Sterbing-D'Angelo, Hartung, Hoffmann, & Bremmer, 2005). Importantly, the majority of VIP neurons have spatially congruent receptive fields across modalities (Schlack et al., 2005; Bremmer, Duhamel, Ben Hamed, & Graf, 2002; Bremmer, Klam, Duhamel, Ben Hamed, & Graf, 2002; Schlack, Hoffmann, & Bremmer, 2002; Bremmer et al., 2001; Duhamel, Colby, & Goldberg, 1998; Colby, Duhamel, & Goldberg, 1993). VIP neurons code stimuli either in eye-centered, head-centered, or hybrid coordinates. The latency of the neural response varies with the modality of the stimulus and the frame of reference used. For example, auditory responses have shorter latencies in head-centered coordinates than in eye-centered coordinates (see Schlack et al., 2005). Neural modeling studies have suggested that the characteristics of area VIP, such as the presence of multiple reference frames, would allow a supramodal sensory representation of space and coordinate transformation across different reference frames including those important for motor planning and execution (Pouget, Deneve, & Duhamel, 2002).
TMS studies have provided evidence that the putative homologue of the ventral intraparietal area in humans (hVIP) is indeed crucially involved in the remapping of tactile stimulation into external coordinates (Azañón, Longo, et al., 2010) and in the realignment of tactile and visual maps across hand postures (see, e.g., Bolognini & Maravita, 2007). For instance, Bolognini and Maravita (2007) reported that the probability of perceiving phosphenes elicited by TMS above the visual cortex (which is considered as a measure of visual cortex excitability; see, e.g., Boroojerdi, Prager, Muellbacher, & Cohen, 2000) was increased if a tactile stimulus was delivered to a finger resting in the same spatial position as the expected phosphene (Bolognini & Maravita, 2007, Experiments 1 and 2). Notably, when the hands were crossed over the midline, the external rather than the anatomical location of the tactile stimulation was crucial. Importantly, repetitive TMS (rTMS) over the right hVIP resulted in an anatomically defined tactile modulation of phosphene reports, most likely due to a transient suppression of the tactile remapping process into a visual reference frame (Bolognini & Maravita, 2007, Experiment 3).
It is, however, yet unknown whether the external remapping process supported by hVIP is crucially involved in aligning auditory and tactile maps as well. To answer this question, we targeted the hVIP in the posterior parietal cortex with TMS-induced transient virtual lesions during an audio-tactile ventriloquist experiment while participants adopted either an uncrossed or crossed hand posture. More specifically, by interfering with the remapping of tactile stimuli from anatomical to external coordinates, we expected to bias the localization of sounds toward the anatomical rather than the external location of a tactile stimulus in the crossed posture, that is, when the two frames of reference are in conflict.
Eighteen right-handed young healthy participants, recruited through the University of Hamburg, took part in the study. One participant was excluded due to a too deep anatomical location of the stimulation site (see TMS section). Therefore, 17 participants (8 men; mean age = 25.6 years, SD = 3.6 years, range = 20–34 years) were considered for the data analysis. All participants were naive to the purpose of the experiment. Inclusion criteria for the study included the absence of neurological and psychiatric illness, no report of epileptic seizures or a family history for epilepsy, no treatment with psychotropic medications, and normal perceptual abilities. All participants filled in a TMS safety questionnaire and provided written informed consent to take part in the study. They were monetarily compensated or received course credits for taking part in the study.
The experimental protocol was approved by the Ethics Committee of the Hamburg Medical Association (Nr. PV3704) and was conducted in accordance with the latest TMS guidelines (Rossi, Hallett, Rossini, Pascual-Leone, & Safety of TMS Consensus Group, 2009) and the ethical standards of the Declaration of Helsinki (2008).
Stimuli and Procedure
Auditory stimuli consisted of 10-msec band-pass filtered white noise bursts (2000–4000 Hz; 2.5 msec rise/fall envelopes) generated using Audacity (Version 1.2.6) and presented at 72-dB(A) sound pressure level measured at the participant's head position. Tones were delivered from one of three loudspeaker locations (left: AL; center: AC; right: AR), positioned at −9°, 0° and 9° and at a distance of 60 cm from the participant's head position. The speaker array was covered with a white acoustically transparent curtain to avoid any visual cues about speaker positions.
Participants were seated with their eyes closed in front of a desk with their head located on a chin-rest to minimize head movements. Their hands rested on the desk with the index fingers laying on custom-made vibro-tactile stimulators, located to the left (TL) and to the right (TR) of the speaker array at a distance of ±18.5° from the central speaker, respectively (Bruns & Röder, 2010a, 2010b). The arms adopted either an uncrossed position (with the left index finger on the TL stimulator and the right index finger on the TR stimulator) or a crossed position (left index finger on the TR stimulator and right index finger on the TL stimulator). Hand posture was alternated between blocks and balanced across participants. For the crossed hands blocks, the arm resting on top was alternated between blocks and the order was counterbalanced between participants.
Participants responded with foot pedals (see below) located under the table. The loudspeakers, tactile stimulators, and foot pedals were all controlled via computer parallel ports using the software Presentation (Version 14.5, Neurobehavioral Systems Inc., Albany, CA, USA) and a custom-built relay box.
Constant white noise was delivered at 60-dB(A) sound pressure level from two additional loudspeakers located at the outer boarder of the desk (70 cm to the left and to the right of the participant's body line) to mask any remaining faint noise elicited by the activation of the tactile stimulators or the relay box.
Each participant took part in eight stimulation blocks. They were asked to perform an auditory localization task while ignoring the tactile stimulation.
Auditory and tactile stimuli were presented as single unimodal events or simultaneously as bimodal events in random order, for a total of 135 trials in each block (15 trials per condition in each block). A total of nine different stimulus conditions (TL, TR, AL, AC, AR, ACTL, ACTR, ALTR, and ARTL) were presented.
For the bimodal stimuli, ACTL and ACTR denoted small spatial discrepancies (18.5°) and ALTR and ARTL denoted large discrepancies (27.5°) between the tactile and the auditory stimulus (see Figure 1).
For each unimodal auditory and each bimodal trial, participants were asked to indicate the perceived location of the tone with a categorical response (left, center, or right) by releasing the corresponding button of the foot pedals: left heel for left responses, right heel for right responses, and right or left toes for center responses (balanced across participants). The following trial was initiated by the participant's response with a variable SOA between 1200 and 1800 msec. Participants were instructed to respond spontaneously, but accuracy was stressed over response speed. Unimodal tactile stimuli did not require any response; they were included as catch trials to control for attentional capture effects, that is, to avoid participants responding to the tactile stimulus, the TMS click, or the scalp sensation caused by the TMS pulse.
Participants were familiarized with the task by performing one block of only auditory trials (without TMS) and an additional block comprising all stimulus types (with TMS) before the main experiment.
The whole experiment took approximately 3 hr. Participants were encouraged to take breaks to prevent fatigue.
TMS was delivered with a 70-mm figure-of-eight coil connected to a Magstim Rapid2 stimulator (Magstim Co. Ltd., Whitland, United Kingdom). The resting motor threshold of the right motor cortex was determined for each participant (M = 60.4%, SD = 6.9% of the maximum stimulator output). This intensity was defined as the minimum stimulation that produced motor-evoked potentials of ≥50 μV peak to peak in the first dorsal interosseous muscle of the hand in at least 5 of 10 trials (Rossini et al., 1994). For the experimental task, stimulation intensity was set to 120% of the resting motor threshold (M = 72.5%, SD = 8.1% of the maximum stimulator output). The target areas were localized and adjusted to individual anatomy based on individual high-resolution MRI scans by using the software Brainsight (Rogue Research, Montreal, Canada). On-line “neuro-navigated” TMS was performed. The coil was oriented with an angle of approximately 45° from the nasion-inion line and the handle pointing rightward. For the hVIP, the canonical tangential position was adjusted in the yaw, pitch, and roll angles to allow actual targeting of the depth of the sulcus, as assessed by the projected trajectory of the magnetic field in Brainsight. For the whole duration of the block, the focus of stimulation was maintained within 3.5 mm from the target.
The cerebral target was defined as the fundus of the right intraparietal sulcus (Bremmer et al., 2001). This location is supposed to correspond to hVIP (mean Montreal Neurological Institute [MNI] coordinates ± SD of our participants: x = 27.15 ± 3.63; y = −54.70 ± 3.39; z = 46.85 ± 3.40). We selected this target location based on previously reported fMRI activations accompanying tactile passive stimulation with the arm crossing the midline (Lloyd, Shore, Spence, & Calvert, 2003) and the coordinates used in previous TMS studies on spatial remapping (Azañón, Longo, et al., 2010; Bolognini & Maravita, 2007). In one of the recruited participants (see Participants section), the aimed target site (MNI coordinates: x = 27.46; y = −53.47; z = 37.86) was located deeply in the parietal cortex, with the z coordinate exceeding 2 SD from the group mean. Therefore, the data set of this participant was excluded from the analysis.
The right primary somatosensory cortex (SI), located 2 cm laterally from the nasion-inion line (mean MNI coordinates ± SD: x = 20.07 ± 1.71; y = −29.39 ± 4.73; z = 72.54 ± 5.59), was used as a control site. We preferred this approach compared with, for example, a stimulation of the vertex, to control for the lateralization of possible unspecific effects of TMS (e.g., auditory clicks) on auditory performance, on the basis of previous studies involving auditory stimuli (see Collignon, Davare, Olivier, & De Volder, 2009; Collignon et al., 2008), and on our own pilot testing. The x coordinate of SI was set to avoid stimulating the hand representation of the somatosensory cortex (Lotze et al., 2003; see also Collignon et al., 2008, 2009).
TMS was well tolerated by all participants. Before the actual start of the experiment, 10 single TMS pulses were delivered for each stimulation site: none of the participants exhibited motor-evoked potentials nor reported perceiving tactile sensations on the hands.
Single pulses of TMS were applied 80 msec after the onset of the sensory stimulation. This delay was chosen on the basis of studies suggesting that the transformation of anatomical coordinates into external coordinates might start as early as 60 msec after touch onset and both anatomical and external coordinates seem to be active at least until 190 msec after delivery of the tactile stimulation (Overvliet et al., 2011; Azañón, Longo, et al., 2010; Heed & Röder, 2010; Azañón & Soto-Faraco, 2008).
Trials with RTs higher than 3000 msec were excluded from the analyses (0.47% of the total number of trials).
The rate of false alarms to unimodal tactile stimulation was calculated and submitted to an ANOVA with repeated-measures factors TMS Site (SI, hVIP) and Posture (uncrossed, crossed; see “False alarms” in the Results section).
To determine whether the site of TMS and the adopted posture affected processing speed, mean RT to unimodal auditory and bimodal stimuli were analyzed in an ANOVA with repeated-measures factors TMS Site (SI, hVIP), Posture (uncrossed, crossed), and Type of Stimulus (unimodal auditory, bimodal; see “RTs” in the Results section). Note that unimodal tactile stimuli did not require a response and, thus, are not present in this analysis (see Stimuli and Procedure section).
Localization scoring followed the procedure described in Bruns and Röder (2010a, 2010b): A left response was coded as −1, a central response as 0, and a right response as +1 (called response units in the following). Means were calculated separately for each condition and participant. Note that means could vary from −1 (indicating all left responses) to +1 (indicating all right responses). To assess whether the localization of unimodal auditory stimuli was affected by TMS and/or by the adopted posture, we analyzed the unimodal auditory localization scores with a repeated-measures ANOVA with TMS Site (SI, hVIP), Posture (uncrossed, crossed), and Sound Location (left, center, right) as factors (see “Unimodal localization performance” in the Results section).
The VE (i.e., the spatial shift in localization due to the tactile discrepant stimulus) was calculated by subtracting the localization score (as defined above) of the unimodal auditory condition from the localization score of the corresponding bimodal condition (e.g., ALTR-AL). To allow for a comparison between the two sides of tactile stimulation, the VE was coded as positive when the localization was shifted toward the external location of the tactile stimulus and as negative when the shift was in the opposite direction (i.e., toward the anatomical location in the crossed hands condition).
Previous studies reported that the degree of spatial discrepancy between the tactile and the auditory stimulus (small for ACTL and ACTR trials and large for ARTL and ALTR, see Stimuli and Procedure section) leads to significant differences in the magnitude of the VE and that these differences are influenced by the adopted hand posture as well (Bruns & Röder, 2010a, 2010b). Therefore, Discrepancy (small vs. large) was included as a factor in the analysis of the VE. Moreover, to disentangle possible differences in the modulation of the VE between the two hemispaces, External Side of Tactile Stimulation was added as a factor.
As a consequence, a repeated-measures ANOVA on the VE scores (in response units) was performed with TMS Site (SI, hVIP), Posture (uncrossed, crossed), Discrepancy (small, large), and External Side of Tactile Stimulation (left, right) as factors (see “Localization performance under bimodal stimulation” in the Results section).
When Mauchly's test was significant and sphericity could not be assumed, Greenhouse–Geisser adjustments to the degrees of freedom were applied; corrected degrees of freedom and p values are reported in the Results section.
Higher-order interactions were followed up with sub-ANOVAs (see “Localization Performance under Bimodal Stimulation” in the Results section), guided by our hypotheses.
False alarms to unimodal tactile stimuli were rare (M = 2.2%, SE = 0.98%). A two-way repeated-measures ANOVA with factors TMS Site (SI, hVIP) and Posture (uncrossed, crossed) indicated that false alarm rates did not differ between conditions (all ps > .14).
The ANOVA with TMS Site (SI, hVIP), Posture (uncrossed, crossed), and Type of Stimulus (unimodal auditory, bimodal) revealed a main effect of Type of Stimulus, F(1, 16) = 8.34, p = .011, indicating that the mean RT for the bimodal stimuli (M = 806 msec, SE = 32 msec) was longer than the mean RT for the unimodal auditory stimuli (M = 781 msec, SE = 39 msec). No other significant main effect or interactions were found (all ps > .05).
Unimodal Localization Performance
Unimodal sound localization performance was analyzed with a repeated-measures ANOVA with TMS Site (SI, hVIP), Posture (uncrossed, crossed), and Sound Location (left, center, right) as factors. A significant main effect of Sound Location, F(2, 32) = 691.03, p < .001, indicated that the participants were able to differentiate the three sound sources. The localization scores (see Methods section) for the left, center, and right sound location all differed from one another and were in the expected direction (Left < Center < Right; all ps < .001 with Bonferroni–Holm correction, see Figure 2). A significant Posture × Sound Location interaction (Mauchly's Test of Sphericity: W = .664, χ2(2) = 6.136, p = .047; F(1.5, 23.96) = 5.58, p = .016; see Figure 2) was found as well. Post hoc comparisons revealed that for the right sound location, the performance was significantly shifted centrally (lower absolute value of the mean localization score) in the crossed compared with the uncrossed posture condition, t(16) = 2.46, p = .025. The same pattern emerged for the left sound location but the contrast was only marginally significant, t(16) = 1.88, p = .079. No significant difference between hand postures was found for the central sound location, t(16) < 1, ns.
Localization Performance under Bimodal Stimulation
The analysis on the VE scores (see “Data Analysis” in the Methods section) were consistent with previous studies on the VE (Bruns & Röder, 2010a, 2010b). We obtained a significant main effect of Posture, F(1, 16) = 16.97, p = .001 (uncrossed: M = .153, SE = .027; crossed: M = −.001, SE = .016) and Discrepancy, F(1, 16) = 5.35, p = .034 (small: M = .062, SE = .01; large: M = .090, SE = .015), as well as a significant two-way interaction of Posture × Discrepancy, F(1, 16) = 7.22, p = .016. These results show a significant difference between the VE score between the small and the large discrepancy in the crossed posture condition, t(16) = 4.51, p < .001 (small: M = −.03, SE = .016; large M = .028, SE = .018) but not in the uncrossed posture condition, t(16) < 1, ns (small: M = .154, SE = .027; large M = .152, SE = .03). Moreover, the factor External Side of Tactile Stimulation was found to be significant, reflecting a larger VE score on the left compared with the right external side (left: M = .128 SE = .022; right: M = .024, SE = .02). Importantly, a significant three-way interaction of TMS Site × Posture × External Side of Tactile Stimulation was obtained, indicating that the difference of the VE scores' means between the two TMS sites differed according to the posture adopted and the external side where tactile stimulation was delivered (see “Sub-ANOVAs with factors TMS and posture for each level of External Side of Tactile Stimulation”). This observation was supported by a marginally significant three-way interaction of TMS Site × Posture × Discrepancy, F(1, 16) = 4.41, p = .052, and a marginally significant four-way interaction of TMS Site × Posture × Discrepancy × External Side of Tactile Stimulation, F(1, 16) = 4.44, p = .051. To investigate a possibly unequal contribution of the small and large discrepancy condition to these interactions, separate sub-ANOVAs for the small and the large discrepancy condition were performed (see “Sub-ANOVAs with factors TMS, Posture, and External Side of Tactile Stimulation for each level of discrepancy”).
Sub-ANOVAs with factors TMS and Posture for each level of External Side of Tactile Stimulation
For the right external side of tactile stimulation, the right hand was stimulated in the uncrossed posture, whereas the left hand was stimulated in the crossed posture. The sub-ANOVA for the right external side of tactile stimulation revealed a significant interaction between TMS Site and Posture, F(1, 16) = 5.53 p = .032, indicating that the difference in the size of the VE differed as a function of TMS Site. This difference was larger in the crossed posture condition than in the uncrossed posture condition (see Figure 5).
The same interaction was marginally significant for the left external side of tactile stimulation, F(1, 16) = 3.14, p = .096 (see Figure 6).
For the right external side of tactile stimulation, we tested whether the stimulation of the hVIP was different from that of the control site by running two-tailed paired samples t tests, separately for each posture condition. A significant difference between the two TMS sites was found for the crossed posture (i.e., when the left hand was stimulated in the right hemispace; t(16) = 2.28, p = .037), but not for the uncrossed posture (i.e., when the right hand was stimulated in the right hemispace; t(16) < 1, ns). Moreover, in the crossed posture, the VE was significantly shifted toward the anatomical side of tactile stimulation when the hVIP was targeted with TMS (one-sample t test, t(16) = −2.66, p = .017), whereas the VE was not significantly different from zero when TMS was applied to the control site (SI; t(16) < 1, ns).
Sub-ANOVAs with factors TMS, Posture, and External Side of Tactile Stimulation for each level of discrepancy
To follow up the marginally significant three-way interaction of TMS Site × Posture × Discrepancy and the four-way interaction of TMS Site × Posture × Discrepancy × External Side of Tactile Stimulation in the overall ANOVA, sub-ANOVAs with TMS, Posture, and External Side of Tactile Stimulation as factors were performed separately for the small and the large discrepancy conditions. No significant interactions were obtained for the large discrepancy condition (all ps > .11). In contrast, we found a significant interaction of TMS Site × Posture × External Side of Tactile Stimulation for the small discrepancy condition, F(1, 16) = 7.22, p = .016.
We further performed sub-ANOVAs with factors TMS Site and Posture for each level of the External Side of Tactile Stimulation (left, right) in the small discrepancy condition. These ANOVAs revealed a significant interaction between TMS Site and Posture for the right External Side of Tactile Stimulation only, F(1, 16) = 9.78, p = .007. Two-tailed paired samples t tests showed a significant difference between the two TMS Sites for both Posture conditions (uncrossed: t(16) = 2.40, p = .041; crossed: t(16) = −2.24, p = .029). This means that when a sound was delivered from the central position, concurrently with the stimulation of the right hand located in the right hemispace (i.e., uncrossed hands), the localization of the sound was more shifted toward the right hemispace under hVIP stimulation compared with SI stimulation (see Figure 7). When the left hand was stimulated in the right hemispace (i.e., crossed hands), sound localization was more shifted toward the left external side for hVIP-TMS compared with SI-TMS (see Figure 7). The latter result, thus, indicates a shift toward the anatomical side of the hand receiving the touch when the hVIP was targeted with TMS. For the uncrossed posture, the VE was significantly different from zero for hVIP-TMS, but not when TMS was applied to the SI (hVIP: t(16) = 2.89, p = .01; SI: t(16) = 1.20, p = .25). When the hands were crossed, the VE was significantly shifted toward the anatomical side of tactile stimulation when the hVIP was targeted with TMS, t(16) = −3.30, p = .005, whereas it was not significantly different from zero when TMS was applied to the SI, t(16) < 1, ns. This means that under TMS to the hVIP in the crossed posture condition a significant VE emerged in the direction of the anatomical rather than the external side of tactile stimulation.
In Experiment 1, SI was selected as a control site assuming that, by avoiding the hand representation within the somatosensory cortex (Collignon et al., 2008, 2009; Lotze et al., 2003), TMS to this area would not affect performance. To exclude the possibility that SI stimulation affected either unimodal or bimodal task performance, we ran a second experiment with both sham TMS (obtained by tilting the coil 180°) to the left and right hVIP and no TMS at all.
Ten of the 18 participants, who had taken part in the first experiment, were recruited for Experiment 2 (control experiment). One participant was excluded from data analysis for not reaching the accuracy criterion in the unimodal conditions (overall accuracy for the center location was below 33%, i.e., chance level). Therefore, nine participants were considered for the data analysis (six men, mean age = 28.3 years, SD = 3.8 years, range = 22–35 years). All volunteers filled in a TMS safety questionnaire and provided written informed consent to take part in the study. They were monetarily compensated for their participation. Volunteers were naive to the purpose of the experiment.
The design and the task was the same as in the main experiment (see “Stimuli and Procedure” section of Experiment 1 and Figure 1). Single unimodal (auditory or tactile stimuli) and bimodal events were presented in a random order. There were nine different stimulus conditions (TL, TR, AL, AC, AR, ACTL, ACTR, ALTR, and ARTL) presented with the hands resting in an uncrossed or crossed posture. Each block consisted of 180 trials. In half of the trials no TMS was present, whereas in the other half of the trials sham TMS pulses (the coil was tilted 180° so that the majority of the magnetic field was not directed toward the scalp) were administered 80 msec after trial onset at the same intensity used for each participant in Experiment 1. Trials with sham TMS and with no TMS were randomly intermixed in each block.
By means of off-line neuronavigation, the coil was either positioned over the right or the left hVIP (mean MNI coordinates ± SD: x = ±27.15 ± 3.63; y = −54.70 ± 3.39; z = 46.85 ± 3.40).
Each participant took part in eight stimulation blocks (four for each Coil Position), thus collecting a total of 20 trials for each single condition. Uncrossed and crossed postures were alternated between blocks. Block order was balanced across participants. The whole experiment lasted for approximately 2 hr. Participants were encouraged to take breaks to prevent fatigue.
Trials with RTs higher than 3000 msec were excluded from the analyses (1.33% of the total number of trials). False alarms to unimodal tactile stimuli were rare (M = 0.56%, SE = 0.14%).
To assess whether the localization of unimodal auditory stimuli was affected by the TMS pulse and/or by the lateralization of the TMS pulse on the scalp, we analyzed the unimodal auditory localization scores (see “Data analysis” section of Experiment 1) with a repeated-measures ANOVA with Coil Position (right hemisphere, left hemisphere), TMS Condition (sham TMS, no TMS), Posture (uncrossed, crossed), and Sound Location (left, center, right; see “Unimodal localization performance” in the Results section).
Furthermore, to determine whether and how the VE was affected by the TMS pulse and/or by the lateralization of the TMS pulse on the scalp, a five-way ANOVA with Coil Position (right hemisphere, left hemisphere), TMS Condition (sham TMS, no TMS), Posture (uncrossed, crossed), Discrepancy (small, large), and External Side of Tactile Stimulation (left, right) as factors was conducted on the VE scores (see “Data analysis” section of Experiment 1).
Unimodal Localization Performance
A four-way ANOVA with Coil Position (right hemisphere, left hemisphere), TMS Condition (sham TMS, no TMS), Posture (uncrossed, crossed), and Sound Location (left, center, right) was performed on the unimodal localization scores. Only a significant main effect of Sound Location was found, F(1, 8) = 331.19, p < .001 (Left < Center < Right, all ps < .001 with Bonferroni–Holm correction).
Localization Performance under Bimodal Stimulation
A five-way ANOVA with Coil Position (right hemisphere, left hemisphere), TMS Condition (sham TMS, no TMS), Posture (uncrossed, crossed), Discrepancy (small, large), and External Side of Tactile Stimulation (left, right) as factors was conducted on the VE scores. As in the main experiment, the main effect of Posture (F(1, 8) = 15.58, p = .004) and Discrepancy (F(1, 8) = 17.70, p = .003) were found to be significant, reflecting a larger VE for the uncrossed posture and for the large discrepancy, respectively. A significant interaction of Posture × Discrepancy emerged, F(1, 8) = 10.83, p = .011, as well, revealing a significant difference of the VE score between the small and the large discrepancy during the crossed hands condition, t(8) = 6.01, p < .001, but not during the uncrossed hands condition, t(8) = 1.49, p = .17. A significant interaction was found between TMS Condition and Posture, F(1, 8) = 39.35, p < .001, indicating that in the uncrossed posture the VE scores were larger for the no TMS condition compared with the sham TMS condition, t(8) = 4.72, p = .002, whereas the opposite was true in the crossed hand condition, t(8) = 4.63, p = .002. No other significant interactions were found.
To exclude that the absence of significant interactions, in particular those including Coil Location, TMS Condition, and Posture (as observed in Experiment 1), were caused by a lack of test power due to a smaller sample size in Experiment 2 than in Experiment 1, we additionally performed one-sample t tests for each Coil Location × TMS Condition for the right external side of stimulation (i.e., the condition for which we had observed a significant anatomically defined VE shift due to hVIP-TMS in Experiment 1). None of the four conditions significantly differed from zero in Experiment 2 (all ts(8) < 1, ns). By contrast, when we calculated one sample t tests for the VE scores of Experiment 1 for the same subsample that took part in Experiment 2, we still found that the VE for the right external side of stimulation during the crossed posture was significantly different from zero in the right hVIP-TMS condition, t(8) = −2.55, p = .034. By contrast, this effect was not significant with TMS to the control site SI, t(8) < 1, ns. Therefore, the reduced sample size is unlikely to account for a lack of the relevant effect in Experiment 2.
The aim of this study was to investigate whether the putative hVIP area in humans has an essential role in the remapping of touch into external coordinates during audio-tactile interactions as assessed with the VE (Bruns & Röder, 2010a, 2010b; Caclin et al., 2002). Participants were asked to indicate the location of auditory stimuli (originating from three different locations: left, center, and right), presented alone (unimodal stimuli) or concurrently with a spatially disparate tactile stimulation of one of the index fingers (bimodal stimuli). Previous studies provided evidence that the audio-tactile VE emerges after the remapping of touch into external coordinates (Bruns & Röder, 2010a, 2010b); that is, the auditory localization was biased toward the external location of the concurrent tactile stimulation. We hypothesized that, when adopting a crossed posture (i.e., when the anatomical and the external reference frames were in conflict), TMS to the hVIP would interfere with the spatial remapping process and thus would cause a VE toward the anatomical rather than the external location of the tactile stimulus.
The main result of our study is that we indeed found an anatomically defined VE during the crossed posture when TMS was applied to the hVIP. This finding suggests that hVIP-TMS, as compared with TMS over the control site SI (primary somatosensory cortex), interfered with the remapping of touch into an external reference frame. More specifically, when the left hand was crossed over the body midline, we observed a VE effect toward the left rather than right external side when the right hVIP was targeted with TMS. That is, participants reported more often a left response when a sound from a central location was presented concurrently with a tactile stimulation on the left hand located in the right external space. This effect cannot be due to the lateralization of the unspecific TMS effects (scalp sensation and sound produced by the TMS pulses) over the right hemisphere, because this would most likely have caused opposite effects, that is, a rightward bias of sound localization and hence more right responses. Furthermore, the lateralization of the unspecific TMS effects was controlled for by choosing a right-lateralized control site for TMS stimulation. This hypothesis is supported by the results of the control experiment (Experiment 2). In fact, left and right side hemisphere sham TMS did not yield different results.
Although anatomical coordinates are promptly available due to the somatotopic organization of somatosensory cortex, previous reports have provided evidence that the computation of external coordinates starts at around 60 msec after the application of a tactile stimulus (Azañón, Longo, et al., 2010). In line with this finding, we selected the 80 msec poststimulus time point for TMS application. Results suggest that we hit the remapping process rather than spatial processes in general because auditory localization proper was not affected by TMS. That is, there were no significant main effects of TMS site on the performance for unimodal auditory stimuli in Experiment 1. Accordingly, in Experiment 2, sham TMS did neither interfere with auditory localization nor with tactile remapping and the resulting VE. Thus, hVIP-TMS specifically interfered with the tactile remapping process and in turn with the cross-modal influence of touch on audition, rather than auditory localization per se.
These findings provide further support for the idea that VIP may serve as a multisensory relay for mapping modality-specific spatial coordinates into external coordinates (see Klemen & Chambers, 2011; Azañón, Longo, et al., 2010). It has been hypothesized that different stages of the somatosensory processing pathway use different frames of reference (Azañón, Longo, et al., 2010; Heed, 2010). The SI is predominantly somatotopically organized, whereas body posture seems to be coded by area 5 (Graziano, Cooke, & Taylor, 2000). It is assumed (Azañón, Longo, et al., 2010; Graziano & Cooke, 2006) that the output of these two areas is combined in the VIP and in the hVIP in monkeys and humans, respectively. This area, in turn, seems to output to other areas including secondary somatosensory cortex (SII; Lewis & Van Essen, 2000). Indeed it has been shown that the position of the arm or the hand with respect to the body modulates the BOLD response in SII (Corradi-Dell'Acqua, Tomasino, & Fink, 2009). Moreover, the VIP of monkeys is strongly interconnected with the ventral premotor cortex, in particular with area F4 (Graziano & Gross, 1994; Colby et al., 1993; Andersen, Asanuma, Essick, & Siegel, 1990; Hyvärinen, 1982). The latter comprises bimodal neurons (i.e., responding to visual and to tactile stimulation) or even trimodal neurons (i.e., responding to visual, auditory, and tactile stimulation), which discharge during movements of the head or the arm. These cells mainly encode sensory stimuli in a body-part-centered reference frame (Graziano, Hu, & Gross, 1997a, 1997b; Fogassi et al., 1996; Graziano, Yap, & Gross, 1994). Therefore, recent studies have suggested that spatial remapping may result in a supramodal spatial representation that serves as a read-out module for the motor system (Cohen & Andersen, 2004; Pouget et al., 2002).
Previous studies have demonstrated that unimodal tactile stimuli are automatically remapped into an external reference frame (Heed, Backhaus, & Röder, 2011; Schicke & Röder, 2006; Röder, Rösler, & Spence, 2004; Shore et al., 2002; Yamamoto & Kitazawa, 2001), even when the spatial features of the task are removed (Azañón, Camacho, & Soto-Faraco, 2010). Furthermore, Bolognini and Maravita (2007) found that spatial remapping subserved by hVIP affects the cross-modal excitability of the visual cortex. Our results extend previous findings by demonstrating that the hVIP is involved in cross-modal remapping processes even when vision is not directly involved.
We found that TMS over hVIP modulated the audio-tactile VE. This was demonstrated when the left hand was located in the right external hemispace but not when the right hand was located in the left external hemispace. The latter result might suggest a division of labor between the left and the right hVIP. Bolognini and Maravita (2007) reported that after rTMS over the right hVIP, the likelihood of reporting a phosphene in the left external hemispace was enhanced by tactile stimulation of the left hand, regardless of hand posture. The question arises as to whether the remapping process subtended by the right hVIP is predominantly related to the contralateral hand. The fact that stimulation of the right hVIP has been found to interfere with the remapping of a tactile stimulus on the left hand into external coordinates within the left hemispace (Azañón, Longo, et al., 2010) would suggest that hVIP is involved in coding the position of the left hand into the external coordinates even when the effector does not cross the body midline.
On the other hand, our results revealed an increase in the magnitude of the VE during right hVIP-TMS compared with right SI-TMS (for the small discrepancy condition) when the right hand was stimulated in the right hemispace. This finding would suggest that there might be a more complex interplay between sensory modalities in determining the remapping process, rather than a simple (anatomically) contralateral encoding of the left hand by the right hVIP. For instance, the somatosensory and the visual receptive fields of neurons in the monkey's intraparietal cortex cover either the ipsilateral or the contralateral hand or they comprise both hands (Obayashi, Tanaka, & Iriki, 2000; Iwamura, Iriki, & Tanaka, 1994). Furthermore, Lloyd et al. (2003) reported an activation of the right hVIP both when the right hand was tactually stimulated in the left hemispace and when the left hand was stimulated in the right hemispace (at least when participants had their eyes closed), whereas the activation shifted to the contralateral (left) hVIP when the right hand was stimulated in the left hemispace in the presence of visual feedback. Such pieces of evidence argue against a simple contralateral (with respect to the hand) organization of the hVIP in humans.
Critically, the modulation of the VE by hVIP-TMS in the crossed posture occurred only for the small spatial discrepancy condition (i.e., when the sound arose from the central location and the left hand was stimulated in the right hemispace), but not for the large discrepancy condition (i.e., when the sound arose from the left location and the left hand was stimulated in the right hemispace). Because tactile stimuli seem to be initially coded anatomically (Shore et al., 2002; Yamamoto & Kitazawa, 2001), any interference with the spatial remapping process is expected to result in less external coding. Hence, with hands crossed, the sound and the touch would be perceived as coming from the same hemispace in the large discrepancy condition. This situation is often referred to as “spatial redundancy” (see, e.g., Occelli, Bruns, Zampini, & Röder, 2012). In some cases, spatial redundancy results in an enhancement of localization performance under multisensory stimulation. However, a previous study on the ventriloquism illusion failed to report a spatial redundancy gain (i.e., an enhancement of localization performance when auditory and tactile information are from the same hemispace; Occelli et al., 2012). Accordingly, no VE was found in our study for the large spatial discrepancy. By contrast, in the small discrepancy condition (i.e., when the sounds were presented from the center), interference with spatial remapping results in perceiving the touch on the left hand in the right hemispace as coming from the left hemispace. This pulls the sound toward the left side, that is, an anatomically defined VE is expected. This is exactly the result pattern observed in our present study.
In the present experiment, hVIP-TMS modulated the spatial reference frame in which audio-tactile ventriloquism operates. hVIP might, thus, represent a multisensory relay for mapping sensory stimuli in coordinates allowing for cross-modal integration.
The study was supported by the European Community's Seventh Framework Programme (Grant Agreement 228916) through the NOMS project, by the Werner-Otto-Foundation (to F. H.), and by the SFB 936 C4 (to F. H.).
Reprint requests should be sent to Chiara Renzi, Brain and Behavioral Sciences Department, University of Pavia, Piazza Botta 6, 27100 Pavia, Italy, or via e-mail: firstname.lastname@example.org.