Abstract

Integration of information across sensory modalities is enhanced when stimuli in both modalities are in the same location. This “spatial rule” of multisensory integration has been primarily studied in humans by comparing stimuli located either in the same versus opposite side of the body midline or in peripersonal versus extrapersonal space, both of which involve large, categorical differences in spatial location. Here we used psychophysics and ERPs to investigate visuo-tactile integration in personal space (i.e., on the skin surface). We used the mirror box technique to manipulate the congruence of visual and tactile information about which finger on either the right or left hand had been touched. We observed clear compatibility effects for both visual and tactile judgments of which finger on the left hand had been touched. No such effects, however, were found for judgments about the right hand. ERP data showed a similar pattern. Amplitude of the vertex P200 potential was enhanced and that of the N2 was reduced for congruent visuo-tactile events on the left, but not the right, hand. Similarly, a later positivity over posterior parietal cortices (P300) showed contralateral enhancement for congruent visuo-tactile events on both the left and right hands. These results provide clear evidence for spatial constraints on visuo-tactile integration defined in personal space and also reveal clear lateralization of these effects. Furthermore, these results link these “ultraprecise” spatial constraints to processing in the right posterior parietal cortex.

INTRODUCTION

Although different sensory modalities have often been studied in isolation, most events have coherent effects across multiple senses, which must be combined to produce a single robust percept (cf. Ernst & Bülthoff, 2004). The fundamentally multisensory nature of perception has been increasingly recognized over the past two decades (e.g., Ghazanfar & Schroeder, 2006; Macaluso & Driver, 2005; Stein & Meredith, 1993). Several different neural mechanisms of multisensory integration have been proposed, such as the multisensory enhancement of neural responses. For example, individual neurons in the superior colliculus follow a computational principle of multisensory enhancement, in that the neural response to a multimodal stimulus is often greater than the sum of responses to the unimodal stimuli delivered individually (Meredith & Stein, 1983). A common spatial location for both modalities was required for this enhancement to hold (Meredith & Stein, 1986). Single neurons in monkey parietal cortex appeared to follow the same principle (Avillac, Ben Hamed, & Duhamel, 2007). This spatial rule appears to be a fundamental constraint on multisensory integration (Stein & Meredith, 1993).

Numerous studies in humans have confirmed similar spatial specificity for visuotactile interactions. These studies have generally involved large, categorical differences in spatial location, such as crossing the hands to compare stimuli in the same or opposite hemispace (e.g., Kennett, Eimer, Spence, & Driver, 2001; Eimer & Driver, 2000; Spence, Nicholls, Gillespie, & Driver, 1998). For example, touch applied to the right hand facilitates perception of visual stimuli in the right hemispace and increases associated visual-evoked potentials (Kennett et al., 2001) relative to equivalent touch on the left hand. Stimuli close to or far from the skin follow a similar spatial principle of multisensory interaction (e.g., Sambo & Forster, 2009; Makin, Holmes, & Zohary, 2007; di Pellegrino, Làdavas, & Farnè, 1997). Little research, however, has investigated the precision of this spatial rule for visuo-tactile interactions in humans or whether it operates in personal space (i.e., on the skin surface itself).

Some recent evidence suggests that the spatial rule does operate in personal space and with a high degree of spatial specificity. For example, Kammers, Longo, Tsakiris, Dijkerman, and Haggard (2009) found that a precise spatial match on the body was required for the “rubber hand illusion,” a bodily illusion produced by synchronously stroking a prosthetic rubber hand and to the participant's unseen hand synchronous to touch applied to a prosthetic rubber hand. The illusion was eliminated by a spatial mismatch between vision and touch regarding which finger (index versus little) was being stroked. This result demonstrates that synchronous visuo-tactile stimulation is not sufficient to generate the illusion. A precise spatial match between the skin locations in each modality is also required. Similarly, Papeo, Longo, Feurra, and Haggard (2010) generated conflict between vision and touch using the “mirror box” technique (see Figure 1; Ramachandran & Rogers-Ramachandran, 1996; Ramachandran, Rogers-Ramachandran, & Cobb, 1995). Participants looked into a mirror aligned with their body midline at the reflection of their right hand, which appeared to be a direct view of their left hand. Simultaneous touch was applied to the middle or ring finger of each hand. On some trials, the same finger was touched on each hand (e.g., middle finger on both hands), producing a congruent visuotactile percept; on other trials, however, different fingers were touched (e.g., ring finger on left hand, middle finger on right hand), producing an incongruent percept in which visual information showed one finger being touched whereas tactile information specified another. Vision had clear effects on the ability to localize touch, with significantly slower and less accurate tactile judgments on incongruent than congruent trials. Furthermore, single-pulse TMS applied over the right TPJ reduced the magnitude of this visual capture of tactile localization.

Figure 1. 

Left: Top view of experimental setup. Right: Participant's perspective in the “view right-hand” condition.

Figure 1. 

Left: Top view of experimental setup. Right: Participant's perspective in the “view right-hand” condition.

These findings suggest that the spatial principle governing visuo-tactile interactions operates in personal space and with higher spatial precision than have been revealed by previous studies comparing left versus right hemispace (e.g., Kennett et al., 2001; Eimer & Driver, 2000) or peripersonal versus extrapersonal space (e.g., Sambo & Forster, 2009; di Pellegrino et al., 1997). Here, we investigate the cortical mechanisms of such ultraprecise location-specific multisensory integration in personal space, by comparing ERPs to visuo-tactile stimuli that were either spatially congruent or slightly spatially incongruent.

METHODS

Participants

Fourteen individuals (four women) between the ages of 18 and 30 years participated after giving informed consent. Participants were right-handed, as assessed by the Edinburgh Inventory (M = 87.0, range = 11.1–100) and reported normal tactile sensitivity and normal or corrected-to-normal vision. All procedures were approved by the local ethics committee.

Apparatus and Materials

The apparatus is shown in Figure 1. Participants sat at a table and looked into a mirror aligned with their body midline. Their two hands were placed symmetrically on either side of the mirror. The tip of the middle finger of each hand was positioned 22.5 cm from the mirror, and the tip of the ring fingers, an additional 5.25 cm away. The mirror could be positioned either facing leftward (so that a participant gazing rightward saw what appeared to be their right hand) or rightward (so that a participant gazing leftward saw what appeared to be their left hand).

Touches were delivered by rigid sticks (7.5 cm in length) attached to four servo motors. At the beginning of each trial, the servos were held in a baseline position 1 cm above the middle and ring fingers of each hand. This baseline position was determined for each finger at the beginning of the session. To measure the delay between sending command to the motors and the time of actual touch, we placed metal blocks in the position where participants' fingers would be and recorded the sound generated when the stick hit the block. The peak acoustic signal corresponding to this contact occurred, on average, 58.5 msec (SD = 5.0 msec) after the sending of the motor command (and the EEG trigger). All ERPs are time-locked to the sending of the command to the motors, but the time of touch is also shown on the figures.

Procedure

There were four trial types, formed by the factorial combination of the finger touched on the hand behind the mirror (middle or ring) and on the hand seen in the mirror (middle or ring). Thus, on half the trials, the same finger was touched on both hands (congruent trials), whereas on the others, different fingers were touched (incongruent trials).

There were eight blocks, alternating between looking toward the left and the right hand. The initial looking direction was counterbalanced across participants. Each block consisted of 200 trials, 50 of each type. In each block, 10% of the trials of each type were randomly designated as response trials after which participants were asked to make unspeeded verbal reports of which finger they saw touched in the mirror and which finger they felt touched behind the mirror. These questions were designed to ensure that participants attended both to vision and touch and to investigate multisensory interactions in localization. Participants were not told that they would need to make responses until well after they had been stimulated, forcing them to attend to both tactile and visual stimuli on all trials. Touch was applied for 350 msec. There was a 1000-msec intertrial interval.

After each block, a questionnaire concerning subjective experiences of the mirror box was verbally administered. The items were as follows: (1) “It felt like I was looking directly at my hand, rather than through a mirror.” (2) “Did it seem like the hand you saw was a right hand or a left hand?” For Item 1, participants rated their agreement using a 7-point Likert scale (+3 = strongly agree, −3 = strongly disagree, 0 = neither agree nor disagree), though they could use any intermediate value. Thus, positive values indicated overall agreement, and negative values indicated overall disagreement. Item 2 required a dichotomous response, after which participants indicated the intensity of their feeling using the Likert scale (+3 = a strong sense of seeing a right hand, −3 = a strong sense of seeing a left hand). Thus, positive values indicated an overall sense of seeing a right hand, and negative values indicated an overall sense of seeing a left hand.

EEG Recording

A SynAmp amplifier system and Scan 4.3 software (Neuroscan, El Paso, TX) were used to record EEG data. Recordings were obtained from 32 scalp electrodes, the 21 electrodes of the standard 10–20 system (Fp1, Fpz, Fp2, F7, F3, Fz, F4, F8, T7, C3, Cz, C4, T8, P7, P3, Pz, P4, P8, O1, Oz, O2), plus an additional 11 electrodes centered over the parietal cortex (C5, C6, TP7, CP5, CP3, CPz, CP4, CP6, TP8, P5, P6), placed according to the 10–10 system. Horizontal electroculogram was recorded bipolarly from electrodes placed on the outer canthi of each eye, and vertical electroculogram was recorded from an electrode below the right eye. The reference electrode was AFz, and the ground was on the chin. Electrode impedances were kept below 5 KΩ. EEG signals were amplified and digitized at 1000 Hz.

Data Analysis

EEG data were analyzed with EEGLAB (Delorme & Makeig, 2004). Data were downsampled to 250 Hz, digitally filtered with a bandpass of 0.3–30 Hz, converted to average reference, and segmented into epochs time-locked to the sending of commands to the servo motors (−500 to 1000 msec). The 100 msec before the command was used for baseline correction. Epochs with blinks (voltage at FPz exceeding ±70 μV between −100 and 800 msec) or other artifacts (voltage at any scalp channel exceeding ±100 μM = 13.9% of trials rejected, SD = 10.6%). For two participants with particularly problematic ocular artifacts, blind source separation with independent components analysis (Jung et al., 2000) on epoched data was used to clean data before automated rejection.

There were three main components of interest on the basis of previous multisensory ERP studies. First, the middle latency P200 component, maximal over vertex (see Figures 4 [left] and 5), was calculated as mean amplitude at Cz between 150 and 300 msec. These vertex potentials are elicited by events in any sensory modality and are thought to reflect stimulus saliency (Mouraux & Iannetti, 2009). Second, the P200 was followed by an N2 component, which was calculated as the mean amplitude at midline electrodes (Fz, Cz, CPz, Pz) between 280 and 340 msec. The N2 component has been claimed to reflect conflict monitoring (Yeung, Botvinick, & Cohen, 2004) and has been found to be enhanced in situations of perceptual conflict (Yeung et al., 2004), including visuo-tactile conflict (Forster & Pavone, 2008). Third, a broad positivity (P300), widespread over bilateral parietal channels but maximal over the right hemisphere (see Figure 4, right), was calculated as mean amplitude at all scalp channels between 300 and 500 msec. Although the present paradigm is quite different from those typically used to evoke the target P300, the P300 we find here shows a quite similar structure and scalp topography. As with the vertex components, the target P300 is elicited by sensory events in all modalities (Yamaguchi & Knight, 1991; Courchesne, Hillyard, & Galambos, 1975; Squires, Squires, & Hillyard, 1975; Sutton, Braren, Zubin, & John, 1965), making both these components clear candidates for cortical correlates of multisensory integration.

Figure 2. 

Results from subjective questionnaire data on the mirror illusion. Error bars are 1 SEM.

Figure 2. 

Results from subjective questionnaire data on the mirror illusion. Error bars are 1 SEM.

No clear unimodal visual-evoked potentials of somatosensory-evoked potentials (SEPs) were observed. This is perhaps unsurprising, because this paradigm was not designed to generate clear unimodal components. The requirement that the tactile stimulus generate a clear visual percept of movement necessarily increased variability in the exact timing of touch (our timing test, described above, revealed an SD of 5 msec). We suspect that this variability may have masked early components. Furthermore, both the visual and tactile stimuli were relatively weak, meaning that unimodal components would be expected to be quite small in any case. Nevertheless, given previous reports of visual modulation of SEPs at earlier time windows, such as the P100 (e.g., Sambo & Forster, 2009; Schürmann, Kolev, Menzel, & Yordanova, 2002), we also investigated whether differences were observed over somatosensory cortices during the latencies at which the P100 and N140 SEP components would be expected to occur. Accordingly, we calculated the mean amplitude at central/parietal channels (C3/4, C5/6, T7/8, CP3/4, CP5/6, TP7/8, P3/4, P5/6, P7/8) during the time window of the P100 (80–125 msec posttouch; 138–183 msec posttrigger, given the measured delay) and the N140 (125–175 msec posttouch; 183–233 msec posttrigger).

RESULTS

Illusion Questionnaire

Participants reported significant agreement with the statement that “it felt like I was looking at my hand directly, rather than through a mirror,” both when looking toward their right hand (M = 1.72), t(13) = 8.25, p < .0001, and toward their left hand (M = 1.49), t(13) = 5.24, p < .0005 (Figure 2). When participants looked toward their right hand, they reported that it felt like they were looking at a right hand (M = 1.27), t(13) = 3.93, p < .005, and when looking toward their left hand, they reported that it felt like they were looking at a left hand (M = −1.15), t(13) = −2.93, p < .02. These results confirm that the mirror box setup successfully created the illusion that participants were directly looking at the hand toward which they gazed and were therefore in a position to integrate touches that they saw in the mirror with touches that they felt on the hand behind the mirror.

Figure 3. 

Behavioral results. Congruence effects for both visual and tactile judgments were found when looking toward the left hand, but not the right hand. Error bars are 1 SEM.

Figure 3. 

Behavioral results. Congruence effects for both visual and tactile judgments were found when looking toward the left hand, but not the right hand. Error bars are 1 SEM.

Behavioral Results

Participants were occasionally asked which finger they had seen touched in the mirror and which finger they felt touched behind the mirror (see Methods). Their performance was analyzed to investigate multisensory interactions in localization (Figure 3). An ANOVA including Congruence and Direction of View as factors revealed a significant interaction between these factors, F(1, 13) = 9.78, p < .01. Inspection of Figure 3 reveals clear congruence effects for both visual and tactile judgments when participants looked toward their left hand, but not their right hand. These results demonstrate both highly precise spatial constraints on multisensory integration in personal space (cf. Papeo et al., 2010) but also suggest hemispheric differences in these effects.

Figure 4. 

Grand mean scalp maps showing EEG activations at several time points.

Figure 4. 

Grand mean scalp maps showing EEG activations at several time points.

In this study, we manipulated whether participants perceived they were looking at their right or left hand. In contrast, a previous study that led up to this work (Papeo et al., 2010) only had participants look toward their left hand. Thus, to investigate directly whether we replicated that finding, we first analyzed results from blocks in which participants looked toward their left hand. There was a significant effect of Congruence, F(1, 13) = 6.48, p < .05, with better performance on congruent than incongruent trials (90.0% vs. 80.1% correct). Planned comparisons revealed significant effects of Congruence both for visual judgments (95.1% vs. 90.3%), t(13) = 2.63, p < .02 (one-tailed), and tactile judgments (84.9% vs. 69.8% correct), t(13) = 2.07, p < .05 (one-tailed).

There was also a significant effect of Judgment Type, F(1, 13) = 13.90, p < .005, with better performance on visual than tactile judgments (92.7% vs. 77.4% correct). There was no interaction between Judgment Type and Congruence, F(1, 13) = 2.04, p = ns. These results replicate our recent finding that conflicting visual information can impair tactile localization (Papeo et al., 2010) and also show that the converse is true as well: tactile information can impair visual judgments of which finger was seen to be touched. On blocks in which participants looked toward their right hand, in contrast, the results were quite different. As before, there was a significant main effect of Judgment Type, F(1, 13) = 12.30, p < .005, with visual judgments being more accurate than tactile judgments (95.0% vs. 83.4% correct). In contrast, however, there was no effect of Congruence, F(1, 13) = 0.55, p = ns, nor were there significant effects for either visual judgments (94.8% vs. 95.1% correct), t(13) = −0.54, p = ns, or tactile judgments (85.9% vs. 80.9% correct), t(13) = 0.81, p = ns. We return to this difference between the hands in discussion.

ERPs

Somatosensory P100 and N140 Time Windows

Grand mean ERPs are shown in Figure 5. ANOVA on mean amplitude within the P100 latency range revealed no significant effect of Congruence, F(1, 13) = 1.33, p > .20, nor any interactions involving Congruence. Analysis of mean amplitude within the N140 latency range similarly yielded no significant effect of Congruence, F(1, 13) = 2.89, p > .10, nor any interactions involving Congruence.

Figure 5. 

Grand mean ERPs at several electrodes. Time 0 indicates when commands were sent to the motors, whereas the dotted vertical lines indicate the estimated time of actual tactile contact of the stimuli with the participant's fingers.

Figure 5. 

Grand mean ERPs at several electrodes. Time 0 indicates when commands were sent to the motors, whereas the dotted vertical lines indicate the estimated time of actual tactile contact of the stimuli with the participant's fingers.

Vertex P200s

ANOVA on mean amplitude revealed a significant interaction between Congruence and Direction of Gaze, F(1, 13) = 5.77, p < .05. In parallel with the behavioral results, there was a significant congruence effect when participants looked toward their left hand (2.35 vs. 2.03 μV), t(13) = 3.59, p < .005, with vision and touch on congruent fingers producing larger amplitudes than vision and touch on different fingers. However, there was no congruence effect when participants looked toward their right hand (2.20 vs. 2.18 μV), t(13) = 0.16.

Vertex N2s

N2 amplitude showed a pattern comparable across conditions to P200s, with ANOVA revealing a significant main effect of Congruence, F(1, 13) = 12.41, p < .005, amplitude being more negative on incongruent trials (0.84 vs. 0.56 μV). This effect was mediated by an interaction of Congruence and Electrode, F(1, 13) = 4.42, p < .01. The magnitude of the congruence effect decreased monotonically from anterior to posterior: congruent–incongruent amplitude = 0.48, 0.37, 0.19, and 0.05 μV at Fz, Cz, CPz, and Pz, respectively). There was additionally a near-significant interaction of Congruence and Direction of Gaze, F(1, 13) = 4.00, p = .067, revealing an overall significant congruent effect when looking toward the left hand (0.85 vs. 0.47 μV), t(13) = 4.49, p < .001, but not when looking toward the right hand (0.82 vs. 0.65 μV), t(13) = 1.68. Thus, for both P200s and N2s, increased negativity was found on incongruent compared with congruent trials, consistent with results from other paradigms (e.g., Forster & Pavone, 2008; Yeung et al., 2004).

Parietal P300s

To investigate the overall topography of this component, we first conducted an ANOVA on mean amplitude grouping electrodes into four regions defined as Anterior (F3/4, F7/8, Fp1/2) and Posterior (C3/4, CP3/4, P3/4, C5/6, CP5/6, P5/6, T7/8, TP7/8, P7/8) channels and by Left versus Right Hemisphere. Direction of View was also included as a factor. There was a significant main effect of Frontal versus Central/Parietal, F(1, 13) = 60.04, p < .0001, with overall positive amplitudes found over parietal channels (1.05 μV), t(13) = 8.20, p < .0001, and overall negative amplitudes over frontal channels (−1.98 μV), t(13) = −6.92, p < .0001. There was also a main effect of Hemisphere, F(1, 13) = 12.77, p < .005, which was modulated by a significant interaction of Hemisphere and Anterior/Posterior, F(1, 13) = 33.26, p < .0001. This interaction revealed a significant right hemisphere lateralization over central/parietal channels (1.70 vs. 0.40 μV), t(13) = 4.97, p < .0005, but not over frontal channels (−1.84 vs. −2.21 μV), t(13) = 1.33. Lastly, there was a significant three-way interaction, F(1, 13) = 17.89, p < .002, which arose because the right lateralization of amplitude over central/parietal channels was significantly reduced when participants looked toward their left hand (1.46 vs. 0.56 μV) compared with when they looked at their right hand (1.80 vs. 0.34 μV), t(13) = 2.97, p < .02.

Having identified a clear posterior topography of the P300, we conducted a further ANOVA on just the posterior channels, including Congruence as a factor. There was a significant effect of Congruence, F(1, 13) = 12.26, p < .005, with increased amplitude on congruent compared with incongruent trials (1.14 vs. 0.97 μV). There was also a near-significant three-way interaction of Congruence, Gaze Direction, and Hemisphere, F(1, 13) = 4.54, p = .053 (see Figure 6). Congruence effects were found contralateral to the hand the participant was looking at: When looking at the right hand, there was a significant congruence effect in the left hemisphere (0.48 vs. 0.12 μV), t(13) = 2.45, p < .03, but not in the right hemisphere (1.82 vs. 1.86 μV), t(13) = −.36; conversely, when looking at the left hand, there was a significant congruence effect in the right hemisphere (1.74 vs. 1.40 μV), t(13) = 2.98, p < .02, but not in the left hemisphere (0.51 vs. 0.50 μV), t(13) = 0.12. This interaction is shown in Figure 6.

Figure 6. 

Mean peak amplitudes of the P300 component. Clear increases in amplitude were observed for congruent compared with incongruent trials, but only in the hemisphere contralateral to the seen hand.

Figure 6. 

Mean peak amplitudes of the P300 component. Clear increases in amplitude were observed for congruent compared with incongruent trials, but only in the hemisphere contralateral to the seen hand.

DISCUSSION

These results demonstrate a novel form of “ultraprecise” spatial constraint on visuo-tactile integration operating in personal space (i.e., on the skin surface). Congruent visual and tactile information about which finger on the left hand was touched influenced judgments about both visual and tactile locations. These results replicate the findings of Papeo and colleagues (2010) for tactile localization judgments and further show similar effects for visual localization. This suggests that finger identification does not involve just a simple dominance of vision over touch (cf. Rock & Victor, 1964) but also a genuine multisensory interaction. Our results also reveal clear neural correlates of this ultraprecise integration. The behavioral findings were mirrored by ERPs, which showed enhancement of the vertex P200 and N2 components and a right parietal P300 component for congruent visuo-tactile information about touch on the left hand, but not the right hand. Previous studies have provided evidence of the importance for multisensory interactions of spatial proximity in peripersonal and extrapersonal space. Along with other recent findings (Papeo et al., 2010; Kammers et al., 2009), the present results provide evidence for highly precise spatial matching of multisensory signals operating in personal space (i.e., on the skin surface).

The ERP data showed an enhancement of late ERPs when visual and tactile information matched, indicating touch on the same finger, compared with conditions where vision and touch were associated with different fingers. This enhancement may be considered a correlate of multisensory integration. Many previous ERP studies have shown effects of the state of one sensory modality on unimodal processing in another (Cardini, Longo, & Haggard, 2011; Longo, Pernigo, & Haggard, 2011; Longo, Betti, Aglioti, & Haggard, 2009; Kennett et al., 2001). Although these studies can show multisensory effects, they cannot reveal the brain basis of multisensory integration per se, because the events analyzed are unimodal. In our study, by contrast, the probe events were both visual and tactile. As a result, it is not possible to separate the components of the ERP because of touch from those because of vision, as in classic unisensory studies. However, our approach has the concomitant advantage of allowing a direct measure of the multisensory neural response itself. That is, we were able to compare ERPs for visual-tactile events in conditions that involved spatially precise multisensory congruence from those in conditions that were less congruent. Behavioral results showed that this congruence contributed to the multisensory integration of visual and tactile information. We could thus identify both early and late ERP components linked to stronger multisensory integration, and we further showed that these integration-related components were maximal over the right parietal cortex. These components could, at least in principle, correspond to the key processing stage of integrating information across two distinct modalities to provide a single overall percept.

Could our results reflect intrasensory congruence between the two tactile stimuli, rather than intersensory congruence between vision and touch? We believe two pieces of evidence clearly point toward effects of visuo-tactile integration, rather than intrasensory tactile interference. First, we explicitly tested this possibility in a previous study using the same paradigm (Papeo et al., 2010). We found no effect of tactile–tactile conflict on either accuracy or RT in judging which finger on a particular hand had been touched, suggesting that the drop in performance on incongruent trials in this study is because of conflicting visual rather than tactile information. Second, the lateralized pattern of results in this study are also inconsistent with a hypothesis of tactile–tactile interference. If congruence effects were simply driven by tactile conflict between the two hands, one might expect them to be equal whether gazing at the left or at the right hand. However, we found clear differences in congruence effects as a function of gaze direction both in behavior and in ERPs, making tactile–tactile interference between the two hands implausible.

Previous studies suggest that the earliest cortical interactions between sensory modalities (∼50 msec poststimulus) may be insensitive to the spatial position of multisensory stimuli (Azañón & Soto-Faraco, 2008; Murray et al., 2005). Multisensory interactions appear to become spatially sensitive only 100–200 msec poststimulus, both for effects of hemispace (Gondan, Niederhaus, Rösler, & Röder, 2005; Teder-Sälejärvi, Di Russo, McDonald, & Hillyard, 2005; Eimer & Driver, 2000) and of location in peripersonal space (Sambo & Forster, 2009; Simon-Dack et al., 2009). In contrast, the present results, showing extremely precise spatial congruence effects in personal space, show maximal differences much later. Although some studies have reported visual modulation of the P100 SEP (Sambo & Forster, 2009; Schürmann et al., 2002), we did not find any differences over somatosensory cortices in this time window nor during the time window of the later N140 component. The reasons for this difference are not entirely clear, but may reflect the finer spatial matching involved in our congruence effects, in comparison with other studies.

The multisensory-related P300 potential appeared to have a parietal focus. It is possible, however, that the positive P300 we observed over posterior parietal channels could result from a central dipole. Nevertheless, a posterior parietal localization would be consistent with extensive primate data, because many cells in the parietal cortex respond both to visual and tactile inputs (Graziano, Cooke, & Taylor, 2000; Duhamel, Colby, & Goldberg, 1998; Iriki, Tanaka, & Iwamura, 1996). These cells generally have spatially congruent visual and tactile receptive fields, reiterating the general principle of multisensory spatial congruence previously found in the superior colliculus. One recent primate study is particularly relevant (Avillac et al., 2007). These authors recorded from neurons in area VIP that showed both visual and tactile responses. When the arm moved, the visual receptive field of these cells followed the movement of the tactile receptive field through space. Most importantly, some of these cells showed a spatially selective supra-additive increase in response rate when a multisensory stimulus was present, relative to the sum of the unimodal responses. For example, a neuron responding to touch on the face would show enhanced responses when a visual stimulus was simultaneously present in the region of peripersonal space close to the tactile receptive field. That is, they showed multisensory enhancement, which was specific to a precise spatial location of stimulation. Our ERP findings resemble this pattern of neural tuning.

Whereas the present EEG data do not allow precise spatial localization of the effects in the brain, several pieces of evidence suggest that the present results may be driven by processing in the TPJ. First, the TPJ is a site of multisensory convergence, showing strong visual, auditory, and somatosensory responses (Matsuhashi et al., 2004), especially to novel stimuli (Downar, Crawley, Mikulis, & Davis, 2000), as well as multisensory processing related to one's own body (e.g., Papeo et al., 2010; Tsakiris, Costantini, & Haggard, 2008; Blanke, Landis, Spinelli, & Seeck, 2004; Leube et al., 2003). Second, Papeo and colleagues (2010) found that single-pulse TMS delivered to the right TPJ 350 msec following touch reduced visuo-tactile interference in the same behavioral paradigm as this study. The location of TMS, which disrupted visuo-tactile integration in that study, is consistent with the peak of the P300 component in this study, which moreover mirrored the behavioral results. Third, Yamaguchi and Knight (1991) found that lesions of the TPJ resulted in reductions of both the vertex P200 and parietal P300 produced by tactile stimuli. This pattern of deficits following TPJ damage mirrors the pattern of enhancements we found in the presence of congruent visuo-tactile information about touch on the left hand.

Could our results be because of multisensory conflict (i.e., a relative negativity on incongruent trials) rather than multisensory integration (i.e., a relative positivity found on congruent trials)? In the absence of a neutral condition, we cannot formally distinguish between a benefit of congruence and a cost of incongruence, but we can compare our results to other relevant published data. In our previous study using this paradigm (Papeo et al., 2010), however, we found that TMS applied to the right TPJ led to a significant reduction of the behavioral congruence effect. That result suggests that the right TPJ is not involved in the resolution of intersensory conflict, because in that case, disrupting it should have prevented conflict resolution, thus increasing the difference in performance between incongruent and congruent trials. Thus, we consider it more probable that the present results reflect successful integration of congruent multisensory stimuli than conflict between incongruent stimuli. On the other hand, several studies found increased negativity of vertex N2 components related to perceptual conflict (e.g., Forster & Pavone, 2008; Yeung et al., 2004). Some authors have interpreted these results as being related to the so-called error-related negativity and suggested that they reflect response conflict (Yeung et al., 2004). Forster and Pavone (2008) found enhanced negativity when visual stimuli near the hand were spatially incongruent with touch, starting in the time window of the N2. The present results are consistent with those findings also, although in this study, effects were found in the time window of the earlier P200 as well.

Left–Right Differences

Clear interactions were obtained between visual and tactile information regarding which finger on the left hand was touched, as in the study of Papeo and colleagues (2010). Intriguingly, no such facilitation was observed in the presence of congruent visuo-tactile information about which finger on the right hand was touched. Two aspects of our ERP results may be relevant to this laterality effect. First, whereas congruence effects were observed in the time window of the P300 regardless of gaze direction, effects in the time windows of the P100 and N2 were found only when participants looked toward their left hand. Second, congruence effects in the time window of the P300 were only found in the hemisphere contralateral to the hand participants looked toward. Thus, the perceptual laterality effect may relate either to differences in the timing or laterality of neural activations, depending on which hand is viewed. Although the present results to not disambiguate between these interpretations, we consider the second more likely. This pattern suggests a link between ultraprecise spatial precision in personal space and processing in the right posterior parietal cortex (PPC). This right parietal focus is consistent with a large body of evidence linking spatial processing of somatic stimuli to the right PPC. For example, right PPC lesions have been implicated in conditions such as neglect of the left side of the body (Guariglia & Antonucci, 1992; Bisiach, Perani, Vallar, & Berti, 1986; Critchley, 1953) and even the feeling that that part of the body no longer exists (asomatagnosia; Critchley, 1953). Neglect of the right side of the body after left hemisphere lesions is less common (Beis et al., 2004). Similarly, the right TPJ has been frequently implicated in own-body perspective-taking (e.g., Arzy, Thut, Mohr, Michel, & Blanke, 2006; Blanke et al., 2005; Zacks, Rypma, Gabriel, Tversky, & Glover, 1999) and bodily illusions such as out-of-body experiences (e.g., Blanke et al., 2004; Blanke, Ortigue, Landis, & Seeck, 2002). Interestingly, our ERP data suggest that the unique contribution of the right hemisphere may be for the earlier computations of spatial congruence, responsible for P200 and N2 effects.

Acknowledgments

This research was supported by Biotechnology and Biological Sciences Research Council (BBSRC) grant BB/D009529/1 to P. H.

Reprint requests should be sent to Matthew R. Longo, Department of Psychological Sciences, Birkbeck, University of London, Malet Street, London WC1E 7HX, United Kingdom, or via e-mail: m.longo@bbk.ac.uk.

REFERENCES

REFERENCES
Arzy
,
S.
,
Thut
,
G.
,
Mohr
,
C.
,
Michel
,
C. M.
, &
Blanke
,
O.
(
2006
).
Neural basis of embodiment: Distinct contributions of temporoparietal junction and extrastriate body area.
Journal of Neuroscience
,
26
,
8074
8081
.
Avillac
,
M.
,
Ben Hamed
,
S.
, &
Duhamel
,
J. R.
(
2007
).
Multisensory integration in the ventral intraparietal area of the macaque monkey.
Journal of Neuroscience
,
27
,
1922
1932
.
Azañón
,
E.
, &
Soto-Faraco
,
S.
(
2008
).
Changing reference frames during the encoding of tactile events.
Current Biology
,
18
,
1044
1049
.
Beis
,
J. M.
,
Keller
,
C.
,
Morin
,
N.
,
Bartolomeo
,
P.
,
Bernati
,
T.
,
Chokron
,
S.
,
et al
(
2004
).
Right spatial neglect after left hemisphere stroke: Qualitative and quantitative study.
Neurology
,
63
,
1600
1605
.
Bisiach
,
E.
,
Perani
,
D.
,
Vallar
,
G.
, &
Berti
,
A.
(
1986
).
Unilateral neglect: Personal and extrapersonal.
Neuropsychologia
,
24
,
759
767
.
Blanke
,
O.
,
Landis
,
T.
,
Spinelli
,
L.
, &
Seeck
,
M.
(
2004
).
Out-of-body experience and autoscopy of neurological origin.
Brain
,
127
,
243
258
.
Blanke
,
O.
,
Mohr
,
C.
,
Michel
,
C. M.
,
Pascual-Leone
,
A.
,
Brugger
,
P.
,
Seeck
,
M.
,
et al
(
2005
).
Linking out-of-body experience and self processing to mental own-body imagery at the temporoparietal junction.
Journal of Neuroscience
,
25
,
550
557
.
Blanke
,
O.
,
Ortigue
,
S.
,
Landis
,
T.
, &
Seeck
,
M.
(
2002
).
Stimulating illusory own-body perceptions.
Nature
,
419
,
269
270
.
Cardini
,
F.
,
Longo
,
M. R.
, &
Haggard
,
P.
(
2011
).
Vision of the body modulates somatosensory intracortical inhibition.
Cerebral Cortex
,
21
,
325
330
.
Courchesne
,
E.
,
Hillyard
,
S. A.
, &
Galambos
,
R.
(
1975
).
Stimulus novelty, task relevance, and the visual evoked potential in man.
Electroencephalography and Clinical Neurophysiology
,
39
,
131
143
.
Critchley
,
M.
(
1953
).
The parietal lobes.
London
:
Edward Arnold & Co
.
Delorme
,
A.
, &
Makeig
,
S.
(
2004
).
EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis.
Journal of Neuroscience Methods
,
134
,
9
21
.
di Pellegrino
,
G.
,
Làdavas
,
E.
, &
Farnè
,
A.
(
1997
).
Seeing where your hands are.
Nature
,
388
,
730
.
Downar
,
J.
,
Crawley
,
A. P.
,
Mikulis
,
D. J.
, &
Davis
,
K. D.
(
2000
).
A multimodal cortical network for the detection of changes in the sensory environment.
Nature Neuroscience
,
3
,
277
283
.
Duhamel
,
J.-R.
,
Colby
,
C. L.
, &
Goldberg
,
M. E.
(
1998
).
Ventral intraparietal area of the macaque: Congruent visual and somatic response properties.
Journal of Neurophysiology
,
79
,
126
136
.
Eimer
,
M.
, &
Driver
,
J.
(
2000
).
An event-related brain potential study of cross-modal links in spatial attention between vision and touch.
Psychophysiology
,
37
,
697
705
.
Ernst
,
M. O.
, &
Bülthoff
,
H. H.
(
2004
).
Merging the senses into a robust percept.
Trends in Cognitive Sciences
,
8
,
162
169
.
Forster
,
B.
, &
Pavone
,
E. F.
(
2008
).
Electrophysiological correlates of crossmodal visual distractor congruency effects: Evidence for response conflict.
Cognitive, Affective, and Behavioral Neuroscience
,
8
,
65
73
.
Ghazanfar
,
A. A.
, &
Schroeder
,
C. E.
(
2006
).
Is neocortex essentially multisensory?
Trends in Cognitive Sciences
,
10
,
278
285
.
Gondan
,
M.
,
Niederhaus
,
B.
,
Rösler
,
F.
, &
Röder
,
B.
(
2005
).
Multisensory processing in the redundant-target effect: A behavioral and event-related potential study.
Perception and Psychophysics
,
67
,
713
726
.
Graziano
,
M. S. A.
,
Cooke
,
D. F.
, &
Taylor
,
C. S.
(
2000
).
Coding the location of the arm by sight.
Science
,
290
,
1782
1786
.
Guariglia
,
C.
, &
Antonucci
,
G.
(
1992
).
Personal and extrapersonal space: A case of neglect dissociation.
Neuropsychologia
,
30
,
1001
1009
.
Iriki
,
A.
,
Tanaka
,
M.
, &
Iwamura
,
Y.
(
1996
).
Coding of modified body schema during tool use by macaque postcentral neurons.
NeuroReport
,
7
,
2325
2330
.
Jung
,
T.-P.
,
Makeig
,
S.
,
Humphries
,
C.
,
Lee
,
T.-W.
,
McKeown
,
M. J.
,
Iragui
,
V.
,
et al
(
2000
).
Removing electroencephalographic artifacts by blind source separation.
Psychophysiology
,
37
,
163
178
.
Kammers
,
M. P. M.
,
Longo
,
M. R.
,
Tsakiris
,
M.
,
Dijkerman
,
H. C.
, &
Haggard
,
P.
(
2009
).
Specificity and coherence of body representations.
Perception
,
38
,
1804
1820
.
Kennett
,
S.
,
Eimer
,
M.
,
Spence
,
C.
, &
Driver
,
J.
(
2001
).
Tactile-visual links in exogenous spatial attention under different postures: Convergent evidence from psychophysics and ERPs.
Journal of Cognitive Neuroscience
,
13
,
462
478
.
Leube
,
D. T.
,
Knoblich
,
G.
,
Erb
,
M.
,
Grodd
,
W.
,
Bartels
,
M.
, &
Kircher
,
T. T.
(
2003
).
The neural correlates of perceiving one's own movements.
Neuroimage
,
20
,
2084
2090
.
Longo
,
M. R.
,
Betti
,
V.
,
Aglioti
,
S. M.
, &
Haggard
,
P.
(
2009
).
Visually induced analgesia: Seeing the body reduces pain.
Journal of Neuroscience
,
29
,
12125
12130
.
Longo
,
M. R.
,
Pernigo
,
S.
, &
Haggard
,
P.
(
2011
).
Vision of the body modulates processing in primary somatosensory cortex.
Neuroscience Letters
,
489
,
159
163
.
Macaluso
,
E.
, &
Driver
,
J.
(
2005
).
Multisensory spatial interactions: A window onto functional integration in the human brain.
Trends in Neurosciences
,
28
,
264
271
.
Makin
,
T. R.
,
Holmes
,
N. P.
, &
Zohary
,
E.
(
2007
).
Is that near my hand? Multisensory representation of peripersonal space in human intraparietal sulcus.
Journal of Neuroscience
,
27
,
731
740
.
Matsuhashi
,
M.
,
Ikeda
,
A.
,
Ohara
,
S.
,
Matsumoto
,
R.
,
Yamamoto
,
J.
,
Takayama
,
M.
,
et al
(
2004
).
Multisensory convergence at human temporo-parietal junction—Epicortical recording of evoked responses.
Clinical Neurophysiology
,
115
,
1145
1160
.
Meredith
,
M. A.
, &
Stein
,
B. E.
(
1983
).
Interactions among converging sensory inputs in the superior colliculus.
Science
,
221
,
389
391
.
Meredith
,
M. A.
, &
Stein
,
B. E.
(
1986
).
Spatial factors determine the activity of multisensory neurons in cat superior colliculus.
Brain Research
,
365
,
350
354
.
Mouraux
,
A.
, &
Iannetti
,
G.
(
2009
).
Nociceptive laser-evoked brain potentials do not reflect nociceptive-specific neural activity.
Journal of Neurophysiology
,
101
,
3258
3269
.
Murray
,
M. M.
,
Molholm
,
S.
,
Michel
,
C. M.
,
Heslenfeld
,
D. J.
,
Ritter
,
W.
,
Javitt
,
D. C.
,
et al
(
2005
).
Grabbing your ear: Rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment.
Cerebral Cortex
,
15
,
963
974
.
Papeo
,
L.
,
Longo
,
M. R.
,
Feurra
,
M.
, &
Haggard
,
P.
(
2010
).
The role of the right temporoparietal junction in intersensory conflict: Detection or resolution?
Experimental Brain Research
,
206
,
129
139
.
Ramachandran
,
V. S.
, &
Rogers-Ramachandran
,
D.
(
1996
).
Synaesthesia in phantom limbs induced with mirrors.
Proceedings of the Royal Society of London, Series B, Biological Sciences
,
263
,
377
386
.
Ramachandran
,
V. S.
,
Rogers-Ramachandran
,
D.
, &
Cobb
,
S.
(
1995
).
Touching the phantom limb.
Nature
,
377
,
489
490
.
Rock
,
I.
, &
Victor
,
J.
(
1964
).
Vision and touch: An experimentally created conflict between the two senses.
Science
,
143
,
594
596
.
Sambo
,
C. F.
, &
Forster
,
B.
(
2009
).
An ERP investigation on visuotactile interactions in peripersonal and extrapersonal space: Evidence for the spatial rule.
Journal of Cognitive Neuroscience
,
21
,
1550
1559
.
Schürmann
,
M.
,
Kolev
,
V.
,
Menzel
,
K.
, &
Yordanova
,
J.
(
2002
).
Spatial coincidence modulates interactions between visual and somatosensory evoked potentials.
NeuroReport
,
13
,
779
783
.
Simon-Dack
,
S. L.
,
Cummings
,
S. E.
,
Reetz
,
D. J.
,
Alvarez-Vazquez
,
E.
,
Gu
,
H.
, &
Teder-Sälejärvi
,
W. A.
(
2009
).
“Touched” by light: Event-related potentials (ERPs) to visuo-haptic stimuli in peri-personal space.
Brain Topography
,
21
,
261
268
.
Spence
,
C.
,
Nicholls
,
M. E.
,
Gillespie
,
N.
, &
Driver
,
J.
(
1998
).
Cross-modal links in exogenous covert spatial orienting between touch, audition, and vision.
Perception and Psychophysics
,
60
,
544
557
.
Squires
,
N. K.
,
Squires
,
K. C.
, &
Hillyard
,
S. A.
(
1975
).
Two variables of long latency positive waves evoked by unpredictable auditory stimuli in man.
Electroencephalography and Clinical Neurophysiology
,
38
,
387
401
.
Stein
,
B. E.
, &
Meredith
,
M. A.
(
1993
).
The merging of the senses.
Cambridge, MA
:
MIT Press
.
Sutton
,
S.
,
Braren
,
M.
,
Zubin
,
J.
, &
John
,
E. R.
(
1965
).
Evoked potentials correlates of stimulus uncertainty.
Science
,
150
,
1187
1188
.
Teder-Sälejärvi
,
W. A.
,
Di Russo
,
F.
,
McDonald
,
J. J.
, &
Hillyard
,
S. A.
(
2005
).
Effects of spatial congruity on audio-visual multimodal integration.
Journal of Cognitive Neuroscience
,
17
,
1396
1409
.
Tsakiris
,
M.
,
Costantini
,
M.
, &
Haggard
,
P.
(
2008
).
The role of the right temporo-parietal junction in maintaining a coherent sense of one's body.
Neuropsychologia
,
46
,
3014
3018
.
Yamaguchi
,
S.
, &
Knight
,
R. T.
(
1991
).
Anterior and posterior association cortex contributions to the somatosensory P300.
Journal of Neuroscience
,
11
,
2039
2054
.
Yeung
,
N.
,
Botvinick
,
M. M.
, &
Cohen
,
J. D.
(
2004
).
The neural basis of error detection: Conflict monitoring and the error-related negativity.
Psychological Review
,
111
,
931
959
.
Zacks
,
J.
,
Rypma
,
B.
,
Gabriel
,
J.
,
Tversky
,
B.
, &
Glover
,
G.
(
1999
).
Imagined transformations of bodies: An fMRI investigation.
Neuropsychologia
,
37
,
1029
1040
.