Humans typically rely upon vision to identify object shape, but we can also recognize shape via touch (haptics). Our haptic shape recognition ability raises an intriguing question: To what extent do visual cortical shape recognition mechanisms support haptic object recognition? We addressed this question using a haptic fMRI repetition design, which allowed us to identify neuronal populations sensitive to the shape of objects that were touched but not seen. In addition to the expected shape-selective fMRI responses in dorsal frontoparietal areas, we observed widespread shape-selective responses in the ventral visual cortical pathway, including primary visual cortex. Our results indicate that shape processing via touch engages many of the same neural mechanisms as visual object recognition. The shape-specific repetition effects we observed in primary visual cortex show that visual sensory areas are engaged during the haptic exploration of object shape, even in the absence of concurrent shape-related visual input. Our results complement related findings in visually deprived individuals and highlight the fundamental role of the visual system in the processing of object shape.
A critical question in cognitive neuroscience concerns whether or not sensory cortices are modality specific or are involved as part of a distributed system in the analysis of stimuli presented in other sensory modalities. In vision, primary visual cortex (V1) extracts and binds fundamental shape information (Field, Hayes, & Hess, 1993), which is combined into progressively more complex shape representations in higher ventral visual cortical areas (Kourtzi & Connor, 2011; Kourtzi, Tolias, Altmann, Augath, & Logothetis, 2003). Here we used fMRI to examine the potential involvement of primary visual cortex and other areas in ventral visual cortex during the haptic processing of objects that were touched but not seen.
Previous imaging studies of haptic shape encoding in the visual stream have predominantly focused on haptic shape sensitivity in higher-order areas such as the lateral occipital complex (LOC). Shape processing via touch has been shown to engage the LOC, and object-related activity in the LOC dissociates from the processing of felt textures (Kim & James, 2010; Naumer et al., 2010; Allen & Humphreys, 2009; Tal & Amedi, 2009; Stilla & Sathian, 2008; Peltier et al., 2007; Amedi, von Kriegstein, van Atteveldt, Beauchamp, & Naumer, 2005; Pietrini et al., 2004; Zhang, Weisser, Stilla, Prather, & Sathian, 2004; Amedi, Jacobson, Hendler, Malach, & Zohary, 2002; Grefkes, Weiss, Zilles, & Fink, 2002; James et al., 2002; Amedi, Malach, Hendler, Peled, & Zohary, 2001). Activation of the LOC during object touch has been noted both in visually deprived individuals (Amedi et al., 2007; Burton, McLaren, & Sinclair, 2006; Sadato et al., 1996), where it might reflect compensatory adaptation, and in healthy sighted observers, where this seems less likely (Amedi et al., 2001).
Evidence for somatosensory processing in primary visual cortex (V1), in contrast to the data on LOC, has largely come from individuals who have been blind from an early age. For example, the early blind (but not sighted controls) have shown activation of visual cortex during Braille reading and other tactile discrimination tasks using PET (Buchel, Price, Frackowiak, & Friston, 1998; Sadato et al., 1996, 1998). Using fMRI, tactile processing of embossed letters can elicit greater activation of early visual cortex in blind than sighted observers (Burton et al., 2006). Similarly, in blind individuals, TMS over visual cortex disrupts haptic letter recognition, demonstrating the necessary involvement of primary visual cortex in such individuals. The same is not the case for sighted controls (Cohen et al., 1997).
Attempts to evaluate whether V1 is involved in processing tactile shapes have predominantly assessed texture perception, involving (e.g.) dot-spacing or roughness discrimination tasks, so the involvement of this area in tactile shape perception is unknown (Merabet et al., 2007; Weisser, Stilla, Peltier, Hu, & Sathian, 2005). In contrast to the paucity of evidence for tactile shape processing in primary visual cortex, emerging data in studies of human audition and somatosensation have provided tantalizing evidence that primary sensory areas can process shape-related information arising from other sensory modalities. For example, visual displays of objects that strongly imply sound or touch can elicit content-specific activity patterns within primary auditory (Meyer et al., 2010) and somatosensory cortex (Meyer, Kaplan, Essex, Damasio, & Damasio, 2011), respectively.
Here we used fMRI in healthy sighted individuals to examine whether low-level (V1) and intermediate-level form-selective area (V4) are involved in processing object shape via touch. To confirm that our haptic repetition design and stimuli were sufficient to elicit repetition effects, we first examined shape-selective responses to palpated objects in the LOC—an area in which we might expect to observe shape-selective repetition effects based on the involvement of this area in previous studies of object touch, as outlined above. To enable a fine-grained analysis of visual cortex function, we used a haptic fMRI repetition design analogous to those previously used in vision (Grill-Spector & Malach, 2001; Grill-Spector et al., 1999) to identify neural populations sensitive to the shape of objects that are touched but not seen. Repetition designs have been applied successfully in the domain of somatosensation to examine brain regions involved in tactile pattern and frequency coding (Tamè et al., 2012; Li Hegner, Lee, Grodd, & Braun, 2010) and high-level shape processing (Tal & Amedi, 2009). In our study, all observers were normally sighted to rule out the involvement of adaptive changes linked to visual deprivation. The palpated objects in our study were not visible to participants and throughout the experiment observers maintained their gaze upon a central fixation point. We functionally isolated low-, intermediate-, and high-level visual shape processing areas using separate visual localizer tasks and determined the extent to which each of these areas were sensitive to haptic shape repetition. Brain areas that are sensitive to haptic shape cues should show a reduced fMRI response when object shape is repeated versus novel.
Ten healthy right-handed observers (seven women, 22–38 years old), with normal or corrected-to-normal vision and normal somatosensation, each participated in three fMRI scans. Participants first completed a haptic fMRI repetition study and subsequently returned to complete two follow-up localizer tasks: one designed to isolate object-selective LOC and the second to isolate intermediate-level color/form processing area V4 and visual cortex. All participants were naïve with respect to the experimental design and hypotheses. Informed consent was obtained in accordance with procedures approved by the university ethics review board.
Stimuli for the haptic fMRI experiment comprised a set of 28 real-world objects and an additional set of six “nonobjects” for use on response trials (Figure 1). The “nonobjects” were 3-D shapes each constituted of parts of real objects. Pilot experiments, involving a different pool of individuals from the fMRI participants, confirmed that the real objects could each be identified within 3 sec and that the “nonobjects” were not easily misidentified as real objects. For the LOC localizer, stimuli consisted of 300 × 300 pixel grayscale images and line drawings of familiar and novel objects and scrambled versions of each set, each with overlapping grid lines as described in previous studies (Kourtzi, Erb, Grodd, & Bulthoff, 2003; Kourtzi & Kanwisher, 2000, 2001). Images were back-projected onto a screen, which was viewed via a mirror attached to the top of the head coil yielding a viewing distance of 65 cm. For the intermediate-level form (V4) and V1 localizer, stimuli consisted of 28,300 × 300 pixel high-resolution color photographs of the same set of objects presented during the haptic experiment, each photographed from a canonical perspective using a Sony (Tokyo) Alpha DSLR-A100 camera. The images were presented on a blank white background and overlaid with a central gray fixation cross (average image size = 120 × 150 pixels) and viewed via a mirror attached to the head coil, as described above.
Procedure and Design
In the haptic fMRI study, participants palpated sequences of real objects with the left hand. The experiment used a blocked design in which somatosensory sequences were interspersed with blocks of baseline fixation. Each stimulus block consisted of 18 consecutive somatosensory events: 16 events were object trials and the remaining two events were catch trials involving “nonobjects.” Participants pressed a response button with the index finger of the right hand each time a catch trial was detected. Catch trials were employed to ensure that participants remained vigilant throughout the runs. Participants performed this task well, and incorrect identifications of nonobject trials were <1%. Importantly, the number of response (catch) trials was identical across all conditions. Participants were not exposed to the stimulus set before the testing session. The haptic experiment used a parametric design in which we modulated the repetition frequency of haptically presented objects. We tested four levels of haptic object repetition (Figure 1A). In the Same condition, a single object was repeatedly presented on all 16 object trials within a block. In the Different condition, a different object was presented on all 16 object trials. Repetition effects in-between the Same and Different conditions were examined using two intermediate conditions. In the Alternate condition, two different objects were presented in alternating order across each trial in a block. Finally, in the Pairs condition, eight different objects were presented within a block, each one twice in succession. Catch trials appeared randomly within each block of object trials (with the exception of the Pairs condition in which these trials arose before or after a pair, never in-between). The identity of the repeated stimulus (e.g., the banana in Figure 1) was substituted across runs (e.g., shaving brush/ball of string/toothbrush) to minimize long-term carryover repetition effects. Each run of the haptic experiment consisted of one block of trials from each of the four conditions, yielding four stimulus blocks within each run (Figure 1C). The order of conditions was counterbalanced within participants using a balanced latin square design. Half of the participants completed conditions according to the order specified by a latin square, and the remaining participants completed conditions according to a mirror (reversed) square.
The stimulus sequence on each haptic trial is illustrated in Figure 1B. Trials began with a 1000-msec central visual fixation cross, followed by a 3500-msec haptic palpation period. The start of the palpation window was signaled by a 500-msec auditory tone, delivered to both the participant and the experimenter. At the onset of the tone, the experimenter placed a stimulus into the participant's open left hand. Participants were instructed to palpate the object for the duration of the trial and to make a button-press response where appropriate (i.e., catch trials). The end of the palpation period was indicated by a second 500 msec auditory tone (lower in pitch than the first) and the simultaneous offset of the visual fixation cross (2500 msec). Participants were instructed to release the object at the onset of the second auditory tone in preparation for the upcoming trial. Each trial had a total duration of 7 sec. With 18 trials per block, this yielded a total block duration of 126 sec. Fixation blocks (27 sec) were positioned at the start and end of the experiment, and in-between each stimulus block, yielding a total run length of 639 sec (with a repetition time [TR] = 3, yielding 213 functional volumes). Participants completed four runs of the haptic experiment.
All participants completed two further localizer tasks designed to identify V1 and intermediate-level color/form-selective area, most likely corresponding to area V4 (Gallant, Shoup, & Mazer, 2000), and high-level shape-selective regions within LOC. The LOC localizer had a blocked design with 16 stimulus blocks presented in a balanced order and interleaved fixation periods of 16 sec each. Twenty images were presented within each block. Images were presented for 250 msec with a blank interval of 550 msec between images. Participants were instructed to passively view the images while fixating. The low- and intermediate-level color/form area localizer used a block design in which participants viewed sequences of the same set of 18 different high-resolution color stimuli that had been palpated in the haptic experiment. Each trial began with a central black fixation cross (1000 msec), followed by the stimulus image (or nonobject) for 500 msec. Images were separated by a blank intertrial interval of 2500 msec. Total trial length was 4000 msec; with 18 images per block, the total block duration was 72 sec. Stimulus blocks in this task were interspersed with three other stimulus conditions, which were used for a separate experiment (not considered further here). Stimulus blocks were interleaved with 18-sec fixation blocks. A further 18 sec of fixation was added to the start and end of each run; the duration of the entire experiment was 378 sec. Block order was counterbalanced within and across participants in a similar fashion to the haptic experiment. Participants completed four runs of the low-level visual/color area localizer and two runs of the LOC localizer. Stimulus presentation and timing were controlled using E-Prime software, except for the LOC localizer, which was controlled using Matlab.
Scanning was carried out on a 3-T Philips (Andover, MA) MRI scanner with an eight-channel SENSE parallel head coil. Functional data for the haptic experiment and V1 localizer task were acquired using a single shot T2* weighted single-shot gradient-echo EPI sequence (echo time = 35 msec, TR = 3000 msec, field of view = 232 × 288 × 175 mm, voxel size = 2.5 × 2.5 × 2.5, 49 axial slices). Scan parameters for the LOC localizer task were as follows: TR = 200 msec, echo time = 30 msec, voxel size = 3.0 × 3.0 × 3.0, 32 axial slices. A total of 213 volumes were collected per run for the Haptic experiment, 126 volumes per run for the V4/primary visual cortex localizer task, and 168 volumes for each LOC localizer run. Functional data were aligned and reinterpolated to a high-resolution anatomical image for each participant obtained using an echo-planar 3-D T1-weighted image (1 × 1 × 1 mm slice thickness, 175 slices, field of view = 232 × 256 × 175).
Data Processing and ROI Analyses
Data were preprocessed and analyzed using Brain Voyager QX (Version 1.10.2, Brain Innovation, Maastricht, Netherlands). Functional data were initially preprocessed by motion correcting to the functional volume acquired closest in time to the anatomical scan and removing linear trends. No high-pass filter was applied to ensure that cycles of interest within each haptic run were not eliminated. 2-D functional images were aligned to the 3-D anatomical data and then transformed into Talairach space. Runs in which translational head motion spikes exceeded 2 mm of translation and/or 2° of rotation were excluded from the analyses (totaling only one run for one participant in the haptic experiment). Haptic data for one participant was excluded because of technical issues during the experiment (Runs 1–4) and above-threshold head motion in two subsequent scans (Runs 5–6).
Voxelwise Group Analyses
We first performed whole-volume voxelwise analyses on the group haptic data. fMRI signals for each participant were spatially smoothed (8 mm, FWHM Gaussian kernel). The data for the main experiment were smoothed to account for variability in the spatial location of functional regions across observers and improve signal-to-noise ratio. Data from each haptic run were analyzed using separate predictor functions for each experimental condition. The experiment had a block design, and all trials were included in the statistical analyses. Predictor functions were generated for the four conditions by convolving a rectangular wave function with a standard hemodynamic response function. Individual participant data were analyzed using a single-subject general linear model (GLM). Data for each participant were processed using a percent signal change (%SC) transformation and a correction for serial (temporal) correlations. Group data were then analyzed using a random effects GLM. Brain regions that showed significant haptic fMRI repetition effects at the whole-brain level were isolated using the contrast [+Different −Same] (Figure 2). The resultant group activation map was set to a statistical threshold of p < .005. A correction for cluster size was not applied to the group maps because small isolated clusters of activated voxels had already been removed (or greatly reduced in number) from the statistical maps by spatial smoothing; an additional cluster correction could therefore eliminate meaningful but spatially restricted activation in smaller areas such as primary visual and somatosensory cortex—areas in which isolated effects can be masked because of reduced signal overlap across observers.
ROIs were isolated separately for each observer using independent functional localizer tasks. For each individual, ROIs were identified by selecting voxels within regions of ventral occipitotemporal cortex that were activated more strongly by intact images of objects than control stimuli. ROIs were isolated by first locating the peak voxel of functional activation within each anatomical area (described separately for each ROI below). Next, the statistical map for the localizer task for each individual was set to a minimum statistical threshold (t > 3.0, p < .001), and a VOI was selected around the peak voxel. LOC was identified by contrasting fMRI responses to grayscale intact objects versus their scrambled counterparts (Kourtzi & Kanwisher, 2000; Malach et al., 1995). As in previous studies in vision (Kourtzi, Erb, et al., 2003; Grill-Spector, Kushnir, Hendler, & Malach, 2000; Grill-Spector et al., 1999), we isolated two subregions of the LOC complex: a region on the lateral surface near the lateral occipital sulcus (area LO) and a ventral occipitotemporal region extending into the posterior and midfusiform gyrus and occipitotemporal sulcus (pFus). Area LO was defined by locating the peak voxel of activation within the lateral convexity of occipitotemporal cortex, while pFus was defined by selecting the peak voxel more ventrally and anteriorly within the fusiform sulcus of inferior temporal cortex. Low- and intermediate-level visual form processing areas were defined by contrasting fMRI responses to colored objects versus fixation in the visual localizer task involving colored objects. Intermediate-level visual form area (V4) was identified by selecting the peak functional voxel within the temporo-occipital junction around the posterior branch of the collateral sulcus or on its lateral aspect within the fusiform gyrus (posterior to pFus; Gallant et al., 2000). Primary visual cortex ROIs were identified by selecting the voxel of peak functional activation on, or adjacent to, the calcarine sulcus. VOIs of up to 5 mm3 (i.e., 125 voxels) for V1 and V4 and 10 mm3 (i.e., 1000 voxels) for LO/pFus were selected around the peak voxel of activation in each region. ROI sizes for V1 and V4 were comparatively smaller than those within LOC to ensure that our analyses within these areas reflected fMRI responses within the defined areas of interest and did not include neighboring areas of occipital cortex (see Figures 4 and 5). ROI sizes for the LOC reflect functional responses within comparatively more spatially extended regions of lateral occipital and ventral temporal cortex (see Figure 3). One participant (denoted by the symbol (†) in Figure 4) showed attenuated activation in the visual localizer task, and consequently a threshold of p < .05 was applied to identify V1 in this instance.
Data from the haptic experiment were extracted from each ROI, and subsequent analyses were performed on unsmoothed data. Functional data were not smoothed in the single-participant ROI analyses because there was no longer a problem of spatial mismatch between the functional regions of different observers and because smoothing could lead to the loss of otherwise meaningful fMRI signals in areas with a smaller spatial scale of activation or carryover activation arising from signals in nearby regions. fMRI signal intensities within each region were averaged, and the time courses were converted to %SC relative to the baseline fixation periods. The %SC fMRI data for each participant were then averaged across conditions and normalized for each time point within each hemisphere (L/R) and ROI, starting from the point of peak response, which occurred at a lag of 6 sec post-stimulus onset (Boynton, Engel, Glover, & Heeger, 1996) to the final volume of the block (volume 43). Data were normalized using the formula (Xt − Min/Max − Min), where Xt is the observed %SC at time point t within a block; Min and Max are the minimum and maximum fMRI responses across the four haptic conditions, respectively. The mean normalized signal changes for each haptic condition were averaged across participants to yield group data. The pattern of fMRI responses in each of the haptic repetition conditions was the same in the left and right hemispheres for each ROI (i.e., fMRI responses were highest in the Different condition, followed by the Pairs and then Alternate conditions, and lowest in the Same condition), so the data were averaged across hemispheres. Repetition effects for each ROI in the haptic experiment were defined based on a main effect of Condition (number of different objects: 16, 8, 2, 1) using a one-way repeated-measures ANOVA (p < .05) and a significant difference between the Different and Same conditions in paired-samples t tests (p < .05; two-tailed).
In the domain of vision, selectivity to object shape in the ventral visual system has been measured based on the strength of fMRI responses to parametric manipulations of object repetition frequency (Grill-Spector & Malach, 2001; Grill-Spector et al., 1999). Here we used an analogous design in the domain of haptics, in which we manipulated the repetition frequency of touched object shape to examine brain regions that are involved in processing and/or representing haptically derived shape cues. We first identified regions across the whole brain that showed sensitivity to haptic shape repetition. Next, we functionally isolated ventral visual areas LOC, V4, and V1, based on their responses to visual stimulation in separate tasks, and examined the sensitivity of each region to haptic shape. We were primarily interested in haptic repetition effects in primary visual cortex; we began our ROI analyses in the LOC to validate the sensitivity of our procedure and design, before examining haptic repetition effects in areas progressively more posterior within the occipital cortex.
Whole-brain Voxelwise Analyses
The design of the haptic experiment is illustrated in Figure 1. Shape-selective responses were examined using a parametric fMRI repetition design in which areas sensitive to object identity were expected to show greater fMRI adaptation (i.e., an attenuated fMRI response) with corresponding increases in repetition frequency (Grill-Spector, Henson, & Martin, 2006). We started by using whole-brain voxelwise contrasts to identify regions that were most sensitive to haptic repetition using the contrast [+Different − Same] (Figure 2). We observed robust bilateral haptic fMRI adaptation effects within lateral and ventromedial regions of occipitotemporal cortex, in the vicinity of the LOC (left hemisphere: 8297 voxels, maximum t value = 18.32; right hemisphere: 1990 voxels, maximum t value = 7.34). Bilateral effects of haptic shape repetition were also observed within dorsal and anterior intraparietal sulcus, the insula, and in the precentral sulcus in the vicinity of ventral premotor area (Chouinard & Paus, 2006). Haptic shape repetition effects were also observed bilaterally in somatosensory cortex, extending along the post-central gyri and sulci and ventrally into the parietal operculum, corresponding to the expected location of secondary somatosensory cortex as described in previous studies (Burton, Sinclair, Wingert, & Dierker, 2008; Eickhoff, Grefkes, Zilles, & Fink, 2007; Eickhoff, Amunts, Mohlberg, & Zilles, 2006; Eickhoff, Schleicher, Zilles, & Amunts, 2006). Shape-selective fMRI responses were observed too within middle frontal gyrus in the right hemisphere. We also found isolated but above-threshold haptic repetition-related clusters in the vicinity of the calcarine sulcus in primary visual cortex (Figure 2, bottom insets), most notably in the left hemisphere. Note that the calcarine structure varies considerably from one individual to the next, and in the context of a random-effects voxelwise GLM, isolated effects can be masked because of reduced signal overlap across observers. Our result was observed despite this constraint. Interestingly, the haptic fMRI adaptation effects within many of the activated areas (e.g., within somatosensory areas, LOC, and V1), although bilateral, appeared to have a greater spatial extent in the left than the right hemisphere.
Next, we measured haptic shape repetition effects within visually defined ROIs along occipitotemporal cortex. Visual areas for our ROI analyses were defined separately for each participant based on visual fMRI tasks conducted separately from the main haptic experiment (see Methods). Haptic repetition effects in each ROI were defined on the basis of two criteria: a main effect of haptic Condition (number of different objects: 16, 8, 2, 1) as defined using a one-way repeated-measures ANOVA, and a significant response difference (Δ) between the Different (D) and Same (S) conditions (ΔD,S two-tailed; Konen & Kastner, 2008). To the extent that a visual area is sensitive to object shape, repetition effects should be maximal in the Same condition (i.e., lowest fMRI response), followed by the Pairs and Alternate conditions, and lowest in the Different condition (i.e., highest fMRI response). Regions that are not sensitive to repetition are expected to show a constant fMRI response across the four conditions.
Haptic Shape Sensitivity in LOC
To validate our procedure and design, parametric effects of haptic repetition were measured first within the LOC—an area that shows strong sensitivity to visual shape repetition (Grill-Spector et al., 1999). Following from analogous studies in vision, repetition effects were examined within two visual subregions of the LOC: area LO, located on the lateral bank of occipitotemporal cortex, and posterior fusiform sulcus (pFus), positioned comparatively more anteriorly on the ventromedial surface of the temporal lobe. Each subregion was localized by contrasting fMRI responses to grayscale images of intact objects with images of their scrambled counterparts (see Methods). Figure 3A illustrates visual ROIs selected within LO (top) and pFus (bottom) for each participant, overlaid on the average high-resolution anatomical image. The effect of haptic shape repetition in the anterior and posterior subregions of the LOC was highly consistent across participants within areas LO and pFus, with qualitatively higher mean fMRI signals in the Different versus Same haptic conditions (Figure 3B). Mean signal changes evoked by the four haptic conditions averaged across participants are displayed in Figure 3C. As expected, repetition effects were highly significant in both subregions (LO: main effect of repetition, F = 40.02, p < .001 [ΔD,St = 8.60, p < .001]; pFus: main effect of repetition, F = 16.84, p < .001 [ΔD,St = 5.61, p = .001]; repetition effects were significant within both left and right hemispheres), and both subregions showed a parametric effect of repetition, with a gradual fMRI response reduction with increasing repetition frequency.
Haptic Shape Sensitivity within Low- and Intermediate-level Visual Form Processing Areas
Next, we examine shape-specific responses within the posterior branch of the collateral sulcus, in an area most likely to correspond with V4 (Gallant et al., 2000). Axial anatomical slices in Figure 4A illustrate visual fMRI responses for each participant within ventral temporal cortex that showed stronger fMRI responses to colored images of objects versus fixation (p < .001; activation shown in yellow). ROIs were centered upon the peak voxel of activation around the posterior branch of the collateral sulcus (shown in purple; see Methods). The effect of haptic shape repetition was highly consistent across participants, with qualitatively higher fMRI signals in the Different versus Same conditions in eight of nine observers (Figure 4B). Analysis of mean fMRI signal changes evoked during the four repetition conditions revealed a significant haptic repetition effect (main effect of haptic repetition: F = 7.10, p = .001; Δ16,1, t = 4.03, p = .004), and there was a reduction in fMRI response with increases in repetition frequency (Figure 4C).
Finally, we examined sensitivity to haptic shape repetition within primary visual cortex. Saggital anatomical slices in Figure 5A illustrate fMRI responses for each observer along the calcarine sulcus in each hemisphere that showed stronger fMRI responses to images of objects versus fixation (p < .001; activation shown in yellow). ROIs in V1 were centered upon the voxel of peak activation on or immediately adjacent to the calcarine sulcus (shown in red; see Methods). As expected, the region of peak activation in the calcarine sulcus elicited by our colored object stimuli was almost identical to that elicited by the grayscale object stimuli used in the LOC localizer. Functional data for the haptic experiment were extracted from each ROI and averaged across hemispheres. The effect of haptic shape repetition within primary visual cortex was strikingly consistent across observers, with qualitatively greater fMRI responses in the Different versus Same conditions in all participants (Figure 5B). Analysis of mean signal changes evoked during the four haptic repetition conditions showed a significant effect of repetition (main effect of repetition: F = 7.10, p = .001; ΔD,St = 4.03, p = .004; repetition effects were significant within both left and right hemispheres; Figure 5C).
Using a haptic fMRI repetition paradigm, we found that visually defined areas in occipital and temporal cortex were involved in analyzing the shape of objects that were touched, but not seen. Although numerous dorsal frontoparietal regions were sensitive to the repetition frequency of haptically explored objects, strong shape-selective responses were observed ventrally in LOC (Figure 2). ROI analyses confirmed that fMRI responses in both area LO and pFus subregions of the LOC were sensitive to haptic shape (Figure 3). In addition to this, primary visual cortex and medial-temporal visual form processing areas also showed a significant reduction in fMRI response with increases in haptic shape repetition (Figures 4 and 5). The pattern of haptic repetition effects we have observed in regions of occipital and ventral temporal cortex is strikingly similar to those reported in analogous fMRI repetition studies in the domain of vision (Grill-Spector et al., 1999). Taken together, our data suggest that low-level visual areas are functionally involved in processing the shape of objects explored with the hand.
We used whole-brain analyses of fMRI repetition effects to isolate regions involved in processing the shape of objects explored via touch. Shape-selective fMRI responses were observed within dorsal frontoparietal areas, including the IPS, primary somatosensory areas along the post-central gyri and sulci and parietal operculum (corresponding to area SII), the precentral sulcus (corresponding to ventral premotor area), anterior insula, and middle frontal gyrus. In addition, bilateral shape-selective fMRI responses were observed within occipital cortex and on the lateral and ventral surface of the temporal lobe within the vicinity of the LOC. The brain regions in parietal, frontal, and temporal cortex that showed fMRI adaptation effects in our group analysis are consistent with areas reported in previous fMRI studies of haptic shape recognition and in current models of haptic object recognition (Lacey & Sathian, 2011; James & Kim, 2010; Lucan, Foxe, Gomez-Ramirez, Sathian, & Molholm, 2010; Lacey, Tal, Amedi, & Sathian, 2009; James, Kim, & Fisher, 2007; Amedi et al., 2002, 2005; James, James, Humphrey, & Goodale, 2005). Interestingly, in our study, many of these fMRI responses were moderately left-lateralized (a greater number of contiguously significant voxels in the left hemisphere as compared with the right). Left lateralization of fMRI responses, particularly in visual object-processing areas along occipitotemporal cortex, has recently been explained as being due to left-lateralized high spatial frequency channels (Woodhead, Wise, Sereno, & Leech, 2011). It is possible that the pattern of our haptic results are related to this, but future experiments would be necessary to address this possibility more directly. The sensitivity to haptic shape we observed within LOC is compatible with a growing list of imaging (James et al., 2002; Amedi et al., 2001), neuropsychological (Allen & Humphreys, 2009), and TMS (Mancini, Bolognini, Bricolo, & Vallar, 2011) studies that have implicated this region in haptic shape recognition (for comprehensive reviews, see Kim & James, 2011; Lacey & Sathian, 2011; James et al., 2005). In contrast to previous reports (Tal & Amedi, 2009; Amedi et al., 2001, 2002), however, we found shape-selective responses to touched objects were not limited to subregions of the LOC but were evident across the complex within both posterodorsal (LO) and anteroventral (pFus) subregions, perhaps reflecting the increased sensitivity of the fMRI repetition procedure.
Haptic Shape Processing in Low- and Intermediate-level Visual Areas
Objects examined via touch alone also elicited shape-specific fMRI responses within low- and intermediate-level and visual form processing areas. In our ROI analyses, fMRI responses in primary visual cortex and V4 were highly consistent across observers and there was a gradual reduction in fMRI response with systematic increases in haptic shape repetition. These data suggest that haptic shape processing engages much of the ventral visual system. Our data provide important new evidence in the domain of vision in humans (Vasconcelos et al., 2011) that a primary sensory area that has not been stimulated directly (with relevant sensory input) can represent information about object shape when this information has been acquired from a different sensory modality (Meyer et al., 2010, 2011).
The observed calcarine response to tactile form in our study concurs with previous reports of calcarine activation in tasks requiring the discrimination of tactile surfaces (Merabet et al., 2007; Burton et al., 2006; Weisser et al., 2005). Here we extend these findings in several important respects. First, previous studies have found a nonspecific response of primary visual cortex to tactile stimulation (i.e., letter [Weisser et al., 2005] or texture [Merabet et al., 2007; Burton et al., 2006] vs. baseline), whereas we used a sensitive repetition design to reveal a stimulus-specific response to tactile form in primary visual cortex—one that was modulated parametrically by changes in object shape. In addition, whereas previous studies have reported primary visual cortex activation in early blind individuals (Burton et al., 2006; Buchel et al., 1998; Sadato et al., 1996, 1998) or in healthy observers following visual deprivation (i.e., 2 hr of blindfolding; Weisser et al., 2005), we observed shape-specific responses in primary visual cortex in healthy observers who were not deprived of visual input, either before or during the haptic task.
These data raise questions about the potential contribution of primary visual cortex to haptic shape recognition. A primary sensory area remote from the site of initial stimulation might be recruited for stimulus processing if it is well suited to the analysis of the relevant sensory input and if doing so confers a perceptual advantage—for example, by improving sensitivity, optimizing processing speed, or reducing computational demand. The integration of pattern information across multiple fingers is understood to be relatively poor, thereby making the “effective field of view” of an object significantly smaller than in vision (Loomis, Klatzky, & Lederman, 1991). Neurons in primary visual cortex have small receptive fields that provide high-resolution information about local changes in contrast, orientation, and spatial frequency. As a preliminary processor of basic units of shape (DeValois & DeValois, 1988), primary visual cortex could provide high-resolution details about haptically explored edges, their spatial frequency, and curvature. Indeed, TMS of visual cortex has been shown to interfere with fine-grained tactile discrimination (Merabet et al., 2004; Zangaladze, Epstein, Grafton, & Sathian, 1999). Similarly, exploring objects via touch is not holistic (as in vision) but involves a characteristically serial feature-by-feature analysis, and the object's position on the receptor surface (the hand) changes during the course of exploration. Primary visual cortex might therefore assist in assembling temporally piecemeal shape cues derived from touch to form a more stable representation of shape.
Multisensory Shape Processing in the Visual System
The neural processes and networks that support multisensory activation of primary sensory areas in humans remain largely undetermined (Schroeder & Foxe, 2005). However, evidence suggests that somatosensory responses in visual areas are mediated by the combined activity of at least two separate mechanisms: a top–down signal arising from multimodal association areas and direct feed-forward signal from long-range cortico-cortical pathways in primary somatosensory cortex (Merabet et al., 2007). Presumably, top–down signals from high-level areas preserve the spatial organization of haptically acquired features via multisensory coordinate transformations (Avillac, Deneve, Olivier, Pouget, & Duhamel, 2005).
The shape-selective responses we observed within LOC are consistent with previous “multimodal” accounts that posit that information from vision and touch converges within LOC (or the “tactile-visual” area LOTV)—an area that forms a critical locus for cross-modal visuohaptic shape representation (Lacey & Sathian, 2011; James & Kim, 2010; Lucan et al., 2010; Lacey et al., 2009; Tal & Amedi, 2009; James et al., 2005, 2007; Peltier et al., 2007; Amedi et al., 2001, 2002, 2005; Beauchamp, 2005). This “convergence zone” is argued to be multimodal or even “amodal” (Kim & Zatorre, 2011; Amedi et al., 2007), in the sense that shape information is represented in abstract form, irrespective of the origin of sensory input. However, we also observed shape-specific responses in low- and intermediate-level visual processing areas and attributing repetition effects in different areas to different underlying selectivity profiles can be problematic. Our data therefore raise the question of whether cross-modal response properties should also be extended to neural populations within primary visual cortex. Alternatively, although LOC is recruited during visual and haptic shape recognition, this could reflect the activity of interdigitated subpopulations of unimodal cells (Banati, Goerres, Tjoa, Aggleton, & Grasby, 2000).
The shape-selective responses we observed in ventral occipital areas could also reflect processes related to visual imagery (Stokes, Thompson, Cusack, & Duncan, 2009; Pietrini et al., 2004; Kosslyn et al., 1999; Kosslyn, Thompson, Kim, & Alpert, 1995). However, Amedi et al. (2001) found that haptic shape-based responses in LOC could not be accounted for by visual imagery, and evidence from patients with visual object agnosia (Allen & Humphreys, 2009) and early blind individuals (Amedi et al., 2007; Sadato et al., 1996) suggests that visual experience and/or imagery are not necessary for LOC activation. Shape-selective responses in visual cortex could also reflect preparatory attention (Stokes, Thompson, Nobre, & Duncan, 2009), the maintenance of object-specific information within visual working memory (Harrison & Tong, 2009), or the retrieval of form-based information from memory (Ferber, Humphrey, & Vilis, 2003). Indeed, retrieving information from visual or verbal memory has been shown to elicit modality-specific fMRI activation within brain regions involved in the initial sensory processing of the same stimuli (Wheeler, Petersen, & Buckner, 2000). It is possible, therefore, that the shape-selective fMRI responses we observed throughout the visual system during object touch reflects a retrieval-based “reactivation” process that propagates to early sensory areas (Buckner & Wheeler, 2001). Our findings raise the intriguing possibility that such shape-based reactivation processes could extend to primary modality-specific areas as well as across sensory modalities. Top–down effects of expectation or prediction have also been argued to provide a plausible explanation for fMR repetition phenomena (Summerfield, Trittschuh, Monti, Mesulam, & Egner, 2008).
We used an oddball identification task to ensure continuity of vigilance during the blocks. Our logic was that, to consistently detect the oddball, observers would have to attend to the shape properties of an object presented within a block. It is worth noting that the same number of oddball occurrences was implemented within each type of block. This means that, before the presentation of a stimulus within a block, there is equivalent uncertainty as to whether or not the upcoming stimulus will be an oddball. Equating for uncertainty of oddball occurrence between block types should result in identical levels of vigilance between the different conditions. If, instead, vigilance is determined by the variability of object shape within a block—irrespective of the occurrence of the oddball—then one might expect greater fMRI responses in the Different versus Same blocks. However, this criticism applies to all fMRI adaptation studies that have employed block designs (Summerfield et al., 2008). Furthermore, if the fMRI responses we observed were predominantly because of attentional effects, one might expect strong fMRI responses in frontoparietal circuits classically involved in attentional control and distractor filtering (Yantis & Serences, 2003; Corbetta & Shulman, 2002). In our whole-brain random effects analysis, however, we observed robust and spatially extensive responses in early and midlevel visual and somatosensory cortices but comparatively reduced responses in frontal and parietal cortex. Nevertheless, it will be informative in future studies to examine directly the extent to which brain regions recruited during visual attentional control match those invoked during haptic shape recognition tasks. Furthermore, future investigations could examine fMRI responses to haptically presented objects that vary parametrically in their shape features (Bodegård, Geyer, Grefkes, Zilles, & Roland, 2001; Hadjikhani & Roland, 1998; Klatzky, Lederman, & Reed, 1989).
Finally, our results do not imply that the visual system is necessary for haptic object recognition, which needs to be tested using intervention approaches. However, we speculate that the visual system is recruited during the recognition of objects via touch because the visual object recognition system is rapid and efficient, and shape cues processed within visual circuits may be relayed back to somatosensory and motor areas to guide ongoing motoric exploration. Whatever the case, our results demonstrate that the visual system is routinely recruited to process the shape of objects explored via touch in normal sighted observers.
We would like to thank Melvyn Goodale and Tutis Vilis for helpful comments on the manuscript, Alessia Correani for her help with running the experiments, and Alex Coros for assistance with data analysis. The work was supported by a grant from the Biotechnology and Biology Research Council, UK.
Reprint requests should be sent to Jacqueline C. Snow, Department of Psychology, University of Nevada, Reno, NV, or via e-mail: firstname.lastname@example.org.