Depth information is necessary for adjusting the hand to the three-dimensional (3-D) shape of an object to grasp it. The transformation of visual information into appropriate distal motor commands is critically dependent on the anterior intraparietal area (AIP) and the ventral premotor cortex (area F5), particularly the F5p sector. Recent studies have demonstrated that both AIP and the F5a sector of the ventral premotor cortex contain neurons that respond selectively to disparity-defined 3-D shape. To investigate the neural coding of 3-D shape and the behavioral role of 3-D shape-selective neurons in these two areas, we recorded single-cell activity in AIP and F5a during passive fixation of curved surfaces and during grasping of real-world objects. Similar to those in AIP, F5a neurons were either first- or second-order disparity selective, frequently showed selectivity for discrete approximations of smoothly curved surfaces that contained disparity discontinuities, and exhibited mostly monotonic tuning for the degree of disparity variation. Furthermore, in both areas, 3-D shape-selective neurons were colocalized with neurons that were active during grasping of real-world objects. Thus, area AIP and F5a contain highly similar representations of 3-D shape, which is consistent with the proposed transfer of object information from AIP to the motor system through the ventral premotor cortex.
Binocular disparity based on differences in the horizontal positions of retinal images provides an important cue for three-dimensional (3-D) object recognition and manipulation. Indeed, binocular disparity information is used for object manipulation while the hand is preshaping to adapt to the intrinsic 3-D properties of an object (Watt & Bradshaw, 2003).
The anterior intraparietal area (AIP) and the ventral premotor cortex are considered critical nodes in the visual control of grasping in nonhuman primates based on single-cell, reversible inactivation and functional imaging results (Nelissen & Vanduffel, 2011; Fluet, Baumann, & Scherberger, 2010; Fogassi et al., 2001; Sakata, Taira, Kusunoki, Murata, & Tanaka, 1997; Jeannerod, Arbib, Rizzolatti, & Sakata, 1995; Gallese, Murata, Kaseda, Niki, & Sakata, 1994; Sakata & Kusunoki, 1992; Taira, Mine, Georgopoulos, Murata, & Sakata, 1990; Rizzolatti et al., 1988). Human imaging studies have reported grasp-related activations in the human AIP and the ventral premotor cortex (Jacobs, Danielmeier, & Frey, 2010; Binkofski et al., 1998; Faillenot, Toni, Decety, Gregoire, & Jeannerod, 1997). In line with early behavioral work on grasping (Jeannerod, 1981), a recent human fMRI study demonstrated activations in the human AIP and ventral premotor cortex elicited by intrinsic rather than extrinsic object properties (Cavina-Pratesi et al., 2010).
We previously demonstrated that neurons in area AIP and area F5a, which is adjacent to F5p, provide a robust representation of 3-D shape (Theys, Pani, van Loon, Goffin, & Janssen, 2012; Srivastava, Orban, De Maziere, & Janssen, 2009). The presence of 3-D shape-selective neurons in F5a does not indicate how these neurons represent disparity-defined 3-D shapes, that is, which aspects of the stimuli are primarily encoded by the neurons. Previous studies have shown that the coding of 3-D shape in area AIP is fast (i.e., short latency), metric (i.e., largely monotonic tuning for the degree of the disparity variation in curved surfaces), and coarse (i.e., selectivity is retained for discrete approximations; Srivastava et al., 2009). In contrast, 3-D shape coding in the inferior temporal cortex (ITC) is slower (longer latencies), more categorical, and highly sensitive to disparity discontinuities such as sharp edges and disparity steps (Janssen, Vogels, & Orban, 2000b). Similar coding differences between dorsal and ventral stream areas have been suggested by human fMRI studies (Preston, Li, Kourtzi, & Welchman, 2008). These differences in the neural representation of 3-D shape between ITC and AIP are consistent with lesion studies in humans (Goodale & Milner, 1992), which have proposed a dichotomy in the primate visual system between the ventral stream supporting perception/object recognition and the dorsal stream supporting action (Marotta, Behrmann, & Goodale, 1997; Dijkerman, Milner, & Carey, 1996): The very detailed representation of 3-D shape in ITC is suitable for object recognition and categorization, whereas the coarser representation of 3-D shape in AIP is suitable for grasping.
We have also recently shown that the 3-D shape representation in AIP is more boundary-based (i.e., weak influence of surface dots and robust selectivity for stimuli with disparity varying along the boundaries of the shape) compared with that in ITC (Theys, Srivastava, van Loon, Goffin, & Janssen, 2012). However, it is unknown whether the 3-D shape representation in F5a is more similar to the one in AIP or that in the ITC. Given that F5a is strongly connected to AIP but not directly with ITC (Gerbella, Belmalih, Borra, Rozzi, & Luppino, 2011; Matelli, Camarda, Glickstein, & Rizzolatti, 1986), we hypothesized that the neural coding of 3-D shape in F5a should be more similar to that in AIP than in ITC.
Recent studies have also begun to investigate the behavioral role of 3-D shape-selective neurons in ITC and F5a. Consistent with the properties of individual ITC neurons (Janssen et al., 2000b), electrical microstimulation of ITC clusters during 3-D shape categorization exerts a profound and predictable influence on both perceptual choices and RTs (Verhoef, Vogels, & Janssen, 2012). In the F5a sector of ventral premotor cortex, in contrast, most 3-D shape-selective neurons were also active during visually guided grasping of real-world objects (Theys, Pani, et al., 2012), suggesting a close relationship between 3-D shape selectivity and visuomotor transformations for grasping. However, no data exist concerning the possible behavioral role of 3-D shape-selective sites in AIP.
First-order disparity coding has been demonstrated in areas V4 (Hinkle & Connor, 2002) and ITC (Janssen, Vogels, & Orban, 1999) in the ventral stream, as well as in areas MT/V5 (Nguyenkim & DeAngelis, 2003), CIP (Katsuyama et al., 2010; Taira, Tsutsui, Jiang, Yara, & Sakata, 2000; Shikata, Tanaka, Nakamura, Taira, & Sakata, 1996), and AIP (Srivastava et al., 2009) in the dorsal stream. Second-order disparity selectivity has been demonstrated in ITC (Janssen et al., 2000b) and AIP (Srivastava et al., 2009). The first goal of this study was to investigate the neural representation of 3-D shape in F5a to compare this representation with that of AIP. We tested whether F5a neurons respond to first-order (as in tilted planar surfaces) or second-order disparity (curved surfaces). We also tested the coarseness of the 3-D shape coding in F5a by comparing the selectivity for smoothly curved surfaces to that for discrete and linear approximations of these surfaces containing disparity discontinuities and by examining the sensitivity of F5a neurons for small differences in the degree of the disparity variation. Finally, to complement our previous study of 3-D shape selectivity and grasping activity in F5a (Theys, Pani, et al., 2012), we also recorded from 3-D shape-selective sites in AIP during visually guided grasping and trained both animals in a perceptual 3-D shape categorization task to be able to relate interindividual differences in the neural representations of 3-D shape to behavioral differences in discrimination performance. To allow a direct comparison between F5a and area AIP, we recorded in both areas (in counterbalanced order) in the same animals using the same stimulus sets and tasks. We found that the 3-D shape representation in F5a was highly similar to that in AIP. Conversely, 3-D shape-selective sites in AIP were also frequently active during object grasping, as in F5a. Thus, the parietofrontal grasping circuit contains two almost identical representations of the depth structure of objects that are both closely related to the visual guidance of the hand during grasping.
Subjects, Surgery, and Recording Procedure
Two rhesus monkeys served as subjects for extracellular microelectrode recordings. The surgical and recording procedures were identical to previous experiments in ITC and AIP (Srivastava et al., 2009; Janssen, Vogels, & Orban, 2000a; Janssen et al., 1999, 2000b). Under isoflurane anesthesia, an MRI-compatible head fixation post and recording wells were implanted. Implantation of the recording chambers was centered on the inferior limb of the arcuate sulcus (area F5) and the lateral bank of the AIP sulcus stereotactically guided using preoperative MRI. All surgical techniques and veterinary care were performed in accordance with the NIH Guide for Care and Use of Laboratory Animals and approved by the local ethical committee of the KU Leuven. The animals were trained to perform a passive fixation task keeping the gaze of both eyes inside a 1-degree fixation window. After a 400-msec fixation period, the stimulus was presented at the fixation point for 600 msec, and if fixation had been maintained, a drop of juice was given as a reward. Stimuli were presented dichoptically using ferroelectric liquid crystal shutters (Displaytech) operating at a frequency of 60 Hz each and synchronized with the vertical retrace of the display monitor (VRG) operating at 120 Hz. As previously reported (Srivastava et al., 2009), no crosstalk was measured between the images presented to the two eyes. Horizontal and vertical eye movements were recorded using an infrared-based camera system sampling at 500 Hz (EyeLink II; SR Research).
Stimuli and Tests
The basic stimulus set used in the search test was identical to the one used in previous experiments (Srivastava et al., 2009; Janssen et al., 2000b) and consisted of disparity-defined 3-D shapes generated by combining a two-dimensional contour and a depth profile. We used thirty-two 3-D shapes portraying curved surfaces, in which four disparity profiles were imposed over eight 2-D shapes filled with a 50% density random-dot pattern (stimulus size: 5.5 deg). For all subsequent tests, stimuli with opposite curvatures were created by interchanging the monocular images between eyes (concave surfaces become convex and vice versa). All neurons were therefore tested with pairs of 3-D shapes (concave–convex) composed of the same monocular images, but with right and left images interchanged.
Recordings in area AIP and F5a were performed according to previous experiments (Theys, Pani, et al., 2012; Verhoef, Vogels, & Janssen, 2010; Srivastava et al., 2009). In the search test—which was identical to that used in our previous studies—we presented the basic stimulus set consisting of 32 stimuli (eight 2-D contours combined with four depth profiles, degree of the disparity variation, i.e., the disparity difference between the center and the edge of the shape: 0.65 deg) at the fixation point during passive fixation. On the basis of responses obtained in this test, the optimal 2-D contour was selected for all subsequent tests. As in our previous studies, we first verified that the selectivity for 3-D shape could not be accounted for by a selectivity for the monocular images per se (disparity test, see Theys, Pani, et al., 2012): If the response difference between the two members of a pair of 3-D shapes (composed of the same monocular images) was significant (t test p < .05) and at least three times greater than the difference in the sum of the monocular responses (i.e., the difference in the summed responses to the monocular presentations of each of the two members of a pair of 3-D shapes), the neuron was judged to be disparity selective. When stereo selectivity was present, we assessed that selectivity for higher-order disparity in the position-in-depth test (for further details, see Janssen et al., 2000b) in which concave and convex 3-D stimuli were presented at five positions in depth, ranging from −0.50° (near) to +0.50° (far).
If a neuron in area F5a was judged to be higher-order disparity selective, the cell was further tested in the disparity order test to assess its selectivity for first- and second-order disparity using various approximations of the original 3-D shapes. This stimulus set has been described in Janssen et al. (2000b) and Srivastava et al. (2009). Briefly, the first-order stimulus approximations consisted of planar surfaces inclined in depth (linear variation in disparity over the vertical axis of the shape) to approximate either the top or the bottom part of the original 3-D shape (Figure 1B). The linear approximations of the concave and convex 3-D shape pairs were derived from the original 3-D shapes as least squares approximations of disparity profiles in the original surfaces and consisted of two tilted planes (linear disparity variations) with a sharp disparity discontinuity in the center of the stimulus. For the inclined 3-D shape pair (see sample neuron in Figure 2), the linear approximation was a single inclined surface and therefore identical to the first-order approximation. We tested linear approximations only for the cosine and Gaussian depth profiles, as in Srivastava et al. (2009). Three different discrete approximations were constructed by dividing the stimulus into three parts with a central region of varying size. These three parts were presented at three positions in depth (disparity amplitude, 0.65°).
In the disparity sensitivity test, the optimal 2-D contour was presented with two depth profiles (e.g., concave and convex) and six degrees of disparity variation (i.e., the disparity difference between the center and the edge of the shape) ranging from 1.3 to 0.03 deg. Because our goal was to compare the representation of 3-D shape in F5a with that in AIP, all stimuli were presented in the center of the display at the fixation point and in the fixation plane. M2 was also used in the AIP study of Srivastava et al. (2009). To allow direct comparison between AIP and F5a, we also recorded from higher-order disparity-selective neurons in the sensitivity test in area AIP of monkey M1.
To investigate the selectivity of F5a neurons for 3-D shapes in which the disparity variation was confined to the boundary or the surface of the stimulus, we presented the same stimuli as in previous studies (Theys, Srivastava, et al., 2012; Janssen, Vogels, Liu, & Orban, 2001). Different Gaussian depth profiles were created by varying the disparity along the surface of the stimulus, along the boundary, or along both surface and boundary (boundary–surface test). In the surface stimuli, the depth profile consisted of a 2-D Gaussian or a 13° × 13° square with the maximum disparity in the center of the shape smoothly approaching zero toward the boundaries.
A subset of 3-D shape-selective neurons was tested in a visually guided grasping task (described in Theys, Pani, et al., 2012). In this task, the monkey was seated in the dark and had to place its right hand in a resting position for 500 msec, after which an LED was illuminated at the bottom of an object. Fixation upon the LED (keeping the gaze inside a 2.5-degree fixation window throughout the trial until the object was lifted) for 500 msec was followed by the illumination of a 3-D object. One of six different objects (a small cylinder, a small cube, a large cylinder, a large sphere, a large cube, and a cylinder with groove) was pseudorandomly presented on a custom-built, vertically rotating carousel at a viewing distance of 28 cm. The dimensions (width, length, and height) of the small and large objects were 15 and 35 mm, respectively. After a variable period of 500–1000 msec of object fixation, an auditory GO cue instructed the monkey to reach, grasp, and lift the object. After a correctly completed trial, a juice reward was given. The resting position and the lifting phase were monitored by fiber-optic cables. Eye movements were recorded using an infrared-sensitive camera system (EyeLink II; SR Research). Eye position signals, neural activity, and photocell pulses were digitized and processed with a digital signal processor (DSP) at 20 kHz (C6000 series; Texas Instruments, Dallas, TX).
For extracellular recordings, tungsten microelectrodes (FHC, MicroProbes) were inserted through a guide tube placed in a standard grid (Crist Instruments). MRI (resolution 0.6 mm isotropic) using glass capillaries filled with a 1% copper sulfate solution inserted into key grid positions confirmed correct positioning of the electrode in the depth of the posterior bank of the inferior limb of the arcuate sulcus (area F5a) and in the lateral bank of the AIP sulcus (area AIP). For area F5a, we reconstructed the electrode penetrations based on the anatomical MRI using BrainVISA (brainvisa.info). Because of the inclination of the recording cylinder, we first traversed area 45B on the anterior bank of the inferior arcuate sulcus, then passed through the sulcus, and finally entered the posterior bank of the inferior arcuate sulcus near the fundus. The patterns of active and silent zones during the recordings were highly consistent with the reconstructed electrode paths. The center of our recording area was located within the fMRI activation elicited by curved surfaces reported by Joly, Vanduffel, and Orban (2009). The recording positions in F5a were identical to the ones in our previous study (Theys, Pani, et al., 2012); microstimulation in these recording positions did not elicit any overt behavioral response. The AIP recordings were performed in posterior AIP, in a region where Sakata et al. reported hand manipulation neurons (Sakata, Taira, Murata, & Mine, 1995).
Matlab (Mathworks, Natick, MA) was used for data analysis. Net neural responses were calculated by subtracting the mean activity in the 400-msec interval preceding stimulus onset from the mean activity between 50- and 450-msec after stimulus onset. In the position-in-depth test, neurons were considered responsive to the spatial variation of disparity (i.e., higher-order disparity selective) if the response to the nonpreferred shape did not significantly (Student's t test) exceed any response to the preferred shape at any position in depth (Janssen et al., 2000b).
In the disparity-order test, neurons were classified as first order if the selectivity for the first-order stimuli was not significantly smaller than the selectivity for the original smoothly curved surfaces, as evidenced by a nonsignificant interaction between 3-D structure (concave–convex) and stimulus type (original vs. first-order stimulus, ANOVA p > .05). Second-order neurons were significantly more selective for the smoothly curved surfaces compared with the first-order approximations (ANOVA p < .05). Discrete neurons showed a significant (t test, p < .05) selectivity for at least one of the discrete approximations.
Depending on the tuning for 3-D shape as determined in the sensitivity test, neurons were classified as monotonic, broadband, or tuned as in Janssen et al. (2000b). Monotonic neurons showed the strongest response to the largest disparity variation or a response statistically indistinguishable (t test) from the response to the second-largest disparity variation and a decline in response for smaller disparities. Broadband neurons responded equally well to all disparity variations (no significant effect of degree of disparity variation in a one-way ANOVA of the responses to the six disparity variations of the preferred 3-D shape). If an optimal disparity variation was found, with a significant decrease in response on either side of this optimal magnitude, the neuron was classified as tuned.
In the grasping task, neural activity (MUA) was aligned to the onset of the illumination above the object, to the time of the go signal, and to the time of the object lift.
All F5a neurons reported here (n = 131) were higher-order disparity selective, that is, showed significant selectivity in the disparity test (consisting of presentations of concave and convex surfaces and monocular presentations) that could not be accounted for by the monocular responses and preserved this selectivity across positions-in-depth. Binocular eye position traces recorded during the position-in-depth test showed only marginal deviations (averaging 0.1 degree of vergence) between the nearest and the farthest position in depth, much smaller than the range of disparities in the position-in-depth test (1 deg). Although visual responses could be recorded over an extended range in and around the arcuate sulcus, higher-order disparity selective neurons were concentrated in area F5a, in line with previous fMRI results (Joly et al., 2009).
Disparity Order Test
To determine whether F5a neurons encode first- or second-order disparities, we tested 71 F5a (M1: n = 35; M2: n = 36) neurons in the disparity-order test, in which the original 3-D shapes were presented together with various approximations of these stimuli. The sample neuron in Figure 2A was highly selective for the inclined depth profile (first column) but was equally selective for a simple first-order approximation of the original 3-D shape pair (second column, ANOVA with factors 3-D Profile and Stimulus Type, interaction ns). This sample neuron was therefore encoding primarily first-order variations in disparity, as present in tilted planes. Furthermore, the discrete approximations (three rightmost columns) also elicited significant selectivity (t tests on the net responses, p < .01 for all three approximations), indicating that the coding of 3-D shape was relatively coarse.
Across our population of F5a neurons, 55% (39/71; M1: n = 14/35; M2: n = 25/36) showed selectivity for first-order stimuli equal to or greater than that for second-order stimuli and were therefore considered first-order neurons, in line with previous studies (Srivastava et al., 2009; Janssen et al., 2000b). The average responses in the disparity-order tests of these first-order neurons are illustrated in Figure 2B. Clearly both the first-order and the discrete approximations evoked strongly selective responses in this population of first-order neurons (t test on the net responses to the preferred and nonpreferred depth profiles, p < .01 for every approximation).
However, we also observed strong second-order selectivity in F5a, as illustrated by the sample neuron in Figure 3A. This neuron was strongly selective for convex versus concave depth profiles and for their linear approximations, but not for the first-order approximations (rightmost column; ANOVA with factors 3-D shape and stimulus type, interaction p < .05). Two of three discrete approximations also elicited significant selectivity (t test, p < .01), which again indicates that the representation of 3-D shape in F5a is relatively coarse.
Neurons showing significantly less selectivity for the first-order stimuli compared with the second-order stimuli (ANOVA, interaction between 3-D profile and stimulus type p < .05) were deemed to be second-order disparity selective. In the average response of our population of second-order disparity selective F5a neurons (n = 32/71, 45%; Figure 3B), no selectivity was present for the first-order stimuli. As in the sample neuron in Figure 3A, two of three discrete approximations yielded significant selectivity across the population (t test p < .05), and 19 neurons (59%) showed selectivity for at least one of the discrete approximations. Interestingly, the average response to the preferred linear approximation differed significantly from the response to the preferred smoothly curved 3-D shape (t test, p < .01; M1: p < .001; M2: p < .05), and 12 neurons discriminated reliably between the preferred smoothly curved 3-D shape and its linear approximation. Therefore, the neural coding of 3-D shape in F5a is, on the one hand, relatively coarse (in view of the frequent selectivity for discrete approximations), but at the same time sensitive enough to signal the small differences in 3-D profile between the smoothly curved surface and its linear approximation, which constitutes a least-squares approximation of the smoothly curved surface. Note that we also encountered 17 neurons that were selective for the full-period sine depth profile, consisting of combined convex and concave profiles (see Figure 2 in Theys, Pani, et al., 2012) and that therefore represents a third-order disparity stimulus. However, we did not perform the disparity-order test for these neurons.
To illustrate the responses and the neural selectivity for first- and second-order stimuli, we plotted the average net responses and response differences for the original preferred 3-D shape against the responses and response differences for the first-order stimuli (Figure 4A), for the first-order and second-order neurons independently. Although second-order neurons frequently fired strongly to first-order stimuli (Figure 4A, left), the response differences were much smaller for first-order stimuli than for second-order stimuli (Figure 4A, right). Similarly, discrete approximations can evoke strong responses in second-order F5a neurons (Figure 4B, left), but the degree of selectivity was weaker for these discrete approximations than for the smoothly curved 3-D surfaces (Figure 4B, right). Finally and in contrast to area AIP, a substantial proportion of F5a neurons (38%) discriminated reliably between a smoothly curved surface and its linear, least-square approximation (see data points below the diagonal in Figure 4C).
Overall, F5a neurons can encode zero-, first-, second-, and possibly third-order disparities, similar to those in AIP and ITC. The neural coding of 3-D shape in F5a shares many features with that in area AIP (e.g., selectivity for discrete approximations), but a subset of the neurons in F5a appear to be more sensitive to subtle differences in the depth profile compared with AIP neurons.
Sensitivity to the Degree of Disparity Variation
To investigate the precision of the 3-D shape representation in area F5a, we measured the sensitivity of F5a neurons to differences in the degree of the disparity variation within the stimulus (an amplitude of 1.3 deg corresponded to a highly curved surface, whereas an amplitude of 0.03 deg was almost flat). In the disparity sensitivity test, 131 neurons (M1: 61; M2: 70) were tested with disparity variations in the stimulus ranging from 1.3 to 0.03 deg. Figure 5 shows the responses of three sample neurons (positive numbers on the x axis indicate the preferred depth profile). The most frequent response pattern in both monkeys (72%; M1: 79%, M2: 66%; z test, ns; Table 1) was a monotonic profile, with the maximal response to the largest disparity variation and a monotonic decline in the response for smaller disparity variations (green line in Figure 5). A small proportion of the neurons (8%; M1: 13%, M2: 4%; z test, ns; Table 1) were significantly tuned to a particular disparity variation: The maximal response was observed for one of the smaller disparity variations, and this response differed significantly (t test p < .05) from the response to the largest disparity variation (blue trace in Figure 5). Finally, 20% of the neurons (M1: 8%; M2: 30%; Table 1) were broadband (red trace in Figure 5) because no significant difference was observed between responses in the preferred range of disparity variations (ANOVA, ns). The proportion of broadband neurons was significantly greater in M2 than in M1 (z test, p < .05). However, the strongest decline in the response was more frequently (38% of the broadband neurons) seen within the nonpreferred range of disparity variations (as illustrated by the sample neuron in Figure 5) than at the change in the sign of the curvature (11%), in contrast to what has been reported in ITC (Janssen et al., 2000b).
M1 = monkey 1; M2 = monkey 2.
Although most F5a neurons showed a monotonic response pattern in the sensitivity test, a fraction of these neurons displayed significant selectivity for small differences in the depth profiles of the stimuli: more than 20% of F5a neurons (28% in M1 and 23% in M2) were significantly selective for the smallest differences in the depth profile (+0.03 vs. −0.03 deg, t test p < .05) and 18 of 131 neurons tested (15%, M1: 18%, M2: 11%) showed the largest response difference at the change in the sign of the disparity curvature (i.e., between the two smallest disparity variations, −0.03 vs. +0.03 deg). Our population of 131 F5a neurons reliably discriminated between the two smallest disparity variations (paired t test of the average responses to the +0.03 and −0.03 deg disparity variations, p < .0001 for both monkeys combined; M1: p < .001; M2: p = .12). Thus, F5a neurons can signal very small differences in the depth profiles of curved surfaces.
The average normalized responses in the disparity sensitivity test are illustrated in Figure 6A for both monkeys independently. In both monkeys, the F5a population showed a largely monotonic response pattern, but in M1 a more pronounced drop in the response was present between the two smallest disparity variations (t test comparing the difference in the normalized response differences between the −0.03 and +0.03 deg disparity variations between the two monkeys, p < .001). To directly compare the representation of 3-D shape in F5a with that in AIP, we also recorded the responses of 71 higher-order AIP neurons in the same animals (n = 38 in M1, n = 33 in M2). The average normalized responses of AIP neurons are illustrated in Figure 6B. Overall the average response in AIP was highly similar to that of F5a. Furthermore, the proportions of monotonic, broadband, and tuned neurons were highly similar in AIP and F5a in both animals: also, monotonic neurons predominated in AIP (M1: 74%; M2: 85%), whereas tuned (M1: 10%; M2: 9%) and broadband (M1: 16%; M2: 6%) neurons represented much smaller fractions of the neurons. In both monkeys, the normalized difference between responses to the −0.03 and +0.03 deg disparity variations did not significantly differ in the two areas (t test, M1: p = .46; M2: p = .82). Therefore, the results of the sensitivity test indicate that the neural representation of 3-D shape in F5a was highly similar to that in AIP.
To determine whether the difference between the neural representations of 3-D shape in our two monkeys was related to interindividual differences in the quality of stereoscopic vision, we trained both animals in a 3-D shape discrimination task (Verhoef et al., 2010, 2012; Verhoef, Vogels, & Janssen, 2011). In this task, either a convex or concave 3-D surface (disparity varied along both the vertical and the horizontal axis, no disparity on the boundaries; Theys, Srivastava, et al., 2012) was presented at the fixation point, and the animal was required to make an eye movement to the left when the stimulus was concave and to the right when the stimulus was convex. The disparity coherence was always 100%. After six training sessions, M1 reached a performance level of 82% correct, whereas M2 still performed at chance level (50% correct) after 12 training sessions. The chance performance of M2 was not because of an inability to learn the task rule, because this animal had learned a simple shape discrimination task (saccade to the left for a square and to the right for a triangle) in five sessions (92% correct). Given the presence of large numbers of disparity-selective neurons in F5a and AIP (Srivastava et al., 2009) in M2, it is unlikely that this animal was stereoblind, but behavioral testing indicated that the quality of its stereoscopic perception was in all likelihood weaker than that of M1. Therefore, the differences we observed between our two animals in the disparity sensitivity test were likely related to differences in stereoscopic perception.
Selectivity for Surfaces and Boundaries in Depth
We previously demonstrated that 3-D shape-selective AIP neurons encode both disparity variations along the boundary and along the surface of the shape and that for the great majority of AIP neurons, boundaries in depth (lacking 3-D surface information) are sufficient for evoking selectivity (Theys, Srivastava, et al., 2012). We tested 18 higher-order F5a neurons with the same stimuli as in our previous study (Theys, Srivastava, et al., 2012): concave and convex curved surfaces with a disparity variation on both the surface and the boundary of the shape (vertical 3-D shape), on the boundary of the shape but not on the surface (silhouettes and outline stimuli), and on the surface but not on the boundary (restricted and large 3-D surfaces). In a manner very similar to those of AIP, most (78%) F5a neurons were selective for at least one of the curved boundaries, whereas a smaller proportion (55%) was selective for the 3-D surface stimuli (data not shown). Although the low number of neurons precludes a detailed comparison between AIP and F5a in terms of 3-D boundary selectivity, the proportions of neurons in F5a were highly comparable to those previously reported for AIP (67% boundary neurons, 53% surface neurons).
3-D Shape Selectivity and Grasping Activity
As a final comparison of 3-D shape-selective sites in F5a and AIP, we recorded multiunit activity (MUA) in 3-D shape-selective AIP sites during delayed visually guided object grasping after assessing higher-order disparity selectivity in the position-in-depth test, as in our previous study of F5a (Theys, Pani, et al., 2012). We found that the great majority of the 3-D shape-selective sites (13/16, 81%) also responded during the fixation and grasping of real-world objects. Because our AIP recordings consisted of MUA, we cannot infer that the same AIP neurons were both 3-D shape selective and active during grasping. Furthermore, we did not test whether 3-D shape-selective AIP sites remained active during grasping in the dark (visuomotor activity), as in F5a. However, at the very least, these data demonstrate that 3-D shape selectivity was colocalized with grasping activity in AIP, as it is in F5a (Theys, Pani, et al., 2012).
In Figure 7, the average population response during visually guided grasping is plotted as a function of time for area AIP (top ). For comparison, we also plotted the average grasping-related activity of 98 higher-order disparity selective MUA sites in F5a (bottom). The visual response in the first 100 msec after light onset appeared stronger and faster in AIP in comparison with F5a (Figure 7, left), but because the latency of the population response strongly depends on the number of recording sites, we cannot directly compare the two areas in this respect. At the moment of object lift ([−100, 200 msec] around object lift) the average activity in AIP declined and the average F5a activity became stronger than in AIP. A mixed-design ANOVA with brain area (AIP–F5a) and epoch ([−100, 200 msec] around object lift and [−500, −200 msec] before object lift) as independent factors revealed a significant interaction, F(1, 108) = 8.18, p = .005. Bonferroni post hoc analysis showed a significant difference (p < .002) between neural activity in the [−100, 200 msec] epoch around the lift and that in the epoch [−500, −200 msec] before the lift (respectively 39.9 ± 8.3 spikes/sec vs. 9.46 ± 7.6 spikes/sec) for AIP, whereas no difference was observed between these epochs for F5a (24.3 [± 3.3] vs. 18.75 [± 3.0]). Although the low number of AIP sites warrants a degree of caution in interpreting these results, they may suggest that 3-D shape-selective AIP sites are most active during the visual analysis of the object, whereas 3-D shape-selective F5a sites are more strongly active during the execution of the grasping movement. Future studies will have to determine to what extent 3-D shape-selective AIP neurons remain active during grasping in the dark, that is, exhibit visuomotor or motor-dominant activity.
We investigated the coding of 3-D shape-selective neurons in area F5a and compared their properties with AIP neurons. We found that F5a neurons could be either first-order or second-order disparity selective. Furthermore, 3-D shape coding in area F5a was fast, robust, largely metric, and coarse, similar to area AIP. In both areas, 3-D shape-selective neurons were embedded in clusters of neurons that also fired during grasping. The coding of 3-D shape information in AIP and F5a is most likely important for translating 3-D object properties into the appropriate motor commands for grasping.
This study is the first detailed comparison between the object representation in AIP and F5a. It is remarkable that neurons in the ventral premotor cortex provide exceedingly detailed visual information about the 3-D-structure of objects (e.g., the selectivity for very small disparity variations and the differences between smoothly curved surfaces and their linear approximation). Our data also demonstrate that at least some F5a neurons encode not only relative disparity (i.e., the disparity difference between the center and the edge of the shape) but also first- and second-order disparities (curvature). Hence, the motor system has access to a robust visual 3-D description of objects, although this information may not necessarily determine the grip type. Murata, Gallese, Luppino, Kaseda, and Sakata (2000) and Raos, Umilta, Murata, Fogassi, and Gallese (2006) compared the object representations in AIP and F5 using multidimensional scaling and cluster analysis and concluded that AIP furnishes a visual object description whereas F5 represents objects in motor terms (i.e., determined by the grip type used to grasp the object). However, the study of Raos et al. (2006) focused on the F5p sector in ventral premotor cortex and most likely did not include F5a neurons.
It is noteworthy that the 3-D shape stimuli we used did not resemble the objects that the monkeys had to grasp; nevertheless, the same ensembles of neurons were active in F5a and AIP during 3-D shape presentation and during object grasping. At least part of this overlap in neural preference for these seemingly disparate stimulus classes may arise from the relatively broad tuning for objects in AIP and F5a, because most neurons in these areas respond to many objects (Pani et al., unpublished observations). Moreover, in addition to the 3-D profile, AIP and F5a neurons also encode the 2-D contour of objects, which may be based on relatively simple shape features that are shared between many objects. Also in the ITC, it is frequently difficult to identify a single object or shape feature that activates the neuron, and many neurons appear to respond to seemingly unrelated shape features, even in the 3-D domain (Janssen et al., 2001).
We have previously investigated the neural coding of 3-D shape in the ITC and in AIP using the same stimuli (Srivastava et al., 2009; Janssen et al., 2000b). The neural representation of 3-D shape information was very similar in areas F5a and AIP. The premotor area F5a contained many neurons for which a discrete approximation was sufficient to evoke selectivity and was similar to area AIP in this regard (Srivastava et al., 2009). The sensitivity to the degree of the disparity variation with an overall monotonic coding and the selectivity for minute disparity differences in a small fraction of the neurons was also very similar to AIP. However, we observed a tendency toward a more elaborate and detailed representation in area F5a because, in both monkeys, neurons were signaling subtle distinctions in depth profiles such as the difference between a linear approximation (a wedge-shaped stimulus) and a smoothly curved 3-D shape, which was not observed in AIP (Srivastava et al., 2009). This refinement suggests that additional processing occurs between the output of AIP and the output of F5a before visual information connects to the motor system. Finally, in both F5a and AIP, 3-D shape-selective neurons were embedded in clusters of neurons that were also active during visually guided grasping. The latter observation highlights the probable behavioral role of 3-D shape-selective neurons in the parietal and premotor cortex, that is, to provide a visual 3-D description of objects for the purpose of programming the grasp. Consistent with this idea, reversible inactivation of 3-D shape-selective sites in AIP causes a marked deficit in grasping but not in the perceptual discrimination of 3-D-structure (Verhoef, Vogels, and Janssen, unpublished observations).
However, we also observed similarities between ITC, AIP, and F5a in 3-D shape coding. We observed zero-order (position in depth), first-order (tilt/slant), and second-order disparity selectivity (curvature) in all three areas. In contrast, earlier visual areas such as V4 contain zero- and first-order neurons (Hinkle & Connor, 2002) but no second-order neurons (Hegde & Van Essen, 2005). Hence, the properties of neurons in lower-tier areas appear to be reiterated at the highest levels in dorsal, ventral, and premotor areas. Furthermore, in a manner similar to neurons in area AIP (Theys, Srivastava, et al., 2012) and area IT (Janssen et al., 2001) F5a cells showed selectivity for 3-D surfaces and boundaries.
The anatomical connectivity between areas AIP and F5a (Gerbella et al., 2011), the response latency difference between AIP and F5a (approximately 10 msec; Theys, Pani, et al., 2012), and the strong resemblance in functional properties (fast and coarse coding, grasping-related activity) suggest a hierarchical parietofrontal 3-D processing network for the control of grasping, distinct from the ventral stream 3-D shape representation in IT. Whether the selectivity in area F5a depends exclusively on input from area AIP needs further confirmation through combined recording and inactivation experiments. The current physiological and anatomical evidence suggests that AIP input could be processed in area F5a and translated into a more motoric code for the F5p neurons, which project to M1 and the spinal cord (Gerbella et al., 2011). Consistent with this hypothesis, virtual lesions of the human AIP using theta-burst TMS disrupt the normal PMv–M1 interactions during grasp preparation (Davare, Rothwell, & Lemon, 2010), suggesting that these PMv–M1 interactions depend on the object information provided by AIP to the premotor cortex.
Human fMRI studies indicate that part of the PMv is also activated more strongly by curved surfaces than by flat surfaces at different positions in depth (Georgieva, Peeters, Kolster, Todd, & Orban, 2009), which could be homologous to the F5a region of the monkey. On the other hand, the homology between the human AIP (hAIP; Begliomini, Wall, Smith, & Castiello, 2007; Cavina-Pratesi, Goodale, & Culham, 2007; Frey, Vinton, Norlund, & Grafton, 2005; Culham et al., 2003; Culham & Kanwisher, 2001) and the monkey AIP may be more questionable. We recorded in the posterior part of AIP (Srivastava et al., 2009), in which strong visual (3-D shape selective) and grasping responses can be measured but no somatosensory responses are found (Pani, Theys, and Janssen, unpublished observations). A range of fMRI studies in humans and monkeys (reviewed in Orban, 2011) suggest that posterior AIP in the monkey may correspond more to the DIPSA region in the human, which is located in the IPS posterior to the hAIP and is also activated by 3-D shape defined by disparity (Durand, Peeters, Norman, Todd, & Orban, 2009). The hAIP, in contrast, is more activated during grasping than during reaching (Culham et al., 2003), responds even to somatosensory stimulation (Bodegard, Geyer, Grefkes, Zilles, & Roland, 2001), but is not (or only weakly) activated by disparity-defined 3-D shape (Durand et al., 2009), similar to the more anterior part of AIP. (Note that DIPSA may also be activated during grasping, but not more than during reaching because of the strong visual responses in this region.) Furthermore, Culham et al. (2003) reported that the hAIP is not activated by images of objects, whereas we observed strong and selective responses to images of objects in the macaque AIP (Romero, Van Dromme, & Janssen, 2012). These apparently conflicting results between human fMRI and single-cell studies may have been caused by the homology between the macaque posterior AIP and a more posterior region in the human IPS (DIPSA) resulting from the expansion of the human IPS areas compared with the monkey. A similar reasoning may apply for the homology between the human lateral occipital complex (LOC; Grill-Spector, Kourtzi, & Kanwisher, 2001; Malach et al., 1995) and monkey ITC.
The question remains as to why two separate but very similar visual representations of 3-D shape should exist in both AIP and F5a for the control of grasping. One explanation could involve differences in neural properties that we did not test for, such as different receptive field sizes. Another possibility is that F5a neurons could respond to larger object parts or more complex shape features than AIP neurons, comparable with the V4– TEO–TE hierarchy in the ventral stream (Kourtzi & Connor, 2011). The observation that F5a neurons, but not AIP neurons, signal the difference between a smoothly curved 3-D shape and its linear approximation suggests that the 3-D shape representation in F5a may be more refined than in AIP and that additional processing is required before connecting to the motor system. Extensive receptive field mapping and a systematic variation of shape features to determine their critical features in each of these two areas could answer such questions. Multiunit recordings in F5a revealed that 3-D shape-selective visual-dominant neurons are colocalized with visuomotor neurons that were active during grasping in the light and in the dark (Theys, Pani, et al., 2012). Furthermore F5p/c neurons showed sluggish and nonselective responses to our 3-D shape stimuli (Theys, Pani, et al., 2012), and previous studies have demonstrated that objects are encoded in motor terms rather than in visual terms in F5p (Raos et al., 2006). Therefore, we hypothesize that the main reason for an intermediate area (F5a) between AIP and F5p may be that the connection between visual information and motor commands occurs in these clusters of visual-dominant and visuomotor neurons in F5a, which then project to F5p. In this interpretation, the grasping-related activity we observed in 3-D shape-selective clusters of AIP would be entirely different in nature: motor-related grasping activity in AIP may arise as a corollary discharge from visuomotor and motor-dominant F5p neurons projecting back to AIP for on-line visual control (Rizzolatti & Luppino, 2001), a hypothesis that deserves experimental testing.
We also observed interindividual differences between the two animals in the disparity sensitivity test: the average F5a and AIP population response in M1 showed a larger decline at the point where the disparity curvature changes sign (between the 0.03 deg preferred amplitude and 0.03 deg nonpreferred amplitude) compared with monkey M2. The observed difference in neural sensitivities was most likely related to a difference in the quality of stereoscopic vision in the two animals, because M1 learned to discriminate disparity-defined curved surfaces in a limited number of training sessions whereas M2 did not. However, M2 was not stereoblind, because we recorded large numbers of disparity-selective neurons in this animal's AIP and F5a. Furthermore, we measured a normal stereo visually evoked potential (VEP) over V1 with disparity stimuli in this animal (Srivastava et al., 2009; Janssen, Vogels, & Orban, 1998). In humans also, a wide range of stereoscopic capacities and stereoanomalies can be observed (Howard & Rogers, 2002). Individual differences in neural sensitivity that we observed suggest that differences between cortical areas have to be interpreted cautiously if the data are not acquired in the same animals. However, our two monkeys were quite comparable in the proportions of zero-, first-, and second-order neurons, as well as in the proportion of neurons showing selectivity for at least one of the discrete stimuli. Furthermore, in both animals the average AIP population response in the sensitivity test was largely monotonic.
Previous monkey fMRI studies found stronger activations for curved surfaces compared with flat surfaces at different positions-in-depth, which were highly localized in F5a (Joly et al., 2009), but very extensive and comprising a large part of AIP and the anterior region of LIP (Durand et al., 2007) in the lateral bank of the IPS. However in our single-cell recording experiments, the extent of the recording area that contained 3-D shape-selective neurons was highly similar in AIP and F5a, matching closely the F5a activation described in Joly et al. (2009). Future studies will have to investigate the neural basis of the difference between F5a and AIP in the hemodynamic response to curved surfaces.
We thank Piet Kayenbergh, Gerrit Meulemans, Stijn Verstraeten, Marc Depaep, Wouter Depuydt, and Inez Puttemans for assistance and Steve Raiguel for comments on the manuscript. This work was supported by Geconcerteerde Onderzoeksacties (GOA 2005/18, 2010/19), Fonds voor Wetenschappelijk Onderzoek Vlaanderen grants (G.0495.05, G.0713.09), Excellentiefinanciering (EF05/014), Programmafinanciering (PFV/10/008), and ERC-StG-260607.
Reprint requests should be sent to Peter Janssen, Laboratorium voor Neuro- en Psychofysiologie, Leuven Medical School, Herestraat 49, bus 1021, B-3000 Leuven, Belgium, or via e-mail: firstname.lastname@example.org.