Abstract

The division of cortical visual processing into distinct dorsal and ventral streams is a key concept in primate neuroscience [Goodale, M. A., & Milner, A. D. Separate visual pathways for perception and action. Trends in Neurosciences, 15, 20–25, 1992; Steele, G., Weller, R., & Cusick, C. Cortical connections of the caudal subdivision of the dorsolateral area (V4) in monkeys. Journal of Comparative Neurology, 306, 495–520, 1991]. The ventral stream is usually characterized as a “What” pathway, whereas the dorsal stream is implied in mediating spatial perception (“Where”) and visually guided actions (“How”). A subpathway emerging from the dorsal stream and projecting to the medial-temporal lobe has been identified [Kravitz, D. J., Saleem, K. S., Baker, C. I., & Mishkin, M. A new neural framework for visuospatial processing. Nature Reviews Neuroscience, 12, 217–230, 2011; Cavada, C., & Goldman-Raiuc, P. S. Posterior parietal cortex in rhesus monkey: I. Parcellation of areas based on distinctive limbic and sensory cortico-cortical connections. Journal of Comparative Neurology, 287, 393–421, 1989]. The current article studies the coordination of visual information typically associated with the dorsal stream (“Where”), with planned movements, focusing on the temporal lobe. We recorded extracellular activity from 565 cells in the human medial-temporal and frontal lobes while 13 patients performed cued hand movements with visual feedback (visuomotor task), without feedback (motor task), or observed visual feedback without motor movement (visual-only task). We discovered two different neural populations in the human medial-temporal lobe. One consists of motor-like neurons representing hand position, speed or acceleration during the motor task but not during the visuomotor or visual tasks. The other is specific to the parahippocampal gyrus (an area known to process visual motion [Gur, M., & Snodderly, D. M. Direction selectivity in V1 of alert monkeys: Evidence for parallel pathways for motion processing. Journal of Physiology, 585, 383–400, 2007; Sato, N., & Nakamura, K. Visual response properties of neurons in the parahippocampal cortex of monkeys. Journal of Neurophysiology, 90, 876–886, 2003]) and encodes speed, acceleration, or direction of hand movements, but only during the visuomotor task: neither during visual-only nor during motor tasks. These findings suggest a functional basis for the anatomical subpathway between the dorsal stream and the medial-temporal lobe. Similar to the recent expansion of the motor control process into the sensory cortex [Matyas, F., Sreenivasan, V., Marbach, F., Wacongne, C., Barsy, B., Mateo, C., et al. Motor control by sensory cortex. Science, 330, 1240–1243, 2010], our findings render the human medial-temporal lobe an important junction in the process of planning and execution of motor acts whether internally or externally (visually) driven. Thus, the medial-temporal lobe might serve as an integration node between the two processing streams. Our findings thus shed new light on the brain mechanisms underlying visuomotor coordination which is a crucial capacity for everyday survival, whether it is identifying and picking up food, sliding a key into a lock, driving a vehicle, or escaping a predator.

INTRODUCTION

Visuomotor coordination is a skill employed in a wide range of activities, from the most fundamental, for example, identifying and picking up food, to the highly specialized, such as endoscopic operation or flying an airplane. These processes require the brain to process visual input information and transform it into an actual output motor plan, relying on high-level perception of the world being visualized. Visual perception has long been an interdisciplinary field integrating insights and data from neuroscience, psychology, and computation (Marr, 1982) and so is the field of motor perception and coordination (Klapp & Jagacinski, 2011; Carmeli, Bar-Yossef, Ariav, Levy, & Liebermann, 2008). In an early stage of cortical visual processing, in the primary visual cortex (V1), the information divides into two processing streams (Goodale & Milner, 1992; Steele, Weller, & Cusick, 1991). The dorsal stream courses through occipitoparietal cortex to the posterior half of the inferior parietal lobule (area PG) and from there to the dorsolateral PFC. The ventral stream, on the other hand, courses through the occipito-temporal cortex to area TE (rostral inferior temporal cortex) and continues into the ventrolateral PFC (Mishkin, Ungerleider, & Macko, 1983; Mishkin & Ungerleider, 1982; Ungerleider & Mishkin, 1982). Under this scheme, the medial-temporal lobe is considered an integral part of the ventral stream. Little evidence exists regarding the role of the medial-temporal lobe in visual guidance of movements and in visuomotor or motor tasks. The hippocampus has recently been described in the context of memory-guided automatic movements (Uemura, 2010) and is believed to participate in the control of the ventral-striatal loop (or limbic loop), which plays a role in the control of motor behavior (e.g., motor reactions to emotions; Fernández-Seara, Aznárez-Sanado, Mengual, Loayza, & Pastor, 2009; van Rooyen, Young, Larson, & Teskey, 2006; Duvernoy, Vannson, Cattin, & Naidich, 2005; Tesche & Karhu, 1999; Halgren, 1991; Miyashita, Rolls, Cahusac, Niki, & Feigenbaum, 1989). The entorhinal cortex represents self-position in the environment in the form of place cells (Howard & Natu, 2005) and grid cells (McNaughton, Battaglia, Jensen, Moser, & Moser, 2006) and velocity for path integration (i.e., inference of position by integration of velocity; Burak & Fiete, 2009; Blair, Gupta, & Zhang, 2008; Howard, Fotedar, Datey, & Hasselmo, 2005; Lengyel & Erdi, 2004). The parahippocampal gyrus is known to play a critical role in spatial navigation (Baumann, Chan, & Mattingley, 2010; Epstein, 2008; Ekstrom et al., 2003; Aguirre & D'Esposito, 1999; Maguire, Frackowiak, & Frith, 1997) and is also associated with certain types of motor output (Yan et al., 2006; Rocca et al., 2004), such as head movements (Robertson, Rolls, Georges-Francois, & Panzeri, 1999), oculomotor tasks (Nagel, Sprenger, Hohagen, Binkofski, & Lencer, 2008; Ploner et al., 2000), and imitation of meaningful actions (Rumiati et al., 2005). In addition to motor-related evidence, however, the parahippocampal gyrus is anatomically connected to several visual areas (e.g., V3A and dorsal V4; Blatt, Pandya, & Rosene, 2003; Felleman & Van Essen, 1991), is known to have receptive field characteristics (Wilson, Babb, Halgren, & Crandall, 1983), and is active during multiple aspects of visual information processing: response to views of familiar places (Ekstrom et al., 2003), encoding object location (Fischer, Spotswood, & Whitney, 2010; Sugiura, Shah, Zilles, & Fink, 2005; Maguire, Burgess, Donnett, O'Keefe, & Frith, 1998; Owen, Milner, Petrides, & Evans, 1996), spatial memory (Ploner et al., 2000), visual processing of moving objects (Gur & Snodderly, 2007; Sato & Nakamura, 2003), perception of biological motion (Rumiati et al., 2005; Ptito, Faubert, Gjedde, & Kupers, 2003), and even reading sentences describing motion (Wallentin, Ostergaarda, Lund, Ostergaard, & Roepstorff, 2005). Nevertheless, the transition from processing input information to generating an output motor plan has remained enigmatic. Our study has therefore focused on the representation of hand movements in the temporal lobe and their generation from visual information during visuomotor coordination.

METHODS

Patients and Electrophysiology

Activity of single neurons was recorded in 13 patients (19–47 years old; mean = 32.2 years, SD = 10.1 years; five right-handed women, two left-handed women, five right-handed men, and one left-handed man) with pharmacologically intractable epilepsy undergoing invasive monitoring with intracranial depth electrodes to identify the seizure focus for potential surgical treatment. The patients participated in 22 recording sessions, each on a different day. Each patient had between 7 and 13 electrodes, implanted based on clinical criteria only (Fried et al., 1999). Their anatomical location was verified by MRI or computed tomography scans, coregistered to preoperative MRI. Each of these electrodes terminated with a set of nine 40-μm platinum–iridium microwires. Signals from these microwires were recorded at ∼28 kHz and bandpass filtered in the range 0.3–3 kHz using a 64-channel acquisition system (Neuralynx, Tucson, AZ) and spike sorted as described by Tankus, Yeshurun, and Fried (2009). The classification between single cells and multiunit cells was determined by the SUMU-LDA algorithm using the highest-accuracy linear discriminator obtained by Tankus, Yeshurun, and Fried (2009) and verified visually: 107 single units and 433 multiunits. Units recorded from in different sessions are treated as different in this study. All studies conformed with the guidelines of the medical institutional review board at the University of California, Los Angeles. All patients provided informed consent after the study was explained to them preoperatively. Participation in this study was completely voluntary.

Experimental Paradigms

Patients performed hand movements to control a computer cursor (with or without visual feedback of the consequences of their actions) or passively viewed the same visual feedback cues (Figure 1). All tasks were performed in the same session, and the visual objects involved, namely the cursor and target, were kept simple and did not incorporate components that would emphasize ventral stream processing to reveal dorsal-stream-like processing in the ventral stream.

Figure 1. 

Task conditions. Left: The visuomotor task: Actual performance of the Center-Out task. Center: The visual-only task: Another individual performs the Center-Out task, whereas the patient watches the screen. Right: The motor task: Cued joystick movement in the absence of visual feedback.

Figure 1. 

Task conditions. Left: The visuomotor task: Actual performance of the Center-Out task. Center: The visual-only task: Another individual performs the Center-Out task, whereas the patient watches the screen. Right: The motor task: Cued joystick movement in the absence of visual feedback.

In the visuomotor task (i.e., hand movements with visual feedback), the patients moved a joystick (Attack3, Logitech) to control a computer cursor. They first had to position the cursor inside a ring-shaped target at the center of the screen. Following a short delay period (random between 2 and 3 sec), the target would “jump” to one of eight randomly lit targets, equispaced on a virtual circle around the center. The patients would then move the cursor to follow the target (“Center-Out”; Moran & Schwartz, 1999). The target would return to the center, with the patients following it, and the task will repeat. Patients performed between 32 and 102 trials per visuomotor session (average = 79.5 trials, SD = 18.0). In the visual-only task, another individual performed the same Center-Out task, while the patient only watched the computer screen with no hand movements. The hand of this individual was outside the visual field of the patient. Patients were explicitly instructed to track the cursor with their eyes. As no eye-tracking was performed, this was monitored by an experimenter. The patients watched between 22 and 115 trials per visual-only session (average = 69.0 trials, SD = 25.2). In the motor task, the patients moved the joystick either to the left or right according to a cue to control for motor responses. Before each run of this task, we played/showed the patient 30 randomly selected cues and excluded from the analysis cue-responsive cells. The cue was either auditory (beep: left: 2500 Hz, 50 msec; right: 400 Hz, 50 msec; “Ready!”: 400 Hz, 200 msec; 17 sessions) or visual for hearing-impaired patients (filled square, 13.23 cm × 13.23 cm, displayed for 200 msec; left: blue; right: green; “Ready!”: red). Vision of the movement was prevented either by having the hand and joystick completely outside the field of view (16 sessions; monitored by an experimenter) or by closing the eyes (six sessions). The patients performed between 13 and 20 trials per session (average = 18.3 trials; SD = 1.8). We limit the movements to the left or right only because of the difficulty in memorizing the association of eight different cues with eight different directions.

To dissociate between hand movements and object–eye movements, seven of the patients controlled the Center-Out task by both the joystick and a stylus pen (Graphire3 6 × 8 tablet, Wacom Technology Co., Vancouver, WA) during the same session. Each device creates a different association between hand and object (and thus eye) movements. With the stylus pen, the speed profile of the object and hand is identical (up to a scaling that maps the working area of the tablet to screen area). With the joystick, a stationary position of the hand is translated into a constant speed of the object, whereas every movement of the hand generates acceleration or deceleration. We recorded from 224 units in eight sessions: 64 units in the parahippocampal gyrus, 52 in the hippocampus, 40 in the entorhinal cortex, 14 in the amygdala, and 54 in the anterior cingulate gyrus.

Data Analysis

Only neurons with a firing rate of ≥0.5 spikes/sec were considered for any type of analysis. To relate the neural activity to the continuous kinematic parameters, speed and acceleration, we adopted the nondirectional correlation analysis suggested by Moran and Schwartz (1999), which divides movement time into 10 equal duration bins in each trial and adds 15 prebins of the same size before movement onset. We also added 15 postbins following the end of movement to test for feedback neural activity as well. We subtracted the tonic background component and added the firing rate bin-by-bin across all eight targets to reveal the nondirectional component of the discharge rate profile (“normalized firing rate”), as described by Moran and Schwartz (1999). We then computed the average lag for each unit based on the maximal absolute Pearson correlation between the binned firing rate and speed profiles and similarly for acceleration. Units with correlations significantly different from 0 (p < .05/31; Bonferroni correction for time lags) were deemed related to speed or acceleration, respectively.

Although the aforementioned analysis can relate the firing rate to speed or acceleration, it does not take into account the interdependencies of the two kinematic parameters. To address this issue, we employed the partial cross-correlation analysis (Stark, Drori, & Abeles, 2006), which estimates linear correlations between neural activity and a given parameter that are not affected by linear correlations with other features at multiple time delays. The method thus computes trial-by-trial correlations between neuronal activity and variations in kinematics. We applied the partial cross-correlation analysis to data aligned to movement onset and only to units that have already been declared significant by the nondirectional correlation method (that is, we required both criteria to hold). In the Results section, we describe only units that adhere to both criteria.

According to the partial cross-correlation method, a neuron related to speed is one that at a certain time lag is significantly responsive to speed independent of acceleration and also independent of position; similarly for relation to acceleration and position. The model for position as well as the computation of optimal time lags and significance criteria were described by Tankus, Yeshurun, Flash, and Fried (2009). A relation was declared significant if more than 50% of the entries in the optimal row/column were significant, and for each entry, the requirement was p < .05 following Bonferroni correction (factor of 71). We examined only time lags between −700 and 700 msec (in steps of 20 msec), despite the 2–3 sec hold period, to avoid mixture of trials. Negative time lags refer to firing before the related movement.

For directional responses, the mean firing rate of single- and multiunits during the response period of each trial (between the “Go!” signal and stop inside a target) was subjected to two different statistical tests, adopted from Crammond and Kalaska (1996). First, a two-way ANOVA (Directions vs. Repetitions) detected statistically different responses with the direction of movement (p < .01). Second, a nonparametric “bootstrapping” test (Cisek, Crammond, & Kalaska, 2003; Georgopoulos, Kettner, & Schwartz, 1988), determined the degree of directional bias in the task-related activity of each unit. Comparing the mean length of the distribution of its discharge across all eight movement directions with the chance of this distribution (by shuffling the directional “label” of each trial) decided whether the cell was directionally tuned (4000 shuffles; p < .01). Only when both criteria were met was a unit declared directionally tuned. We avoided the third (Rayleigh) test of Crammond and Kalaska (1996), because in many cases the observed tuning curves were not unimodal.

In the visual-only task, an experimenter monitored that the patients are attentive to the task. Nevertheless, the percentage of units related to the kinematic parameters was very low in all regions (in every region: <2%). To verify that the decreased neuronal modulations are not because of reduced attention to the passive observation task (Khan, Heinen, & McPeek, 2010; Garza, Eslinger, & Barrett, 2008; Chiba, Yamaguchi, & Eto, 2005; Boussaoud, 2001), we repeated the analysis using data limited to only the first 5 min of every session, during which higher attention of the subject was observed. This analysis found very low percentages of cells related to the kinematic parameters (speed: 0% [0/565]; acceleration: 0.4% [2/565]; position: 0.4% [2/565]).

RESULTS

We recorded from 565 units and analyzed 540 of them (25 were excluded due to response to the cue itself): 124 cells in the parahippocampal gyrus, 161 in the hippocampus, 87 in the entorhinal cortex, 59 in the amygdala, and 109 in the ACC. Using the three tasks (visuomotor, visual-only, and motor), we found two distinct populations of cells in the medial-temporal lobe. One population was related to the speed and/or direction of hand movement during visuomotor coordination, but not during visual-only or motor tasks. The other is a population of motor-like cells, which were related to the kinematic parameters speed, acceleration, and position during the motor task, but not during the visuomotor or visual-only tasks.

The first population of visuomotor units was found mainly in the parahippocampal gyrus (PHG), wherein 9% (11/124) of the units were related to speed during the visuomotor task, but not during the visual-only or motor tasks. In all other regions, the percentage of speed-related units during the visuomotor task was low (<6%), and its proportion differed significantly from that of the parahippocampal gyrus (p < 3E−4, χ2(1) = 13.6). Of the speed-related parahippocampal units, 36% (4/11) were also related to acceleration at different time lags during the visuomotor task, whereas the percentage of position-related visuomotor units was very low in all recorded regions (<1% in each area).

For example, the nondirectional component of the firing rate of a neuron in the right PHG was highly correlated with the average movement speed during the visuomotor task (r = .89) but showed much lower correlation during the visual-only or motor tasks (r = 0.62 and .69, respectively; Figure 2). Analyzing the relation of the firing rate of this cell with the speed profile on a trial-by-trial basis using partial cross correlation showed significant response to speed that is independent of acceleration only during the visuomotor task (Figure 2, graphs on the right).

Figure 2. 

An example unit in the right PHG whose firing rate is significantly related to speed during the visuomotor task (A), but not during the visual-only task (B) or motor task (C). Left: The outer perimeters of A, B, and C show the raw (i.e., unsmoothed and untransformed) firing rates during movements to each target in each of the three conditions aligned to movement onset. The vertical and horizontal calibration bars on the left of the figures represent 5 spikes/sec and 500 msec, respectively. The red vertical dashed lines represent the portion of the histogram that was used to generate the central figure. The histograms were then smoothed and transformed according to the method of Moran and Schwartz (1999; see Experimental Methods). The resulting nondirectional profile (red; in the central graphs) is highly correlated to the average movement speed (overlaid blue profiles) during the visuomotor task, but neither during the visual-only task nor the motor task (as indicated also by the correlation coefficients, r). Right: Each graph shows the t statistic of the partial cross correlation between the firing rate and speed (top graph) or acceleration (bottom graph) that is independent of its relation to acceleration or speed at different time lags, respectively. The x axis is the time lag of acceleration, whereas the y axis is that of speed (in msec). A strong horizontal response stripe in the top graph thus indicates a relation to speed that is independent of acceleration, as can be seen in the visuomotor part (but not the visual-only or motor part) at a speed-related time lag of −240 msec (i.e., the firing precedes the movement).

Figure 2. 

An example unit in the right PHG whose firing rate is significantly related to speed during the visuomotor task (A), but not during the visual-only task (B) or motor task (C). Left: The outer perimeters of A, B, and C show the raw (i.e., unsmoothed and untransformed) firing rates during movements to each target in each of the three conditions aligned to movement onset. The vertical and horizontal calibration bars on the left of the figures represent 5 spikes/sec and 500 msec, respectively. The red vertical dashed lines represent the portion of the histogram that was used to generate the central figure. The histograms were then smoothed and transformed according to the method of Moran and Schwartz (1999; see Experimental Methods). The resulting nondirectional profile (red; in the central graphs) is highly correlated to the average movement speed (overlaid blue profiles) during the visuomotor task, but neither during the visual-only task nor the motor task (as indicated also by the correlation coefficients, r). Right: Each graph shows the t statistic of the partial cross correlation between the firing rate and speed (top graph) or acceleration (bottom graph) that is independent of its relation to acceleration or speed at different time lags, respectively. The x axis is the time lag of acceleration, whereas the y axis is that of speed (in msec). A strong horizontal response stripe in the top graph thus indicates a relation to speed that is independent of acceleration, as can be seen in the visuomotor part (but not the visual-only or motor part) at a speed-related time lag of −240 msec (i.e., the firing precedes the movement).

But whose kinematic profile do these neurons plan: that of the hand or that of the object (the cursor), which is also shared with the eye tracking it? By dissociating hand trajectory from object–eye trajectory (see Experimental Paradigms under Experimental Methods), we were able to find examples of parahippocampal cells planning hand movement as well as cells planning object or eye movements. With the stylus pen, the activity of 9.4% (6/64) of the parahippocampal cells was related to speed (all other regions: 1.9% (3/160); significantly different proportion: p = .01, χ2(1) = 6.7). A third of these units encoded kinematic parameters in a manner consistent with encoding hand movements (speed with the stylus pen and acceleration with the joystick; position with the stylus pen and speed with the joystick), which cannot be explained by object or eye movements. Another third of the responsive units responded similarly for both control devices (i.e., independent of actual hand movements) suggesting encoding of kinematic parameters of the object or the tracking eye.

The parahippocampal gyrus is also the only region where we found directionally tuned units during the visuomotor task (PHG: 12% [15/124]; all other regions: 0% [0/416]; significantly different proportion: p = 6E−13, χ2(1) = 51.8). The vast majority of these units (13/15, 87%) responded neither during the visual-only task nor during the motor task. For example, a unit tuned to the up and down directions during the visuomotor task showed no significant directional tuning during the visual-only and motor tasks (Figure 3).

Figure 3. 

Directional visuomotor responses 200–600 msec after the “Go!” signal. A single unit in the left parahippocampal gyrus is responsive during the visuomotor task (A), but not during the visual-only (B) or motor (C) task. Above each peristimulus time histogram (Firing Rate [spikes/sec] vs. Time [sec]) is the corresponding raster plot. Each graph is displayed at the direction of the corresponding target. Asterisks denote directions of significant response. In A, the neuron responds during movement in the up or down directions. The responses in B and C are not statistically significant.

Figure 3. 

Directional visuomotor responses 200–600 msec after the “Go!” signal. A single unit in the left parahippocampal gyrus is responsive during the visuomotor task (A), but not during the visual-only (B) or motor (C) task. Above each peristimulus time histogram (Firing Rate [spikes/sec] vs. Time [sec]) is the corresponding raster plot. Each graph is displayed at the direction of the corresponding target. Asterisks denote directions of significant response. In A, the neuron responds during movement in the up or down directions. The responses in B and C are not statistically significant.

During the motor task, we found a nontrivial (>7%) percentage of neurons significantly related to the kinematic parameters (position, speed, and acceleration) in all recorded temporal lobe areas (to at least one parameter in each area; see Figure 4, left). The vast majority of units in this population were not significantly related to these parameters during the visuomotor or visual-only tasks (Figure 4, right). We thus found a second population of neurons in the medial-temporal lobe, motor-like neurons, which were related to kinematic parameters during the motor task, but not during the visuomotor or visual-only tasks.

Figure 4. 

Percentages of units whose firing rate significantly correlated with the kinematic parameters (position, speed, and acceleration) in each recorded area according to both criteria (nondirectional correlation and partial cross correlation) during the motor task (left) and during the motor task but not during the visuomotor or visual-only task (right). The similarity between the two graphs demonstrates that the vast majority of units related to kinematic parameters during the motor task did not exhibit this behavior during the visuomotor or visual-only task.

Figure 4. 

Percentages of units whose firing rate significantly correlated with the kinematic parameters (position, speed, and acceleration) in each recorded area according to both criteria (nondirectional correlation and partial cross correlation) during the motor task (left) and during the motor task but not during the visuomotor or visual-only task (right). The similarity between the two graphs demonstrates that the vast majority of units related to kinematic parameters during the motor task did not exhibit this behavior during the visuomotor or visual-only task.

An example is a neuron in the left PHG that was highly correlated (r = .97) with the average speed profile during the motor task, but exhibited a much lower correlation during the visuomotor or visual-only tasks (r = .71 and .59, respectively; Figure 5). The partial cross-correlation method of the firing rate of this cell with speed that is independent of acceleration was significant during the motor task, but not during the visuomotor or visual-only tasks (Figure 5, graphs on the right). The partial cross correlation of the firing rate with acceleration that is independent of speed was not significant. An acceleration-related example unit from the right hippocampus appears in Figure 6. Its nondirectional firing rate follows the acceleration profile closely (r = .96) during the motor task, but not during the visuomotor or visual-only tasks (r = .71 and .61, respectively), with significant partial cross correlation with acceleration that is independent of the correlation with speed.

Figure 5. 

An example unit in the left PHG whose firing rate is significantly related to speed during the motor task (A), but not during the visuomotor task (B) or visual-only task (C) according to both the nondirectional correlation and partial cross-correlation methods. The figure is organized similarly to Figure 2. For this unit, the relation of the firing rate to speed that is independent of acceleration occurs at a speed-related time lag of −220 msec during the motor task.

Figure 5. 

An example unit in the left PHG whose firing rate is significantly related to speed during the motor task (A), but not during the visuomotor task (B) or visual-only task (C) according to both the nondirectional correlation and partial cross-correlation methods. The figure is organized similarly to Figure 2. For this unit, the relation of the firing rate to speed that is independent of acceleration occurs at a speed-related time lag of −220 msec during the motor task.

Figure 6. 

An example unit in the left PHG whose firing rate is significantly related to acceleration during the motor task (A), but neither during the visuomotor task (B) nor visual-only task (C) according to both the nondirectional correlation and partial cross-correlation methods. The figure is organized similarly to Figure 5, except that here the blue graphs represent movement acceleration. Right: A strong vertical response stripe in the lower graph indicates a relation to acceleration that is independent of speed, as can be seen in the motor part (but not the visuomotor or visual-only part) at an acceleration-related time lag of 220 msec.

Figure 6. 

An example unit in the left PHG whose firing rate is significantly related to acceleration during the motor task (A), but neither during the visuomotor task (B) nor visual-only task (C) according to both the nondirectional correlation and partial cross-correlation methods. The figure is organized similarly to Figure 5, except that here the blue graphs represent movement acceleration. Right: A strong vertical response stripe in the lower graph indicates a relation to acceleration that is independent of speed, as can be seen in the motor part (but not the visuomotor or visual-only part) at an acceleration-related time lag of 220 msec.

Thus, we find consistent relations to movement, vision and their combination, because the two populations are almost completely limited to response to one task (visuomotor or motor) but not to the other two: All (100%, 11/11) PHG units that were speed-related during the visuomotor task did not respond significantly during the visual-only or motor task, and vice versa: the vast majority of units responsive during the motor task did not respond significantly during the visuomotor or visual-only tasks (Figure 4). This is also consistent with the fact that there were very few responses during the visual-only task (in every region: <2%).

DISCUSSION

Our study revealed and characterized two different populations of cells in the medial-temporal lobe: visuomotor cells, which are correlated with the kinematics of actual movements only in the presence of visual input, and motor-like neurons that participate in hand movements in the absence of visual input.

Our finding of a population of visuomotor speed- and direction-related neurons in the parahippocampal gyrus focuses on an early stage in the process of sensorimotor transformations: transformation between visual input and hand kinematics (output), which precedes known transformations between extrinsic and intrinsic coordinate frames of the acting limb (Kakei, Hoffmand, & Strick, 2003). The parahippocampal involvement in visuo-motor transformations suggests a physiological basis for the anatomical subpathway between the dorsal stream and the medial-temporal lobe (Kravitz, Saleem, Baker, & Mishkin, 2011; Cavada & Goldman-Raiuc, 1989). Thus, the subpathway, which connects the angular gyrus and superior occipital gyrus areas (Rushworth, Behrens, & Johansen-Berg, 2006) or the lateral intraparietal area (Blatt, Andersen, & Stoner, 1990) with the parahippocampal cortex, might be the source of action-guiding visual information to the visuomotor population we described in the parahippocampal gyrus. These findings suggest that the medial-temporal lobe—specifically the parahippocampal gyrus—plays a functional role in both “what” and “where”/“how” streams and render it an important junction in the process of planning and execution of motor acts. Hence, the medial-temporal lobe might serve as an integration node between the two processing streams.

The visuomotor population also expands the known role of the parahippocampal gyrus in spatial navigation, which includes path planning for locomotion based on visual information and, in particular, the relation to locomotive speed and running direction (Doeller, Barry, & Burgess, 2010). Thus, the parahippocampal gyrus is responsible, at the kinematic level, not only for locomotive navigation but also for spatial navigation of the hand or an object controlled by the hand (the cursor in our case). Further studies are needed to dissociate object movement from eye movements. This also provides a kinematic explanation for the recently found parahippocampal activation during motor execution (Mukamel, Ekstrom, Kaplan, Iacoboni, & Fried, 2010).

The population of motor-like neurons was related to kinematic parameters in all recorded temporal areas (H, PHG, EC and A) and according to two different computational methods. This evidence is in line with the recently described participation in motor control of areas classically considered as sensory cortex (Matyas et al., 2010). Our findings also extend movement-related evidence in the hippocampus (Halgren, 1991) and describe them at the kinematic level. As such, the hippocampus and parahippocampal gyrus not only represent places in a 3-D virtual environment (Ekstrom et al., 2003) but also the position of the hand. This implies that body shape and coordinates are represented in greater detail in these areas than reported so far: not only in large-scale coordinate systems in which the body location (“place”) is represented but also in proximity frames of reference, where different body parts (e.g., hands) can interact with the surrounding. Whereas evidence relating parahippocampal neural activation with motor control is very limited and lacks for kinematics (Yan et al., 2006), the current results demonstrate the encoding of hand speed and acceleration in this area.

In the EC, the only significantly encoded kinematic parameter in the motor-like population appears to be position, which expands its role from representation of 3-D position (location) of the subject, as in place (Howard & Natu, 2005) or grid (McNaughton et al., 2006) cells, to also that of the hand during movement. We suggest that the EC may thus participate in the formation of maps that allow one to orient in space, in terms of both self and hand locations. The amygdala was shown to be involved in movement-related perception (Bonda, Petrides, Ostry, & Evans, 1996) or feeling (Nakagawa et al., 2003; Uno et al., 2000) and in incentive processes controlling the performance of goal-directed instrumental actions (Shiflett & Balleine, 2010). Our findings enhance the understanding of these movement-related phenomena by relating cellular activity to kinematic parameters of an actual movement. These results can potentially reflect actual control of hand movements, recalls of a previously learnt trajectory from memory (Bar, 2009; Gelbard-Sagiv, Mukamel, Harel, Malach, & Fried, 2008) or arbitrary associations between antecedents (auditory or visual stimuli) and actions (movements; Murray, Bussey, & Wise, 2000; Wise & Murray, 1999, 2000).

To summarize, we found two populations of cells: motor-like, in multiple temporal areas (PHG, H, EC, A), and visuomotor, mainly in the parahippocampal gyrus, and demonstrated their participation in higher levels of motor activity, driven by either internal input or external visual input. Interestingly, even this high-level participation encodes the detailed, low-level kinematics. Our findings elaborate the known functional organization of the motor control system to also include the temporal lobe and has far reaching implications on our understanding of the generation of the motor plan from visual or internal inputs, on the visual processing pathways, and on the functional role of the temporal lobe.

Acknowledgments

We thank the patients for participating in our study. We also thank R. Mukamel, N. Suthana, and E. Behnke for their assistance; E. Stark and M. Abeles for the code of the partial cross-correlation method; and S. Shoham for insightful comments on this manuscript. This research was supported by Grant 2497 of the Paralyzed Veterans of America Research Foundation, the Dana Foundation, NINDS, Lady Davis Fellowship, and the L&L Richmond research fund.

Reprint requests should be sent to Ariel Tankus, Faculty of Biomedical Engineering Technion, Israel Institute of Technology, Haifa, 32000 Israel, or via e-mail: arielta@gmail.com.

REFERENCES

Aguirre
,
G. K.
, &
D'Esposito
,
M.
(
1999
).
Topographical disorientation: A synthesis and taxonomy.
Brain
,
122
,
1613
1628
.
Bar
,
M.
(
2009
).
The proactive brain: Memory for predictions.
Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences
,
364
,
1235
1243
.
Baumann
,
O.
,
Chan
,
E.
, &
Mattingley
,
J.
(
2010
).
Dissociable neural circuits for encoding and retrieval of object locations during active navigation in humans.
Neuroimage
,
49
,
2816
2825
.
Blair
,
H.
,
Gupta
,
K.
, &
Zhang
,
K.
(
2008
).
Conversion of a phase- to a rate-coded position signal by a three-stage model of theta cells, grid cells, and place cells.
Hippocampus
,
18
,
1239
1255
.
Blatt
,
G. J.
,
Andersen
,
R. A.
, &
Stoner
,
G. R.
(
1990
).
Visual receptive field organization and cortico-cortical connections of the lateral intraparietal area (area LIP) in the macaque.
The Journal of Comparative Neurology
,
299
,
421
445
.
Blatt
,
G. J.
,
Pandya
,
D. N.
, &
Rosene
,
D. L.
(
2003
).
Parcellation of cortical afferents to three distinct sectors in the parahippocampal gyrus of the rhesus monkey: An anatomical and neurophysiological study.
The Journal of Comparative Neurology
,
466
,
161
179
.
Bonda
,
E.
,
Petrides
,
M.
,
Ostry
,
D.
, &
Evans
,
A.
(
1996
).
Specific involvement of human parietal systems and the amygdala in the perception of biological motion.
Journal of Neuroscience
,
16
,
3737
3744
.
Boussaoud
,
D.
(
2001
).
Attention versus intention in the primate premotor cortex.
Neuroimage
,
14
,
S40
S45
.
Burak
,
Y.
, &
Fiete
,
I.
(
2009
).
Accurate path integration in continuous attractor network models of grid cells.
PLoS Compututational Biology
,
5
,
e1000291
.
Carmeli
,
E.
,
Bar-Yossef
,
T.
,
Ariav
,
C.
,
Levy
,
R.
, &
Liebermann
,
D. G.
(
2008
).
Perceptual-motor coordination in persons with mild intellectual disability.
Disability and Rehabilitation
,
30
,
323
329
.
Cavada
,
C.
, &
Goldman-Raiuc
,
P. S.
(
1989
).
Posterior parietal cortex in rhesus monkey: I. Parcellation of areas based on distinctive limbic and sensory corticocortical connections.
Journal of Comparative Neurology
,
287
,
393
421
.
Chiba
,
Y.
,
Yamaguchi
,
A.
, &
Eto
,
F.
(
2005
).
A simple method to dissociate sensory-attentional and motor-intentional biases in unilateral visual neglect.
Brain and Cognition
,
58
,
269
273
.
Cisek
,
P.
,
Crammond
,
D. J.
, &
Kalaska
,
J. F.
(
2003
).
Neural activity in primary motor and dorsal premotor cortex in reaching tasks with the contralateral versus ipsilateral arm.
Journal of Neurophysiology
,
89
,
922
942
.
Crammond
,
D. J.
, &
Kalaska
,
J. F.
(
1996
).
Differential relation of discharge in primary motor cortex and premotor cortex to movements versus actively maintained postures during a reaching task.
Experimental Brain Research
,
108
,
45
61
.
Doeller
,
C. F.
,
Barry
,
C.
, &
Burgess
,
N.
(
2010
).
Evidence for grid cells in a human memory network.
Nature
,
463
,
657
661
.
Duvernoy
,
M.
,
Vannson
,
J.
,
Cattin
,
F.
, &
Naidich
,
T.
(
2005
).
The human hippocampus: Functional anatomy, vascularization and serial sections with MRI.
Berlin
:
Springer
.
Ekstrom
,
A. D.
,
Kahana
,
M. J.
,
Caplan
,
J. B.
,
Fields
,
T. A.
,
Isham
,
E. A.
,
Newman
,
E. L.
,
et al
(
2003
).
Cellular networks underlying human spatial navigation.
Nature
,
425
,
184
187
.
Epstein
,
R.
(
2008
).
Parahippocampal and retrosplenial contributions to human spatial navigation.
Trends in Cognitive Sciences
,
12
,
388
396
.
Felleman
,
D.
, &
Van Essen
,
D.
(
1991
).
Distributed hierarchical processing in the primate cerebral cortex.
Cerebral Cortex
,
1
,
1
47
.
Fernández-Seara
,
M.
,
Aznárez-Sanado
,
M.
,
Mengual
,
E.
,
Loayza
,
F.
, &
Pastor
,
M.
(
2009
).
Continuous performance of a novel motor sequence leads to highly correlated striatal and hippocampal perfusion increases.
Neuroimage
,
47
,
1797
1808
.
Fischer
,
J.
,
Spotswood
,
N.
, &
Whitney
,
D.
(
2011
).
The emergence of perceived position in the visual system.
Journal of Cognitive Neuroscience
,
23
,
119
136
.
Fried
,
I.
,
Wilson
,
C. L.
,
Maidment
,
N. T.
,
Engel
,
J.
,
Behnke
,
E.
,
Fields
,
T. A.
,
et al
(
1999
).
Cerebral microdialysis combined with single-neuron and electroencephalographic recording in neurosurgical patients.
Journal of Neurosurgery
,
91
,
697
705
.
Garza
,
J.
,
Eslinger
,
P.
, &
Barrett
,
A.
(
2008
).
Perceptual-attentional and motor-intentional bias in near and far space.
Brain and Cognition
,
68
,
9
14
.
Gelbard-Sagiv
,
H.
,
Mukamel
,
R.
,
Harel
,
M.
,
Malach
,
R.
, &
Fried
,
I.
(
2008
).
Internally generated reactivation of single neurons in human hippocampus during free recall.
Science
,
322
,
96
101
.
Georgopoulos
,
A. P.
,
Kettner
,
R. E.
, &
Schwartz
,
A. B.
(
1988
).
Primate motor cortex and free arm movements to visual targets in three-dimensional space: II. Coding of the direction of movement by a neuronal population.
Journal of Neuroscience
,
8
,
2928
2937
.
Goodale
,
M. A.
, &
Milner
,
A. D.
(
1992
).
Separate visual pathways for perception and action.
Trends in Neurosciences
,
15
,
20
25
.
Gur
,
M.
, &
Snodderly
,
D. M.
(
2007
).
Direction selectivity in V1 of alert monkeys: Evidence for parallel pathways for motion processing.
Journal of Physiology
,
585
,
383
400
.
Halgren
,
E.
(
1991
).
Firing of human hippocampal units in relation to voluntary movements.
Hippocampus
,
1
,
153
162
.
Howard
,
M.
, &
Natu
,
V.
(
2005
).
Place from time: Reconstructing position from a distributed representation of temporal context.
Neural Networks
,
18
,
1150
1162
.
Howard
,
M. W.
,
Fotedar
,
M. S.
,
Datey
,
A. V.
, &
Hasselmo
,
M. E.
(
2005
).
The temporal context model in spatial navigation and relational learning: Toward a common explanation of medial temporal lobe function across domains.
Psychological Review
,
112
,
75
116
.
Kakei
,
S.
,
Hoffmand
,
D. S.
, &
Strick
,
P. L.
(
2003
).
Sensorimotor transformations in cortical motor areas.
Neuroscience Research
,
46
,
1
10
.
Khan
,
A.
,
Heinen
,
S.
, &
McPeek
,
R.
(
2010
).
Attentional cueing at the saccade goal, not at the target location, facilitates saccades.
Journal of Neuroscience
,
30
,
5481
5488
.
Klapp
,
S. T.
, &
Jagacinski
,
R. J.
(
2011
).
Gestalt principles in the control of motor action.
Psychological Bulletin
,
137
,
443
462
.
Kravitz
,
D. J.
,
Saleem
,
K. S.
,
Baker
,
C. I.
, &
Mishkin
,
M.
(
2011
).
A new neural framework for visuospatial processing.
Nature Reviews Neuroscience
,
12
,
217
230
.
Lengyel
,
M.
, &
Erdi
,
P.
(
2004
).
Theta-modulated feedforward network generates rate and phase coded firing in the entorhino-hippocampal system.
IEEE Transactions on Neural Networks
,
15
,
1092
1099
.
Maguire
,
E. A.
,
Burgess
,
N.
,
Donnett
,
J. G.
,
O'Keefe
,
J.
, &
Frith
,
C. D.
(
1998
).
Knowing where things are: Parahippocampal involvement in encoding object locations in virtual large-scale space.
Journal of Cognitive Neuroscience
,
10
,
61
76
.
Maguire
,
E. A.
,
Frackowiak
,
R. S.
, &
Frith
,
C. D.
(
1997
).
Recalling routes around London: Activation of the right hippocampus in taxi drivers.
The Journal of Neuroscience
,
17
,
7103
7110
.
Marr
,
D.
(
1982
).
Vision. A computational investigation into the human representation and processing of visual information.
New York
:
W.H. Freeman
.
Matyas
,
F.
,
Sreenivasan
,
V.
,
Marbach
,
F.
,
Wacongne
,
C.
,
Barsy
,
B.
,
Mateo
,
C.
,
et al
(
2010
).
Motor control by sensory cortex.
Science
,
330
,
1240
1243
.
McNaughton
,
B.
,
Battaglia
,
F.
,
Jensen
,
O.
,
Moser
,
E.
, &
Moser
,
M.
(
2006
).
Path integration and the neural basis of the “cognitive map.”
Nature Reviews Neuroscience
,
7
,
663
678
.
Mishkin
,
M.
, &
Ungerleider
,
L.
(
1982
).
Contribution of striate inputs to the spatial functions of parieto-preoccipital cortex in monkeys.
Behavioural Brain Research
,
6
,
57
77
.
Mishkin
,
M.
,
Ungerleider
,
L.
, &
Macko
,
K.
(
1983
).
Object vision and spatial vision: Two cortical pathways.
Trends in Neurosciences
,
6
,
414
417
.
Miyashita
,
Y.
,
Rolls
,
E.
,
Cahusac
,
P.
,
Niki
,
H.
, &
Feigenbaum
,
J.
(
1989
).
Activity of hippocampal formation neurons in the monkey related to a conditional spatial response task.
Journal of Neurophysiology
,
61
,
669
678
.
Moran
,
D.
, &
Schwartz
,
A.
(
1999
).
Motor cortical representation of speed and direction during reaching.
Journal of Neurophysiology
,
82
,
2676
2692
.
Mukamel
,
R.
,
Ekstrom
,
A. D.
,
Kaplan
,
J.
,
Iacoboni
,
M.
, &
Fried
,
I.
(
2010
).
Single-neuron responses in humans during execution and observation of actions.
Current Biology
,
20
,
750
756
.
Murray
,
E. A.
,
Bussey
,
T. J.
, &
Wise
,
S. P.
(
2000
).
Role of prefrontal cortex in a network for arbitrary visuomotor mapping.
Experimental Brain Research
,
133
,
114
129
.
Nagel
,
M.
,
Sprenger
,
A.
,
Hohagen
,
F.
,
Binkofski
,
F.
, &
Lencer
,
R.
(
2008
).
Cortical mechanisms of retinal and extraretinal smooth pursuit eye movements to different target velocities.
Neuroimage
,
41
,
483
492
.
Nakagawa
,
A.
,
Uno
,
A.
,
Horii
,
A.
,
Kitahara
,
T.
,
Kawamoto
,
M.
,
Uno
,
Y.
,
et al
(
2003
).
Fos induction in the amygdala by vestibular information during hypergravity stimulation.
Brain Research
,
986
,
114
123
.
Owen
,
A. M.
,
Milner
,
B.
,
Petrides
,
M.
, &
Evans
,
A. C.
(
1996
).
A specific role for the right parahippocampal gyrus in the retrieval of object-location: A positron emission tomography study.
Journal of Cognitive Neuroscience
,
8
,
588
602
.
Ploner
,
C. J.
,
Gaymard
,
B. M.
,
Rivaud-Péchoux
,
S.
,
Baulac
,
M.
,
Clémenceau
,
S.
,
Samson
,
S.
,
et al
(
2000
).
Lesions affecting the parahippocampal cortex yield spatial memory deficits in humans.
Cerebral Cortex
,
10
,
1211
1216
.
Ptito
,
M.
,
Faubert
,
J.
,
Gjedde
,
A.
, &
Kupers
,
R.
(
2003
).
Separate neural pathways for contour and biological-motion cues in motion-defined animal shapes.
Neuroimage
,
19
,
246
252
.
Robertson
,
R. G.
,
Rolls
,
E. T.
,
Georges-Francois
,
P.
, &
Panzeri
,
S.
(
1999
).
Head direction cells in the primate pre-subiculum.
Hippocampus
,
9
,
206
219
.
Rocca
,
M.
,
Agosta
,
F.
,
Mezzapesa
,
D.
,
Falini
,
A.
,
Martinelli
,
V.
,
Salvi
,
F.
,
et al
(
2004
).
A functional MRI study of movement-associated cortical changes in patients with Devic's neuromyelitis optica.
Neuroimage
,
21
,
1061
1068
.
Rumiati
,
R. I.
,
Weiss
,
P. H.
,
Tessari
,
A.
,
Assmus
,
A.
,
Zilles
,
K.
,
Herzog
,
H.
,
et al
(
2005
).
Common and differential neural mechanisms supporting imitation of meaningful and meaningless actions.
Journal of Cognitive Neuroscience
,
17
,
1420
1431
.
Rushworth
,
M.
,
Behrens
,
T.
, &
Johansen-Berg
,
H.
(
2006
).
Connection patterns distinguish 3 regions of human parietal cortex.
Cerebral Cortex
,
16
,
1418
1430
.
Sato
,
N.
, &
Nakamura
,
K.
(
2003
).
Visual response properties of neurons in the parahippocampal cortex of monkeys.
Journal of Neurophysiology
,
90
,
876
886
.
Shiflett
,
M. W.
, &
Balleine
,
B. W.
(
2010
).
At the limbic-motor interface: Disconnection of basolateral amygdala from nucleus accumbens core and shell reveals dissociable components of incentive motivation.
The European Journal of Neuroscience
,
32
,
1735
1743
.
Stark
,
E.
,
Drori
,
R.
, &
Abeles
,
M.
(
2006
).
Partial cross-correlation analysis resolves ambiguity in the encoding of multiple movement features.
Journal of Neurophysiology
,
95
,
1966
1975
.
Steele
,
G.
,
Weller
,
R.
, &
Cusick
,
C.
(
1991
).
Cortical connections of the caudal subdivision of the dorsolateral area (V4) in monkeys.
Journal of Comparative Neurology
,
306
,
495
520
.
Sugiura
,
M.
,
Shah
,
N.
,
Zilles
,
K.
, &
Fink
,
G.
(
2005
).
Cortical representations of personally familiar objects and places: Functional organization of the human posterior cingulate cortex.
Journal of Cognitive Neuroscience
,
17
,
183
198
.
Tankus
,
A.
,
Yeshurun
,
Y.
,
Flash
,
T.
, &
Fried
,
I.
(
2009
).
Encoding of speed and direction of movement in the human supplementary motor area.
Journal of Neurosurgery
,
110
,
1304
1316
.
Tankus
,
A.
,
Yeshurun
,
Y.
, &
Fried
,
I.
(
2009
).
An automatic measure for classifying clusters of suspected spikes into single cells versus multiunits.
Journal of Neural Engineering
,
6
,
056001
.
Tesche
,
C. D.
, &
Karhu
,
J.
(
1999
).
Interactive processing of sensory input and motor output in the human hippocampus.
Journal of Cognitive Neuroscience
,
11
,
424
436
.
Uemura
,
K.
(
2010
).
A revised clinical assessment of motor and memory disturbances.
Neurologia Medico-Chirurgica
,
50
,
707
712
.
Ungerleider
,
L.
, &
Mishkin
,
M.
(
1982
).
Two cortical visual systems.
In D. Ingle, M. Goodale, & R. Mansfield (Eds.),
Analysis of visual behavior.
Cambridge, MA
:
MIT Press
.
Uno
,
A.
,
Takeda
,
N.
,
Horii
,
A.
,
Sakata
,
Y.
,
Yamatodani
,
A.
, &
Kubo
,
T.
(
2000
).
Effects of amygdala or hippocampus lesion on hypergravity-induced motion sickness in rats.
Acta Oto-Laryngologica
,
120
,
860
865
.
van Rooyen
,
F.
,
Young
,
N. A.
,
Larson
,
S. E. M.
, &
Teskey
,
G. C.
(
2006
).
Hippocampal kindling leads to motor map expansion.
Epilepsia
,
47
,
1383
1391
.
Wallentin
,
M.
,
Ostergaarda
,
S.
,
Lund
,
T. E.
,
Ostergaard
,
L.
, &
Roepstorff
,
A.
(
2005
).
Concrete spatial language: See what I mean?
Brain and Language
,
92
,
221
233
.
Wilson
,
C.
,
Babb
,
T.
,
Halgren
,
E.
, &
Crandall
,
P.
(
1983
).
Visual receptive fields and response properties of neurons in human temporal lobe and visual pathways.
Brain
,
106
,
473
502
.
Wise
,
S. P.
, &
Murray
,
E. A.
(
1999
).
Role of the hippocampal system in conditional motor learning: Mapping antecedents to action.
Hippocampus
,
9
,
101
117
.
Wise
,
S. P.
, &
Murray
,
E. A.
(
2000
).
Arbitrary associations between antecedents and actions.
Trends in Neurosciences
,
23
,
271
276
.
Yan
,
L.
,
Wu
,
D.
,
Wang
,
X.
,
Zhou
,
Z.
,
Liu
,
Y.
,
Yao
,
S.
,
et al
(
2006
).
Intratask and intertask asymmetry analysis of motor function.
NeuroReport
,
17
,
1143
1147
.