Abstract
Oculomotor tracking of moving objects is an important component of visually based cognition and planning. Such tracking is achieved by a combination of saccades and smooth-pursuit eye movements. In particular, the saccadic and smooth-pursuit systems interact to often choose the same target, and to maximize its visibility through time. How do multiple brain regions interact, including frontal cortical areas, to decide the choice of a target among several competing moving stimuli? How is target selection information that is created by a bias (e.g., electrical stimulation) transferred from one movement system to another? These saccade–pursuit interactions are clarified by a new computational neural model, which describes interactions between motion processing areas: the middle temporal area, the middle superior temporal area, the frontal pursuit area, and the dorsal lateral pontine nucleus; saccade specification, selection, and planning areas: the lateral intraparietal area, the frontal eye fields, the substantia nigra pars reticulata, and the superior colliculus; the saccadic generator in the brain stem; and the cerebellum. Model simulations explain a broad range of neuroanatomical and neurophysiological data. These results are in contrast with the simplest parallel model with no interactions between saccades and pursuit other than common-target selection and recruitment of shared motoneurons. Actual tracking episodes in primates reveal multiple systematic deviations from predictions of the simplest parallel model, which are explained by the current model.
INTRODUCTION
The primate retina affords wide-field detection of visual pattern and motion, as well as narrow-field scrutiny of visual details via its central zone, the fovea. A typical oculomotor episode involves detection of visual stimuli by receptors in the retinal periphery, cortical selection of one of the detected stimuli as a goal for scrutiny, and genesis of eye movements that enable foveation of the goal. Two eye movement systems cooperate to achieve foveation if the goal stimulus is moving: the saccadic (SAC) and the smooth pursuit eye movement (SPEM). The SAC eye movement system generates ballistic, high-velocity, open-loop ocular rotations that quickly cancel the difference between the initial angle of gaze and the angle needed to foveate the goal. The SPEM system generates continuous, moderate-velocity, closed-loop ocular rotations that prolong foveation of mobile goal stimuli by trying to match gaze velocity to stimulus velocity. For sufficiently fast stimuli, prolonged foveation requires re-engagement of the SAC system to generate “catch-up” saccades.
It is critical that the two eye movements systems coordinate so that they (1) choose the same target as the goal for future movement, and (2) maximize visibility of that target. A prior report (Grossberg, Srihasam, & Bullock, submitted) introduced a new model of SAC–SPEM interactions, and reported data-simulation comparisons that revealed how the model achieved the second of these requirements. This report presents additional data-simulation comparisons which illustrate how circuit interactions ensure that both systems generate movements directed to the same target.
It might be thought that unitary targeting is not a significant problem. However, one system can operate without the other, a typical scene contains an assortment of salient stimuli with some stationary and some moving, and the motion-sensitive anatomical pathways that generate SPEM are largely separate from, and evolved after, those that generate SAC (Krauzlis, 2005; Krauzlis, Liston, & Carello, 2004). What interactions choose a unitary goal among several competing stimuli? Is goal selection information transferred from one system to the other, and if so, how?
Although SAC and SPEM systems have often been assumed to have a single, shared, target selection mechanism, recent data from monkeys suggest that each system has a selection mechanism that can operate independently of the other. In these experiments, two different stimuli appear and begin to move in different directions. Location, shape, direction of motion, or color help the monkey distinguish a task-relevant target from an irrelevant distractor (Garbutt & Lisberger, 2006; Adler, Bala, & Krauzlis, 2002; Gardner & Lisberger, 2001; Krauzlis, Zivotofsky, & Miles, 1999; Ferrera & Lisberger, 1995). Because the latency for SPEM initiation is shorter than for SAC initiation, it is possible to behaviorally assess whether the SPEM system always reflects the same choice as the slower-starting SAC system. Indeed, on a substantial fraction of trials, the initial SPEM tracks the distractor. However, immediately after a saccade to the correct target, the continuing SPEM tracks that target rather than the distractor. Such results suggest that each of the SAC and SPEM systems makes its own initial choice of goal, but that the SAC choice overrides the SPEM system's choice (Adler et al., 2002; Gardner & Lisberger, 2001, 2002; Horwitz & Newsome, 2001; Ferrera & Lisberger, 1995).
Before the model of Grossberg et al. (submitted), no model had addressed the functional anatomy of SAC–SPEM interactions for target selection and coordination. Although a lot is known about the mechanisms involved in saccadic target selection (Thomas & Pare, 2007; Schiller & Tehovnik, 2003, 2005; Arai, McPeek, & Keller, 2004; Krauzlis et al., 2004; Basso & Wurtz, 2002; McPeek & Keller, 2002; Thompson, Hanes, Bichot, & Schall, 1996; Schall, Hanes, Thompson, & King, 1995), less is known about how smooth-pursuit target selection occurs (Case & Ferrera, 2007; Garbutt & Lisberger, 2006; Krauzlis, 2005), or about how these two eye movement systems interact to select the same target among many distractors (Adler et al., 2002; Gardner & Lisberger, 2001, 2002; Krauzlis et al., 1999). Our new model proposes a mechanistic explanation for, and quantitatively simulates, the key generalizations that have emerged from systematic empirical studies of target tracking eye movements concerning how the smooth pursuit system selects its target, and how the two eye movement systems interact to select the same target. After reviewing the SAC–SPEM model's circuitry and operation, the present article explains how model circuits, notably those dependent on the frontal cortex, can explain and simulate challenging behavioral and electrophysiological data from experiments on target selection.
MODEL OVERVIEW
The model (Figure 1) consists of two parallel yet interacting processing streams for the control of SPEM and SAC movements. Neuroanatomy-based models of SPEM and SAC eye movements were used as starting points for this work: the SACCART model of learning and performance using a multimodal movement map in the superior colliculus (SC; Gancarz & Grossberg, 1999; Grossberg, Roberts, Aguilar, & Bullock, 1997), the TELOS model of learned cue-guided voluntary selection of saccade plans (Brown, Bullock, & Grossberg, 2004), and a model of motion perception and SPEM command genesis in the middle superior temporal area (MST; Pack, Grossberg, & Mingolla, 2001). A complete mathematical specification of the model, and details regarding simulations, are provided in the Supplementary Material section.
Modeled interactions among brain regions implicated in oculomotor control. In (A) black boxes denote areas belonging to the saccadic eye movement system (SAC), white boxes the smooth-pursuit eye movement system (SPEM), and gray boxes, both systems. LIP = lateral intraparietal area; FPA = frontal pursuit area; MST = middle superior temporal area; MT = middle temporal area; FEF = frontal eye fields; NRTP = nucleus reticularis tegmenti pontis; DLPN = dorsolateral pontine nuclei; SC = superior colliculus; CBM = cerebellum; MVN/rLVN = medial and rostrolateral vestibular nuclei; PPRF = peripontine reticular formation; TN = tonic neurons. (B) Constituents of the saccade generator in the PPRF, and the projection of omni-pause neurons to the pursuit neurons of the MVN/rLVN. Arrowheads terminate excitatory connections, and circles terminate inhibitory connections. OPN = omni-pause neurons; LLBN = long-lead burst neuron; EBN = excitatory burst neuron; IBN = inhibitory burst neuron; TN = tonic neurons.
Modeled interactions among brain regions implicated in oculomotor control. In (A) black boxes denote areas belonging to the saccadic eye movement system (SAC), white boxes the smooth-pursuit eye movement system (SPEM), and gray boxes, both systems. LIP = lateral intraparietal area; FPA = frontal pursuit area; MST = middle superior temporal area; MT = middle temporal area; FEF = frontal eye fields; NRTP = nucleus reticularis tegmenti pontis; DLPN = dorsolateral pontine nuclei; SC = superior colliculus; CBM = cerebellum; MVN/rLVN = medial and rostrolateral vestibular nuclei; PPRF = peripontine reticular formation; TN = tonic neurons. (B) Constituents of the saccade generator in the PPRF, and the projection of omni-pause neurons to the pursuit neurons of the MVN/rLVN. Arrowheads terminate excitatory connections, and circles terminate inhibitory connections. OPN = omni-pause neurons; LLBN = long-lead burst neuron; EBN = excitatory burst neuron; IBN = inhibitory burst neuron; TN = tonic neurons.
The SPEM Stream
Figure 1A gives an overview of the structure of the model SPEM circuit. The model's smooth pursuit stream contains visual area middle temporal (MT)-like cell types, which are distinguished by selectivity for different combinations of the direction and speed of visual stimuli that fall within their retinotopic receptive fields (Albright, 1984; Maunsell & van Essen, 1983a). The 800 model MT cells provide inputs to the model's MST cells, which pool their MT inputs in a way that makes them direction-selective and speed-sensitive, but not speed-selective. Because these MST cells also receive inputs corresponding to current eye velocity, they can compute an internal estimate of predicted target velocity that remains accurate even as eye velocity grows to match target velocity, and thus, gradually cancels the target-related retinal image motion that drives MT cells.
The predictive estimate of target velocity computed by model MST cells provides a basis for the model's frontal cortical representation of desired pursuit velocity. In particular, the frontal pursuit area (FPA), at the rostral bank of the arcuate sulcus, receives inputs from both MST and MT (Tian & Lynch, 1996a, 1996b; Huerta, Krubitzer, & Kaas, 1987). Model and real FPA cells have high direction selectivity and speed sensitivity, but almost no speed selectivity (Tanaka & Lisberger, 2002b; Gottlieb, Bruce, & MacAvoy, 1993).
The model FPA cells project (Giolli et al., 2001; Brodal, 1980) to the model nucleus reticularis tegmenti pontis (NRTP), which includes two types of cells: acceleration cells and velocity cells (Ono, Das, Economides, & Mustari, 2005; Ono, Das, & Mustari, 2004; Suzuki, Yamada, Hoedema, & Yee, 1999; Yamada, Suzuki, & Yee, 1996). Model NRTP velocity cells integrate the output of NRTP acceleration cells. The latter compute the difference between an excitatory target velocity command from FPA and an inhibitory eye-velocity signal from the vestibular nuclei. This difference estimates the eye acceleration that is needed to match target velocity. These two classes of cells allow the NRTP to play a key role in SPEM initiation.
One possible source of this inhibitory eye velocity signal to NRTP might be from the ventrolateral thalamus. Neurons in the ventrolateral thalamus discharged before or during initiation of pursuit, and the firing rate was proportional to the speed of target motion in a preferred direction. When the tracking target was extinguished briefly during maintenance of pursuit, these neurons continued firing, indicating that they carried extraretinal eye movement signals (Tanaka, 2005).
Parallel to the FPA–NRTP pathway, a second pathway exists for the transmission of SPEM-related information from the cortex to the cerebellum (CBM) via the pons. This pathway, apart from being implicated in maintenance of SPEM (Mustari, Fuchs, & Wallman, 1988; Suzuki & Keller, 1984), also helps to program saccades made to moving objects. Model MT cells project to the dorsal lateral pontine nucleus (DLPN) cells of the brain stem, which then project to the CBM (Figure 1A). In the model, the DLPN cells have similar speed and directional selectivities as MT cells, but they lack their retinotopic specificity.
The SAC Stream
In the model SAC system, retinotopically organized visual signals are processed to produce saccadic target choices in the model lateral intraparietal area (LIP) and the frontal eye fields (FEF). FEF outputs serve as inputs to corresponding retinotopic loci in the burst and buildup cell layers (Sommer & Wurtz, 2000, 2004; Munoz & Wurtz, 1995a, 1995b) of the motor error map of the model's SC. There is also communication between these SC layers, such that activated loci in the burst cell layer excite corresponding cells in the buildup cell layer (cf. Grossberg et al., 1997).
Outputs from the SC reach the CBM and the saccade generator circuit (Figure 1B) in the paramedian pontine reticular formation (PPRF), which contains populations of SAC- and SPEM-related cells, some of which provide direct input to the oculomotor neurons that innervate eye muscles. Model saccadic control signals from cerebellar and SC stages converge at model long-lead burst neurons (LLBN). LLBN activity encodes gaze-position error. LLBN cells excite corresponding excitatory burst neurons (EBN), which project to the tonic neurons (TN). TNs integrate inputs from EBNs and excite the model oculomotor neurons. The EBNs also excite inhibitory burst neurons (IBN), which in turn inhibit the LLBNs, thereby completing an internal negative feedback loop that controls ballistic saccades. Except during saccades, the EBNs receive strong inhibition from model omni-pause neurons (OPNs), so-called because they pause deeply to disinhibit saccades of all directions. For reviews of these anatomical connections and neurophysiological properties, see Büttner & Büttner-Ennever, 1989, 2005; Moschovakis & Highstein, 1994; Fuchs, Kaneko, & Scudder, 1985; for reviews of computational models that incorporate such features, see Girard & Berthoz, 2005).
Shared Omni-pausers
OPNs are located in the nucleus raphe interpositus. The pursuit neurons (PNs) found in the vestibular nuclei are modeled as receiving input from the CBM and projecting directly to the TNs, which are thus shared by SAC and SPEM systems. The PNs are weakly inhibited by, and themselves inhibit, the OPNs, also shared by both systems. About 50% of the OPNs show reduced activity (a 34% drop) during smooth pursuit (Missal & Keller, 2002), whereas most OPNs pause more deeply during saccades (Munoz, Dorris, Pare, & Everling, 2000; Everling, Pare, Dorris, & Munoz, 1998). Thus, the spontaneously active and inhibitory OPNs normally oppose both saccades and SPEM. Shallow pausing by OPNs can release SPEM but not saccades, whose release requires deeper pauses.
Cerebellar Learning Calibrates SPEM and SAC Commands
Learning is needed to keep SAC and SPEM metrics accurate as eye muscles and other system parameters change. Inactivation or lesion of the CBM causes deficits in the ability to adapt both SAC and SPEM (Takagi, Zee, & Tamargo, 1998). Each model cell in the retinotopic SC, the speed-sensitive DLPN, and the direction-sensitive NRTP sends signals to the CBM (Thier & Ilg, 2005). These signals are modified by adaptive weights learned within the CBM. The adaptive saccade-related cerebellar outputs reach the model PPRF region of the brain stem (Figure 1A), the location of the saccade generator (Figure 1B). Similarly, the weighed pursuit-related cerebellar outputs reach the PNs in the vestibular nuclei of the brain stem.
SPEM System Inhibition of SAC Initiation via an MT–SC–OPN Pathway
Behavioral data (simulated in Grossberg et al., submitted) suggest the existence of an intelligent mechanism to control saccade initiations during SPEM, notably to inhibit saccades when targets are already foveal or parafoveal. Pursuit-related neural activity is reliably observed in the SC: Rostral parts of SC (rSC) contain cells that respond to both SPEM and SAC eye movements (Krauzlis, Basso, & Wurtz, 2000). As schematized in Figure 2, area MT sends strong excitatory projections to the rSC (Collins, Lyon, & Kaas, 2005; Maioli, Domeniconi, Squatrito, & Riva Sanseverino, 1992; Davidson & Bender, 1991; Wall, Symonds, & Kaas, 1982; Spatz & Tigges, 1973), which in turn provides the main excitatory input to the OPNs (Everling et al., 1998; Gandhi & Keller, 1997; Pare & Guitton, 1994; Munoz & Wurtz, 1993a, 1993b). Our hypothesis that rSC projects to the OPN is consistent with data from Büttner-Ennever, Horn, Henn, and Cohen (1999), which state that “there are multiple projections directly onto OPNs from the rostral SC but not from the caudal SC associated with large gaze shifts….” Also, the place where FEF fibers terminate appears identical to the region where SC fibers terminate (Büttner & Büttner-Ennever, 1989). Stanton, Goldberg, and Bruce (1988) found direct projections from the fundus of the arcuate sulcus (FEF) to a region in nucleus ralphe (OPNs) identical to where SC fibers terminate. Thus, we hypothesize that OPNs might be getting foveal input from both the FEF and the SC. In the model (Figures 1 and 2), foveal and parafoveal cells in area MT, which are active when pursued targets are on or near the fovea, inhibit saccades via an excitatory pathway from MT to rSC to the OPNs.
Cortico-colliculo-reticular control of saccade initiation. The figure illustrates a pathway from foveal and parafoveal cells in cortical area MT to the rostral pole of the SC and then to the OPNs in the PPRF. In the model, effective tracking causes MT foveal cells to become active. This, in turn, activates fixation cells present in the rostral SC. Such SC cells excite OPNs, which can inhibit saccade initiation or suspend ongoing saccades.
Cortico-colliculo-reticular control of saccade initiation. The figure illustrates a pathway from foveal and parafoveal cells in cortical area MT to the rostral pole of the SC and then to the OPNs in the PPRF. In the model, effective tracking causes MT foveal cells to become active. This, in turn, activates fixation cells present in the rostral SC. Such SC cells excite OPNs, which can inhibit saccade initiation or suspend ongoing saccades.
Target Selection in the Two Streams
Selecting a saccadic target present among competing distracters, as in a visual search paradigm (Treisman & Gormican, 1988), requires dissociation of target stimuli from others. Visually responsive neurons in the FEF show this discrimination. Late phase activity of these neurons was accentuated for targets and attenuated for distractors that excited their motor receptive fields (Schall & Thompson, 1999; Thompson, Bichot, & Schall, 1997; Thompson et al., 1996; Schall, Hanes, et al., 1995). Stimulation of an LIP, FEF, or SC neuron increases the probability of choosing the stimulus within its motor field as the target (Carello & Krauzlis, 2004; Schiller & Tehovnik, 2001, 2003), and reversible inactivation of the FEF, LIP, and SC produces significant saccadic target selection deficits (McPeek & Keller, 2004; Schiller & Tehovnik, 2003; Wardak, Olivier, & Duhamel, 2002).
In the model, interactions in the parieto-frontal circuit (LIP–FEF–LIP) are critical for selecting targets among competing distractors. Figure 3 schematizes interactions in the SAC pathway that lead to target selection. FEF planning neurons (similar to the FEF visual motor cells in Schall, Hanes, et al., 1995), which receive input from lower-level visual areas (not shown), send outputs to the LIP, FEF output neurons (similar to the FEF motor cells in Schall, Hanes, et al., 1995), SC buildup neurons, and the striatum of the basal ganglia (BG; Schall, Morel, King, & Bullier, 1995). In amphibians and all land vertebrates, the BG interact with the optic tectum (OT), or its homologue, the SC, to control orienting action (Marin, Smeets, & Gonzalez, 1998; Butler & Hodos, 1996). In mammals, the BG interact with frontal cortical areas to control orienting, cognitive, and manipulative behaviors (Strick, Dum, & Picard, 1995; Passingham, 1993; Hikosaka & Wurtz, 1983). Lesions of the BG cause disorders such as Parkinsonian akinesia, Huntington's chorea, and ballism (Albin, Young, & Penney, 1989), suggesting a strong link between the BG and areas involved in movement control. These interactions provide a natural basis for differentiating plan activation and plan execution. Brown et al. (2004) developed a detailed neural model of how the FEF and the BG interact with movement-controlling areas such as the SC.
Fronto-parietal circuit interactions leading to target selection. The figure shows connections in the model's SAC pathway that help choose a target among competing stimuli. FEF planning neurons, LIP neurons, and SC buildup neurons send collaterals to a decision stage representing the basal ganglia and thalamus (BG–thalamus). Once a BG–thalamus decision is made regarding the choice of target, a GO signal is sent from the thalamus to the FEF planning and output neurons. The signal from the thalamus boosts the activity of all cells in the corresponding part of the FEF. With the help of the GO signal and mutual inhibition, network interactions amplify a weak initial advantage by one planning cell into a winner-take-all choice of that planning cell and its corresponding output cell. The figure also shows that the target selection decision is transmitted from FEF output cells via the SC and the CBM to the LLBNs in the saccade generator (in the PPRF; see also Figure 1).
Fronto-parietal circuit interactions leading to target selection. The figure shows connections in the model's SAC pathway that help choose a target among competing stimuli. FEF planning neurons, LIP neurons, and SC buildup neurons send collaterals to a decision stage representing the basal ganglia and thalamus (BG–thalamus). Once a BG–thalamus decision is made regarding the choice of target, a GO signal is sent from the thalamus to the FEF planning and output neurons. The signal from the thalamus boosts the activity of all cells in the corresponding part of the FEF. With the help of the GO signal and mutual inhibition, network interactions amplify a weak initial advantage by one planning cell into a winner-take-all choice of that planning cell and its corresponding output cell. The figure also shows that the target selection decision is transmitted from FEF output cells via the SC and the CBM to the LLBNs in the saccade generator (in the PPRF; see also Figure 1).
The current SAC–SPEM model approximates BG-mediated decision-making by sending a GO signal excitation to the FEF if the sum of the FEF, LIP, and SC signals associated with a particular saccadic vector is large enough to exceed a threshold. In vivo, this signal results from striatal disinhibition of the thalamus, and can be regarded as opening a normally closed gate to release the planned movement (Lo & Wang, 2006; Brown et al., 2004; Wurtz & Hikosaka, 1986). Once the gate is open, the FEF cell having maximal activity is selected as the target, and it suppresses other target representations via mutual inhibition.
We predict that the smooth eye movement part of the FEF, known as the FPA, is involved in decision-making for SPEM and for coordinated tracking by SPEM and SAC. The FPA, located near the rostral bank of the arcuate sulcus, is strongly innervated by the MST (Tian & Lynch, 1996a, 1996b; Huerta et al., 1987). FPA cells have high direction-tuning and almost no speed selectivity (Lynch & Tian, 2005; Tian & Lynch, 1996a, 1996b, 1997; Gottlieb et al., 1993). Lesions in the FPA cause SPEM deficits, including reduction of pursuit gain and directional deficits (Shi, Friedman, & Bruce, 1998; Keating, Pierre, & Chopra, 1996). Electrical stimulation of the FPA causes SPEM, and, if applied during natural SPEM, such stimulation increases SPEM gain if the natural pursuit direction and the stimulated direction are the same (Carey & Lisberger, 2004; Tanaka & Lisberger, 2002a, 2002b, 2002c; Gottlieb et al., 1993).
In the model, an SPEM target choice can be made independently by the FPA under the influence of visual motion inputs from the MT and the MST. Directions of all the moving objects in the environment form retinotopically organized peaks of activity of neuronal populations in the MT. We hypothesize that the FPA, which lies downstream from the MT and the MST, converts this sensory signal to a motor signal for movement. This assumption is in agreement with data showing multiple peaks in the MT, before selection occurs downstream (Treue, Hol, & Rauber, 2000). This fact is reflected in model dynamics (compare Figure 7E and F below). Related data about selection of saccadic movement commands beyond MT can be found in Roitman and Shadlen (2002) and Shadlen and Newsome (2001), and are modeled in Grossberg and Pilly (2008).
A GO signal that represents the excitatory thalamo-cortical output of the BG–thalamus system is released when the FPA signal to the BG exceeds a threshold for movement initiation. This assumption accords with recent data (Cui, Yan, & Lynch, 2003) indicating that the FPA connects with the BG–thalamus system in a way that parallels FEF connections with that system (see Brown et al., 2004).
Although both SAC and SPEM systems can reach decisions, the model also provides for coordination. Krauzlis et al. (1999) found that once a target has been selected by the saccadic system, and a saccade is initiated, it is rare for the eye to move along the direction of any contending stimulus after the saccade. Figure 1A shows that there is a path in the model (and in vivo) from the FEF to the LIP to the MT to the MST to the FPA. This model pathway mediates transfer of the SAC system's target choice to the SPEM system. This transfer enables the SAC system to override the SPEM system. Although the FEF and the FPA are nearby in the frontal cortex, the model links them via other areas, and not directly, in accord with both computational and anatomical constraints.
The LIP is key in that linkage. First, it receives collaterals from both saccadic and smooth pursuit pathways (Tian & Lynch, 1996a, 1996b; Schall, Morel, et al., 1995; Maunsell & Van Essen, 1983b). Second, its activity strongly reflects a monkey's decision about a subsequent eye movement, and choice-related up-modulation of direction-tuned cells are seen regardless of whether the sensory inputs driving this decision are stationary or moving stimuli (Schiller & Tehovnik, 2005; Gold & Shadlen, 2000, 2001; Shadlen & Newsome, 2001). Grossberg and Pilly (2008) describe a model for how motion stimuli influence LIP decision-making. Third, activation of LIP affects target choice without affecting the saccade metrics such as saccade duration or velocity (Thomas & Pare, 2007; Schiller & Tehovnik, 2005; Wardak et al., 2002; Schiller & Chou, 1998).
RESULTS: COMPARISONS OF SIMULATIONS WITH DATA
The Supplementary Material section provides simulation details, including the system of differential equations that, together with external inputs, govern the model circuit's dynamics.
Simulation 1. Simulating How Voluntary Choice of a Saccadic Eye Movement Target is Made by Interactions among Frontal, Subcortical, and Parietal Areas
Consider the case of selecting a saccadic target among two stimuli (see Figure 4A). The initial phase of activity (50–100 msec) is similar for both target and distractor (see Figure 4B, C, and D), consistent with observations that visual neurons of the FEF show transient activity as early as 50 msec after stimulus onset (Schall, Hanes, et al., 1995). But the second phase of activity is selective for the target. FEF planning neurons representing the target increase, and other neurons representing distractors decrease, in their activity (Figure 4B). Because the two stimuli are identical, there is an equal probability for either to be chosen as target. The choice is the result of a competitive race between the cells coding the two stimuli. Attentional bias, in the form of oddball size, shape, or color stimulus, can help break the symmetry and determine the winner. In the present case, no such bias was present, so noise was added at the FEF stage. After an interval of slow separation between the competing representations, which begins in FEF cells at the time marked by arrows in Figures 4B and D, the BG–thalamus stage reaches threshold. Its output (GO signal) to FEF initiates the rapid separation that is visible first in FEF planning and output cells (Figure 4B and C), and after a brief delay, in LIP cells (Figure 4C, arrow). Thus, the model predicts that the choice is made by the frontal circuit and rapidly relayed to the LIP. Cortical lateral inhibition mediates the model's rapid suppression of the unchosen representation.
Saccadic target selection: paradigm and simulations. (A) schematizes the paradigm used to simulate saccadic target selection. Two similar stimuli are flashed on either side of, and at the same eccentricity from, a central fixation point. After 150–200 msec, the model generates a saccade that brings the fovea onto whichever of the two stimuli was selected as the target. (B) to (D) illustrate how this selection is achieved by the model. In each panel, a dotted trace shows the activity of the neuron with the distractor (nonchosen stimulus) in its RF, whereas the thick trace shows the activity of a neurons with the target in its RF. (B) shows the activities of two FEF planning neurons, (C) shows two LIP neurons and (D) shows activities of FEF output (presaccadic burst) neurons. During the initial 100 msec after stimulus presentation, the activities evolve similarly for each pair. Around 120 msec, a small separation opens between the competing neuron types in the FEF (arrows), but not in the LIP. Around 140 msec, the activity (not shown) of the BG–thalamus decision process reaches its threshold and a GO signal is generated. This is followed immediately by rapid growth in FEF activations for the target and rapid suppression of FEF activations for the distractor. This choice induces a similar choice in LIP (arrow, in C).
Saccadic target selection: paradigm and simulations. (A) schematizes the paradigm used to simulate saccadic target selection. Two similar stimuli are flashed on either side of, and at the same eccentricity from, a central fixation point. After 150–200 msec, the model generates a saccade that brings the fovea onto whichever of the two stimuli was selected as the target. (B) to (D) illustrate how this selection is achieved by the model. In each panel, a dotted trace shows the activity of the neuron with the distractor (nonchosen stimulus) in its RF, whereas the thick trace shows the activity of a neurons with the target in its RF. (B) shows the activities of two FEF planning neurons, (C) shows two LIP neurons and (D) shows activities of FEF output (presaccadic burst) neurons. During the initial 100 msec after stimulus presentation, the activities evolve similarly for each pair. Around 120 msec, a small separation opens between the competing neuron types in the FEF (arrows), but not in the LIP. Around 140 msec, the activity (not shown) of the BG–thalamus decision process reaches its threshold and a GO signal is generated. This is followed immediately by rapid growth in FEF activations for the target and rapid suppression of FEF activations for the distractor. This choice induces a similar choice in LIP (arrow, in C).
Figure 5 compares the simulated activity of model FEF planning and output neurons with FEF electrophysiological recordings (Schall, Hanes, et al., 1995) from a visual search experiment in which targets were clearly discriminable from distractors. In the simulation, this difference was mimicked by making one stimulus 1.5 times as strong as another one. The stronger stimulus was always chosen, and the activation profiles in the model are qualitatively similar to the recordings.
Role of the FEF in target selection decisions: data and simulations. The right column presents model FEF planning (A, B) and output (C, D) neuron activities for comparison with real (left column) visuomotor (A, B) and presaccadic/motor (C, D) neuron activities recorded during saccadic target selection in a visual search paradigm (Schall, Hanes, et al., 1995). (A) and (C) depict activities of neurons representing to-be-foveated targets, whereas (B) and (D) depict activities of neurons representing distractors. The shaded areas in the box inserts (one in each panel) indicate the neurons' receptive fields. Targets are indicated by filled black squares, distractors by white squares. The small vertical arrows in (A) and (B) in the right column indicate the time of saccade initiation. For (A) and (B), time zero represents stimulus onset. For (C) and (D), time zero represents saccade onset.
Role of the FEF in target selection decisions: data and simulations. The right column presents model FEF planning (A, B) and output (C, D) neuron activities for comparison with real (left column) visuomotor (A, B) and presaccadic/motor (C, D) neuron activities recorded during saccadic target selection in a visual search paradigm (Schall, Hanes, et al., 1995). (A) and (C) depict activities of neurons representing to-be-foveated targets, whereas (B) and (D) depict activities of neurons representing distractors. The shaded areas in the box inserts (one in each panel) indicate the neurons' receptive fields. Targets are indicated by filled black squares, distractors by white squares. The small vertical arrows in (A) and (B) in the right column indicate the time of saccade initiation. For (A) and (B), time zero represents stimulus onset. For (C) and (D), time zero represents saccade onset.
Simulation 2. Simulating How Voluntary Choice of an SPEM Target is Made by the FPA
Often, one of several moving stimuli is chosen for tracking by SPEM, in the absence of any saccades. If two moving stimuli are nearby the initial gaze position, then the direction of this SPEM often begins as the vector average of the two stimulus motion directions, then quickly evolves into pursuit along the direction of one stimulus. The second phase reflects a decision, which is also signaled by an increase in SPEM gain: The ratio of eye velocity to target velocity (SPEM gain) is much lower during the initial vector averaging phase than after the system has committed to pursuit of one target. Figure 6 highlights the idea that the phase transition occurs once a decision process in the BG–thalamus exceeds the threshold for generating a GO signal that increases the output from FPA averaging cells to subcortical SPEM generators.
Model of visual motion and SPEM pathway. The figure shows model cortical and subcortical connections along the visual motion and SPEM pathway. In addition to their visual input (see Figure 1), MT+ and MT− receive input from the LIP. The receptive fields of model LIP and MT are aligned retinotopically. MT− cells project to the MSTv and MT+ cells project to the MSTd. FPA input (FPAi) or WTA cells receives projections from MSTv cells and, in turn, projects to FPA output (FPAo) or averaging cells. FPAo cells also get input from MSTd cells via FPA summation (FPAs) cells. Strong mutual inhibition, and connections with a decision stage representing basal ganglia and thalamus (BG–thalamus), help FPAo cells achieve target selection. This information is carried to the cerebellum via acceleration and velocity cells of the NRTP.
Model of visual motion and SPEM pathway. The figure shows model cortical and subcortical connections along the visual motion and SPEM pathway. In addition to their visual input (see Figure 1), MT+ and MT− receive input from the LIP. The receptive fields of model LIP and MT are aligned retinotopically. MT− cells project to the MSTv and MT+ cells project to the MSTd. FPA input (FPAi) or WTA cells receives projections from MSTv cells and, in turn, projects to FPA output (FPAo) or averaging cells. FPAo cells also get input from MSTd cells via FPA summation (FPAs) cells. Strong mutual inhibition, and connections with a decision stage representing basal ganglia and thalamus (BG–thalamus), help FPAo cells achieve target selection. This information is carried to the cerebellum via acceleration and velocity cells of the NRTP.
Figure 7 presents simulation results that illustrate the model's ability to generate a similar transition between vector averaging and choice of one target for high-gain SPEM, and also shows the consequence of the decision for the activity of model FPA output cells. Unlike saccadic vector averaging, which holds for an entire saccade, and occurs only when there is small angular separation between the potential targets and when SAC latency is very short (Arai et al., 2004; Ottes, Van Gisbergen, & Eggermont, 1984), pursuit averaging is more commonly observed (Gardner & Lisberger, 2001, 2002; Tanaka & Lisberger, 2002c; Recanzone & Wurtz, 1999). In the model, the difference between the two systems occurs because the output cells in the FPA can become partly activated before a decision is reached, unlike output cells of the FEF. Such a lower threshold for output from the FPA than the FEF makes behavioral sense because averaging low-gain SPEM incurs no visual cost, whereas a premature saccade would impose costs in the form of an interval of reduced vision and defoveation of the target.
Simulated smooth-pursuit target selection for two stimuli that are moving toward each other. The figure illustrates how activities of various cells help decide a target for SPEM. The paradigm used is a modified double-target paradigm. The initial unsigned speeds of the two stimuli are the same, and the position–speed combinations are such that no saccade is needed to initiate tracking of either of the two stimuli. (A) Stimulus 1 starts at [−4° H, 5° V] and moves with a speed of 20 deg/sec along the 305° direction; Stimulus 2 starts at [−4° H, −5° V] and moves at 20 deg/sec along the 45° direction. (B) shows the horizontal and (C) the vertical eye position components, and the corresponding two components of the two stimulus positions. In both (B) and (C), the thin black line is the trace of the stimulus selected as target and the thin dotted line represents the trace of the distractor. The thick black line represents the eye trace. Arrows in the panels represent the time at which the eye moves along that particular axis (i.e., B shows the time at which the eye moves in the horizontal direction and the arrow in C indicates the time at which the eye moves in the vertical direction). Because the two stimuli move in oblique directions, the eye moves along the vector-averaged direction (rightward direction here) before making a target choice decision; this is evident from the arrow positions in (B), (C), and (D). (D) shows the horizontal and vertical eye and target velocity traces. The thick black trace rising in the positive (upward) direction shows the horizontal eye velocity component and the thin dotted trace dropping in the negative direction shows the later-starting vertical component of eye velocity. These component velocities asymptote at the target velocity components, 20 deg/sec H and 20 deg/sec V. (E) and (F) illustrate the crucial SPEM model activities enabling choice. (E) shows activities of two FPA output (“averaging”) cells (see Figure 6) and (F) illustrates the activities of MSTv cells. Thick black lines in both (E) and (F) show the activity trace of the cell whose preferred direction was along the direction of motion of the stimulus selected as target and the thin dotted line represents the activity trace of the cell whose preferred direction was along the direction of motion of the distractor. At around 150 msec, FPA cells show early signs of target selectivity. Between 150 and 200 msec, vector averaging takes place as the FPA contains representations of both target and distractor. FPA output neurons project to a decision stage conceptualized as a BG–thalamus stage partly analogous to that in the SAC system. At around 190 msec, the decision stage activity reaches a threshold and generates a GO-signal to the FPA. This boosts the activity of the cell with maximal activity, causing it to be selected.
Simulated smooth-pursuit target selection for two stimuli that are moving toward each other. The figure illustrates how activities of various cells help decide a target for SPEM. The paradigm used is a modified double-target paradigm. The initial unsigned speeds of the two stimuli are the same, and the position–speed combinations are such that no saccade is needed to initiate tracking of either of the two stimuli. (A) Stimulus 1 starts at [−4° H, 5° V] and moves with a speed of 20 deg/sec along the 305° direction; Stimulus 2 starts at [−4° H, −5° V] and moves at 20 deg/sec along the 45° direction. (B) shows the horizontal and (C) the vertical eye position components, and the corresponding two components of the two stimulus positions. In both (B) and (C), the thin black line is the trace of the stimulus selected as target and the thin dotted line represents the trace of the distractor. The thick black line represents the eye trace. Arrows in the panels represent the time at which the eye moves along that particular axis (i.e., B shows the time at which the eye moves in the horizontal direction and the arrow in C indicates the time at which the eye moves in the vertical direction). Because the two stimuli move in oblique directions, the eye moves along the vector-averaged direction (rightward direction here) before making a target choice decision; this is evident from the arrow positions in (B), (C), and (D). (D) shows the horizontal and vertical eye and target velocity traces. The thick black trace rising in the positive (upward) direction shows the horizontal eye velocity component and the thin dotted trace dropping in the negative direction shows the later-starting vertical component of eye velocity. These component velocities asymptote at the target velocity components, 20 deg/sec H and 20 deg/sec V. (E) and (F) illustrate the crucial SPEM model activities enabling choice. (E) shows activities of two FPA output (“averaging”) cells (see Figure 6) and (F) illustrates the activities of MSTv cells. Thick black lines in both (E) and (F) show the activity trace of the cell whose preferred direction was along the direction of motion of the stimulus selected as target and the thin dotted line represents the activity trace of the cell whose preferred direction was along the direction of motion of the distractor. At around 150 msec, FPA cells show early signs of target selectivity. Between 150 and 200 msec, vector averaging takes place as the FPA contains representations of both target and distractor. FPA output neurons project to a decision stage conceptualized as a BG–thalamus stage partly analogous to that in the SAC system. At around 190 msec, the decision stage activity reaches a threshold and generates a GO-signal to the FPA. This boosts the activity of the cell with maximal activity, causing it to be selected.
Figure 6 shows that two further classes of model FPA cells, which correspond to what Tanaka and Lisberger (2002c) called “summation” and “WTA” (winner-take-all; see Grossberg, 1973, 1982, for how to design networks with a WTA property) cells, receive inputs from model areas MSTd and MSTv, respectively, and excite FPA output (averaging) cells. Figure 8A shows data on all three types of FPA cells, and Figure 8B shows how the model simulates the qualitative differences between these cell types under the three conditions tested. These patterns illustrate that both summation cells and WTA cells come close to their maximum activities levels during the vector averaging phase (i.e., before a choice has been made), whereas the output (averaging) cells show much lower than maximal activity until the disappearance of one target results in choosing the one that remains. This property suggests that the averaging cells are output cells that reflect the BG–thalamus decision process.
Simulated effect of FPA stimulation on smooth-pursuit target selection. (A) shows the measured activities of three types of cells found in the FPA (Tanaka & Lisberger, 2002c). A double-target and a single-target paradigm were used. In the former, two stimuli appeared at the same time at 4° eccentricity on either side of the fixation point. They start moving toward each other at 20 deg/sec, and at 200 msec, which coincides with the time their trajectories cross, one of the stimuli is extinguished. The remaining stimulus should be selected as target and tracked. In the single-target paradigm, a single target appeared at 4° eccentricity and started moving toward the fixation point at 20 deg/sec. The eccentricities and velocities were set so as not to elicit any saccades. In each column, the color of the neuron activity trace codes the condition under which the activity was recorded. Red trace: double-target paradigm, with the remaining target's motion in the direction preferred by the neuron; Blue trace: double-target paradigm, with the remaining target's motion in the direction opposite to that preferred by the neuron. The black traces are activities from single-target control trials, in which target motion matched the cell's preferred direction. (B) shows simulations of activities of the corresponding three types of model cells, in the same paradigms. The top row in (A) and (B) shows “averaging” or “output” cells, the middle row shows “summation” cells, and the bottom row shows “winner-take-all” or “input” cells. (C) and (D) show simulation results when the model FPA was stimulated during SPEM target selection. The paradigm used was the double-target paradigm described above. The thick black bar on the x-axis in each panel represents the interval (0–200 msec) during which a left-motion-tuned model FPA output cell was stimulated. In both (C) and (D), thin black and thin dotted black lines represent the kinematic traces of two stimuli. The thin black line is the trace of the stimulus selected as target, the dotted line that of the distractor. The thick black trace represents the trace of the eye through time. The top rows in (C) and (D) represent target and eye position traces. The bottom rows represent the stimuli and eye velocity traces. The left column (C) shows the case when the leftward (plotted as downward) moving stimulus was extinguished after 200 msec. The right column (D) shows the case when the rightward-moving (plotted as upward) stimulus was extinguished. In (D), the model continues to track the artificial-stimulation-selected target, which remains visible. In (C), the target initially selected for SPEM tracking by artificial FPA stimulation in the model is abandoned once the stimulation ceases and the target stimulus disappears. The model takes around 200 msec to generate a catch-up saccade (at around t = 400 msec) to the rightward-moving target, after which the eye smoothly tracks it.
Simulated effect of FPA stimulation on smooth-pursuit target selection. (A) shows the measured activities of three types of cells found in the FPA (Tanaka & Lisberger, 2002c). A double-target and a single-target paradigm were used. In the former, two stimuli appeared at the same time at 4° eccentricity on either side of the fixation point. They start moving toward each other at 20 deg/sec, and at 200 msec, which coincides with the time their trajectories cross, one of the stimuli is extinguished. The remaining stimulus should be selected as target and tracked. In the single-target paradigm, a single target appeared at 4° eccentricity and started moving toward the fixation point at 20 deg/sec. The eccentricities and velocities were set so as not to elicit any saccades. In each column, the color of the neuron activity trace codes the condition under which the activity was recorded. Red trace: double-target paradigm, with the remaining target's motion in the direction preferred by the neuron; Blue trace: double-target paradigm, with the remaining target's motion in the direction opposite to that preferred by the neuron. The black traces are activities from single-target control trials, in which target motion matched the cell's preferred direction. (B) shows simulations of activities of the corresponding three types of model cells, in the same paradigms. The top row in (A) and (B) shows “averaging” or “output” cells, the middle row shows “summation” cells, and the bottom row shows “winner-take-all” or “input” cells. (C) and (D) show simulation results when the model FPA was stimulated during SPEM target selection. The paradigm used was the double-target paradigm described above. The thick black bar on the x-axis in each panel represents the interval (0–200 msec) during which a left-motion-tuned model FPA output cell was stimulated. In both (C) and (D), thin black and thin dotted black lines represent the kinematic traces of two stimuli. The thin black line is the trace of the stimulus selected as target, the dotted line that of the distractor. The thick black trace represents the trace of the eye through time. The top rows in (C) and (D) represent target and eye position traces. The bottom rows represent the stimuli and eye velocity traces. The left column (C) shows the case when the leftward (plotted as downward) moving stimulus was extinguished after 200 msec. The right column (D) shows the case when the rightward-moving (plotted as upward) stimulus was extinguished. In (D), the model continues to track the artificial-stimulation-selected target, which remains visible. In (C), the target initially selected for SPEM tracking by artificial FPA stimulation in the model is abandoned once the stimulation ceases and the target stimulus disappears. The model takes around 200 msec to generate a catch-up saccade (at around t = 400 msec) to the rightward-moving target, after which the eye smoothly tracks it.
Note that summation cells show a decline in activation during SPEM maintenance (in the latter part of the trial), whereas WTA cells show sustained activity during this interval. This difference is explicable in the model because summation cells receive input from area MSTd cells, which become much less active as successful SPEM cancels target motion signals in the retinal frame (since there are no background stimuli), and because WTA cells receive inputs from MSTv cells, which remain active even during successful pursuit because they receive a corollary discharge of eye velocity (Pack et al., 2001).
If the FPA is involved in deciding which stimulus to follow, then stimulation of the FPA should bias the result during target selection. Because microstimulation in the FPA causes eye movements along the particular direction represented at the stimulation site, any stimulus moving along this direction should be favored and selected over competing stimuli. Data of Tanaka and Lisberger (2), Figure 6) confirm this idea. The simulation results plotted in Figure 8C and D show that this is also true of the model. Initially, there are two stimuli moving in opposite directions (left and right directions in our simulations) and toward the fixation point. Selective stimulation of left-tuned model FPA output cells causes leftward (plotted as downward) SPEM. At 200 msec after the stimulus turns on, one of the stimuli disappears (in Figure 8C, the left-moving stimulus disappears; in Figure 8D, the right-moving stimulus disappears). At the same time, the stimulation to the FPA ceases. Figure 8C exemplifies a case in which the stimulated direction does not match that of the target that remains after stimulation offset. Following stimulation offset, the eye stops moving leftward, makes a rightward catch-up saccade, and then pursues the right-moving target. Figure 8D illustrates what happens when the electrically stimulated direction is the same as the direction of the remaining target. Although the SPEM latency is the same for stimulation and nonstimulation cases, the eye reaches target velocity faster when the FPA is stimulated.
Simulation 3: Simulating How Pursuit Initiation is Altered by a Moving Distractor
The latency of SPEM initiation depends on the distribution of directions of moving stimuli. Adding a second moving stimulus will increase or decrease the latency of SPEM initiation if the second stimulus moves, respectively, in the same or the opposite direction as the chosen target (Ferrera & Lisberger, 1995). Note that in the case of oppositely moving stimuli, the vector average will be no movement. In such cases, no SPEM will be observed until the system truly makes a choice. In the model, stimuli moving in different directions activate cells in the MT and the MST that are tuned to these directions. Such direction-sensitive cells project to the CBM and later output stages via two pathways (Figure 1A): one via the DLPN and another via the FPA and the NRTP. Although both branches can contribute to short-latency SPEM via vector averaging, neither will do so when the average of two opposing motions is zero. But if two potential targets are moving in the same direction, MT, MST, and FPA summation cells coding this direction will often be more active, notably if there are two motion inputs within their receptive fields. The summation cells of the FPA almost double their firing rate (Figure 8A, middle panel) if they contain two targets in their receptive field. This leads to faster activation of FPA output cells. Faster activation of FPA output cells leads to faster CBM cell activation, and thus, to faster pursuit initiation. Figure 9 illustrates these effects and shows model simulations (right) next to behavioral data (Ferrera & Lisberger, 1995).
Effect of distractor direction on latency of smooth pursuit initiation. The figure shows how the direction of a distractor affects the latency of SPEM initiation. The paradigm simulated was a modified double-stimulus paradigm. Two stimuli were flashed at the same time, on the same side, and at the same eccentricity, relative to the fixation point, but vertically displaced from each other. They start moving horizontally, either in the same or in opposite directions. The figure compares model eye velocities with the data (reprinted, with permission, from Ferrera & Lisberger, 1995) during the first 300 msec of tracking. In data and model, when the direction of motion of the distractor (gray box in right column) matches that of the target (black box in right column), SPEM is initiated earlier than if the motion directions mismatch. This is seen in the trace offsets in both the right (model) and left (data) columns.
Effect of distractor direction on latency of smooth pursuit initiation. The figure shows how the direction of a distractor affects the latency of SPEM initiation. The paradigm simulated was a modified double-stimulus paradigm. Two stimuli were flashed at the same time, on the same side, and at the same eccentricity, relative to the fixation point, but vertically displaced from each other. They start moving horizontally, either in the same or in opposite directions. The figure compares model eye velocities with the data (reprinted, with permission, from Ferrera & Lisberger, 1995) during the first 300 msec of tracking. In data and model, when the direction of motion of the distractor (gray box in right column) matches that of the target (black box in right column), SPEM is initiated earlier than if the motion directions mismatch. This is seen in the trace offsets in both the right (model) and left (data) columns.
Simulation 4: Simulating How Saccadic Target Choice Overrides the Choice Made by Smooth Pursuit
Behavioral and neurophysiological data indicate that target selection can occur independently in the SAC and SPEM subsystems, as simulated above. Independent selection raises a coordination problem: Which choice dominates in a tracking episode, and how is the coordination achieved? We hypothesize that SAC choice overrides SPEM choice. Data from behavioral (Adler et al., 2002) and microstimulation (Gardner & Lisberger, 2001, 2002) experiments support this claim. Figure 10 compares model simulations (right) of the Gardner and Lisberger (2002) paradigm with their published data (left). In this double-stimulus paradigm, where natural saccades to either stimulus are equiprobable, an FEF site with a known directional preference and movement receptive field is stimulated after pursuit initiation, just as one stimulus enters the movement field, but before any natural saccades would occur. The stimulation evokes a saccade to the stimulus, even if the presaccadic SPEM was tracking the other stimulus. Moreover, SPEM velocity after the evoked saccade almost always matches the velocity of the saccadic target, not the velocity of the stimulus being tracked before the evoked saccade. Thus, SAC choice overrides an existing SPEM choice. In the model, these properties emerge because focal FEF stimulation activates a corresponding retinotopic zone in the LIP which, in turn, activates a retinotopic zone in the MT and the MST, within which inputs from the motion of the favored target will activate the correct direction-tuned cells that project to later stages of the SPEM system (Figures 1 and 6).
Saccadic choice drives smooth pursuit choice. The figure illustrates how saccadic choice, when forced via FEF stimulation, drives a corresponding choice within the SPEM system. (A) shows vertical and horizontal eye position traces when the FEF was stimulated at a sampled location during fixation in Phase 1 of the experimental paradigm (Gardner & Lisberger, 2002) simulated. In this case, stimulation evoked a 4° oblique saccade and allowed the experimenters to predict the part of the visual field from which the same cell could be activated. (B) illustrates that in Phase 2, they then made two stimuli appear at the same eccentricity on two sides of the fixation point. They start moving away from each other, such that one of the stimuli would enter the previously mapped FEF cell's movement field. If that FEF cell is stimulated just as the target enters its movement field and before a natural saccade is initiated, then the SPEM system will always choose to pursue that target rather than the alternative. (C) and (D) show horizontal position and velocity traces, respectively. Similarly, (E) and (F) show vertical position and velocity traces. Blue and green lines show the traces of kinematic variables for the two moving stimuli. Black traces show the relatively long-latency, natural catch-up saccades to either stimuli on trials with no FEF stimulation, whereas red traces show the relatively short-latency saccades evoked by FEF stimulation. Eye velocity after the saccade matches exactly that of the target near its current position, whether the target was naturally chosen or chosen by FEF stimulation.
Saccadic choice drives smooth pursuit choice. The figure illustrates how saccadic choice, when forced via FEF stimulation, drives a corresponding choice within the SPEM system. (A) shows vertical and horizontal eye position traces when the FEF was stimulated at a sampled location during fixation in Phase 1 of the experimental paradigm (Gardner & Lisberger, 2002) simulated. In this case, stimulation evoked a 4° oblique saccade and allowed the experimenters to predict the part of the visual field from which the same cell could be activated. (B) illustrates that in Phase 2, they then made two stimuli appear at the same eccentricity on two sides of the fixation point. They start moving away from each other, such that one of the stimuli would enter the previously mapped FEF cell's movement field. If that FEF cell is stimulated just as the target enters its movement field and before a natural saccade is initiated, then the SPEM system will always choose to pursue that target rather than the alternative. (C) and (D) show horizontal position and velocity traces, respectively. Similarly, (E) and (F) show vertical position and velocity traces. Blue and green lines show the traces of kinematic variables for the two moving stimuli. Black traces show the relatively long-latency, natural catch-up saccades to either stimuli on trials with no FEF stimulation, whereas red traces show the relatively short-latency saccades evoked by FEF stimulation. Eye velocity after the saccade matches exactly that of the target near its current position, whether the target was naturally chosen or chosen by FEF stimulation.
Figure 11 shows the predicted activation dynamics for two competing cells at each of five stages of the model during an episode in which an SAC choice overrides an SPEM choice. The series of panels, from top to bottom, corresponds to the path of information transfer, across involved stages, for communicating the choice from the SAC to the SPEM system. Thus, Figure 11A plots FEF planning neurons; Figure 11B, LIP saccadic neurons; Figure 11C, MSTv direction cells; and Figure 11D, FPA output averaging cells. Solid lines represent the traces of cells whose preferred directions match the stimulus that is eventually chosen as target by the SAC system. In Figure 11D, the FPA stage of the SPEM system initially chooses the other stimulus (dashed trace; see first arrow in Figure 11D). Note that the time at which the FPA flips its choice (second arrow in Figure 11D) is mere milliseconds after the FEF makes its choice. This allows the SPEM system to establish a good representation of the new target's velocity before the saccade is actually generated. This enables the postsaccadic SPEM to exhibit a velocity that is already matched to the target velocity.
Simulating how target selection by the SAC system overrides prior choice by the SPEM system. The figure shows model simulations of the paradigm used by Adler, Bala, & Krauzlis (2002). If SPEM tracks one of two moving stimuli before a saccade to the other, the postsaccadic SPEM always has a velocity appropriate to the saccadic target. Simulated activities of FEF planning (A), LIP saccadic (B), MSTv (C), and FPA (D) output (“averaging”) neurons are shown. The thick black line in each denotes the activity of a neuron containing the final target for SAC and SPEM within its RF. The dotted line represents the activity of a neuron containing the stimulus that attracted the initial SPEM within its RF. All activities are synchronized with target onset time. Arrows in each panel indicate the time when the first (and, if present, the second) sign of selectivity is observed. In (D), the first arrow indicates the SPEM system's initial target selection. The second arrow indicates SAC choice overriding the SPEM system's earlier selection. The SAC choice is conveyed from the FEF to the FPA via the LIP–MT–MST–FPA pathway (Figure 6). The timeline of arrows illustrates this transfer. The SAC choice bias starts to grow at 75 msec. This evolving bias is conveyed to the LIP and becomes visible around 90 msec, and appears in the MSTv at 110 msec. It takes another 40 msec for the FPA to flip its choice.
Simulating how target selection by the SAC system overrides prior choice by the SPEM system. The figure shows model simulations of the paradigm used by Adler, Bala, & Krauzlis (2002). If SPEM tracks one of two moving stimuli before a saccade to the other, the postsaccadic SPEM always has a velocity appropriate to the saccadic target. Simulated activities of FEF planning (A), LIP saccadic (B), MSTv (C), and FPA (D) output (“averaging”) neurons are shown. The thick black line in each denotes the activity of a neuron containing the final target for SAC and SPEM within its RF. The dotted line represents the activity of a neuron containing the stimulus that attracted the initial SPEM within its RF. All activities are synchronized with target onset time. Arrows in each panel indicate the time when the first (and, if present, the second) sign of selectivity is observed. In (D), the first arrow indicates the SPEM system's initial target selection. The second arrow indicates SAC choice overriding the SPEM system's earlier selection. The SAC choice is conveyed from the FEF to the FPA via the LIP–MT–MST–FPA pathway (Figure 6). The timeline of arrows illustrates this transfer. The SAC choice bias starts to grow at 75 msec. This evolving bias is conveyed to the LIP and becomes visible around 90 msec, and appears in the MSTv at 110 msec. It takes another 40 msec for the FPA to flip its choice.
DISCUSSION
Primates use a combination of SAC and SPEM to track a selectively attended moving object. Recent data on monkeys performing a target selection task indicate that each system has its own selection mechanism, but that in the event of a conflict, the SAC system's choice overrides the choice made by the SPEM system (Adler et al., 2002; Gardner & Lisberger, 2001, 2002; Horwitz & Newsome, 2001; Ferrera & Lisberger, 1995). This article and its companion (Grossberg et al., submitted) describe a neural circuit model that is capable of simulating these and a large number of additional data concerning how the SAC and SPEM systems interact to select and track targets among distractors in a coordinated way.
In our model, saccadic target selection emerges from interactions of frontal, parietal, and collicular regions with the BG. Visually responsive cells in the FEF, SC, and LIP show target/distractor discrimination at about the same time after target onset: FEF, 130–150 msec (Thompson et al., 1996; Schall, Hanes, et al., 1995); SC, 110–130 msec (McPeek & Keller, 2002); and LIP, 132 ± 2.3 msec (Thomas & Pare, 2007). However, when compared with respect to saccadic initiation time, t = 0, larger population of FEF and SC cells show earlier activity: FEF, 78% at t = −53 msec; SC, 98% at t = −45 msec; LIP, 60% at t = −26 msec. This accords with our claim that the FEF and the SC plan and execute a saccade and this information is passed to the LIP to coordinate and plan further eye movements.
The anatomical loci and processes that are most critical for SPEM target selection are still unsettled. Our model is based on the hypothesis of a partial parallelism between the SAC system and the SPEM system. Notably, SPEM target selection emerges as a result of interactions between the BG and a frontal area, the FPA. Stimulation of the FPA, or any other region downstream from it in the SPEM circuit, notably NRTP or a pursuit zone of the CBM, causes SPEM within a very short latency: FPA, 25–35 msec (Tanaka & Lisberger, 2002b; Gottlieb et al., 1993); NRTP, 20 msec (Yamada et al., 1996); CBM, 10 msec (Belknap & Noda, 1987). This is also true in the model. However, stimulating upstream from the FEF, at the MST or MT, modulates existing pursuit but never evokes SPEM from a voluntarily maintained fixation (Komatsu & Wurtz, 1989). The same would be true in the model if it were augmented to include a STOP signal channel to complement the GO signal channel. Such a STOP signal channel could suppress voluntary fixation responses to MT and MST stimulation in a manner analogous to that shown for the SAC system in Brown et al. (2004). These considerations reinforce the hypothesis of partial SAC–SPEM parallelism, and the more specific hypothesis that the voluntary gating associated with SPEM target selection occurs in the FPA, which connects with the BG–thalamus system in a way that is analogous to the FEF.
Before noting the limits of the parallelism in our model, it is useful to consider and evaluate an alternative view. There is growing evidence for rSC involvement in pursuit. Some nonfixation neurons in the rSC show target/distractor discrimination. Stimulation of rSC inhibited pursuit in the ipsilateral visual hemifield but had little effect on ongoing pursuit of targets in the contralateral hemifield (Basso, Krauzlis, & Wurtz, 2000; Krauzlis et al., 2000; Krauzlis, Basso, & Wurtz, 1997). Stimulation also increased the probability of selecting a stimulus that appeared contralateral to the site of stimulation irrespective of target motion direction (Carello & Krauzlis, 2004). The rSC thus shows two necessary properties for target selection: target/distractor discrimination and forced bias via stimulation. However, stimulation of the rSC does not generate SPEM, and the bias generated by the rSC stimulation is irrespective of the direction of target motion. One possible explanation for these two effects comes from the bilateral connections between the SC and the LIP. The LIP sends collaterals to the intermediate and deep layers of the SC (Lynch, Graybiel, & Lobeck, 1985) and gets return projections from the SC via the pulvinar (Clower, West, Lynch, & Strick, 2001). The LIP shows both these properties for SAC target selection and so, rSC activity might simply reflect the activity of foveal LIP cells during SPEM.
Although the model's SPEM and SAC subsystems each use a frontal–BG–thalamic loop for voluntary gating of target selection, there are some notable differences in the connectivity in the current version of the model. In the SAC system, FEF planning cells, SC buildup cells, and LIP cells all project to the BG–thalamus decision stage and help choose the target. Grossberg et al. (1997) modeled how activity representing multiple possible saccadic targets can coexist during the preparatory phase in the deeper layers of the SC, until competition determines the final choice of saccadic target. Gancarz and Grossberg (1999) further developed these ideas to predict and simulate how the FEF and the SC may work together to determine a saccadic choice. The current model further develops these concepts.
In particular, the FEF motor output cells are distinct from the planning cells, and do not become active enough to drive downstream SAC generators until they receive a GO signal from the BG–thalamus decision stage. In contrast, FPA averaging cells, which are here modeled as the FPA cells that send outputs to downstream SPEM generators, are the sole source of input to the SPEM part of the BG–thalamus decision stage, as well as the sole cortical recipient of the GO signal that initiates high-gain pursuit. Given a low threshold for SPEM initiation downstream from the FPA, the current model cannot avoid exhibiting a phase of low-gain vector-averaging SPEM before the BG–thalamus decision process forces a choice and initiates high-gain pursuit of whichever target direction representation survives the enhanced competition induced by the action of the GO signal.
This lack of full parallelism between the FEF–BG–thalamus and the FPA–BG–thalamus circuits is in accord with data indicating that vector averaging is much more common in the SPEM than in the SAC system. This may be because the visual consequences of SAC vector averaging are highly negative. The same is not true for SPEM vector averaging. In any case, some other variants of the SPEM circuit are also compatible with the data and computational constraints (Figure 6). For example, similar behavior can be achieved if both the MSTv-recipient WTA and averaging cells of the FPA project to the BG–thalamus decision stage, provided that weights and thresholds are adjusted accordingly. What would not work would be a projection from the model FPA's MSTd-recipient summation cells to the BG, if they have a large relative weighting. Such a circuit would exhibit mid-pursuit failures of the GO signal because the discharge rate of the summation cells declines sharply during maintained successful pursuit. This is because the MT cells that drive them lose their major excitatory input when successful pursuit cancels target motion in the retinal frame. In contrast, the MSTv cells (which ultimately drive FPA WTA and averaging cells in the current model) receive an efference copy signal that sustains their discharge during successful pursuit (Pack et al., 2001).
What pathway ensures coordination between the SAC and SPEM systems if their decisions are initially in conflict? The model embodies the hypothesis that the LIP possesses characteristics suitable for acting as a conduit for a multistep transfer of information between the SAC and SPEM systems. Alternatively, it might be proposed that SAC target choice could be transferred to the SPEM system in a single step, via a direct projection from the FEF to the FPA. Although these regions are nearby, no such direct projection is known. How such a projection might be made to work is also unclear because of incongruent representations at the FEF and the FPA. The FEF has a retino-centric vector map, such that each cell explicitly codes a saccade having a particular magnitude and direction and implicitly codes a single retinotopic locus. The FPA, on the other hand, has a directional map, such that each cell explicitly codes a preferred direction of SPEM motion regardless of the target's retinotopic locus. So any direct “alignment” could only be based on directions, not on retinotopic positions. As such, any would-be selective signal from the FEF to the FPA would be ambiguous: There might be many targets moving in the same direction, but at different positions on the retina.
To effect the transfer of choice from SAC to SPEM, the model instead incorporates the well-established projection from the FEF to the LIP to the MT. The transfer can be unambiguous as to retinal locus because the FEF, the LIP, and the SC all embody retino-centric maps. In the model, each region is represented by a 15 × 15 grid with the center [7, 7] representing the fovea. The LIP-to-MT projection can be thought of as a position-based attentional bias signal. This assumption is in tune with data that show MT/MST neurons increasing their activity if a target is present within their retinotopic receptive fields. Enhancement ranges from ∼9%, for random dot coherent motion stimuli (Seidemann & Newsome, 1999), to ∼25–35% for single-stimulus motions (Treue & Maunsell, 1999; Ferrera & Lisberger, 1995). In accord with this, model MT cells, whose receptive fields overlap the retinotopic position of the SAC target choice, get a boost in input. This accords with data that spatial cues reduce latency of eye movement generation more than form or motion cues (Adler et al., 2002). Of course, spatial cueing, by itself, is sometimes insufficient. Because the model has no feature or form representations, in its current form, it cannot simulate how the real SAC system could force a choice, based on color or form, between two moving stimuli falling within the same LIP receptive field.
Once the same stimulus is selected as a target, the SAC and SPEM systems must make further decisions and adjustments to produce the kind of tracking that optimizes visual and cognitive processing of the object. Even in the rare case of linear, constant-speed object motion, target speed can easily exceed the maximum SPEM speed so further decisions include when to generate catch-up saccades. Because catch-up saccades may overshoot, and because jump-back saccades degrade vision, further intelligent adjustments include briefly lowering pursuit gain (the ratio of SPEM velocity to target velocity) below unity, in order to allow the target to more quickly catch-up with the moving line of gaze. A companion paper about the model (Grossberg et al., submitted) treats a large number of such coordination examples and shows a good match between simulation results and experimental data.
Acknowledgments
Authorship is in rotated alphabetical order. D. B., S. G., and K. S. were supported in part by NSF grant SBE-0354378. S. G. was also supported in part by the Office of Naval Research (ONR N00014-01-1-0624).
Reprint requests should be sent to Stephen Grossberg, Department of Cognitive and Neural Systems, Boston University, 677 Beacon Street, Boston, MA 02215, or via e-mail: steve@bu.edu.