The Discrete and Continuous Brain: From Decisions to Movement—And Back Again

The nervous system faces a dual challenge in shaping behavior. To induce changes in the external world, it is necessary to contract muscles or secrete chemicals. Such processes necessarily involve the manipulation of continuous variables—muscle length or chemical concentration. In addition, animals must make decisions. To do so, they must entertain several different possible courses of action, or policies. Ultimately, they must select one of these actions or policies that are necessarily discrete. We draw on recent work that considers the interactions between the neuronal processing of discrete and continuous quantities (Friston, Parr, & de Vries, 2017). To make this more concrete, we focus on the oculomotor system. Sampling the visual world entails decisions about where to look and the implementation of these decisions by contraction of the extraocular muscles.

We use perceptual inference performed by the networks supporting eye movements as a way of motivating and illustrating the theoretical challenge we want to address. However, the treatment we offer generalizes to any system that involves the physical implementation of categorical decisions. The ideas presented in this article complement previous treatments of cognitive time (VanRullen & Koch, 2003), including the notion of a perceptual moment (Allport, 1968; Shallice, 1964; Stroud, 1967) and the suggestion that brain oscillations act as discrete clocks to support this type of computation (Buschman & Miller, 2009, 2010). They also resonate with recent developments in machine learning (Linderman et al., 2017) and some of the problems faced in modern robotics (Cowan & Walker, 2013; Schaal, 2006). In short, the coupling of categorical decision making and dynamic perception (and motor control) raises some deep questions about the temporal scheduling of perception (and action).

Oscillatory rhythms in measured brain activity have been linked to cyclical perceptual processes (Buzsaki, 2006), with theta and alpha cycles as the most popular hypothesized units of perceptual time (VanRullen, 2016). In endorsement of this, the timing of processing relative to the phase of certain oscillations appears to be important (Buzsáki, 2005). While there is some controversy concerning the frequency of the perceptual clock, an advantage to focusing on the oculomotor system is that we can evade this issue. The frequency of spontaneous saccadic sampling is around 4 Hz, allowing us to commit to a theta rhythm. Conveniently, this is the frequency often associated with attentional and central executive (decision) functions (Chelazzi, Miller, Duncan, & Desimone, 1993; Duncan, Ward, & Shapiro, 1994; Hanslmayr, Volberg, Wimber, Dalal, & Greenlee, 2013; Landau & Fries, 2012; VanRullen, 2013), as opposed to sensory processes associated with faster frequencies (Drewes & VanRullen, 2011; Dugué, Marque, & VanRullen, 2011; Ergenoglu et al., 2004; van Dijk, Schoffelen, Oostenveld, & Jensen, 2008).

The oculomotor system is a distributed network that includes brain stem, cortical, and subcortical regions (Parr & Friston, 2017a). An important point of contact between the cortical oculomotor networks and those in the brain stem is the superior colliculus (Raybourn & Keller, 1977), found in the midbrain. This structure receives a dual input from the cortex (Fries, 1984) and the basal ganglia (Hikosaka & Wurtz, 1983) and provides an important input to the brain stem oculomotor nuclei. In the following, we argue that the connectivity implied by active inference is consistent with a role for the superior colliculus as an interface between the discrete and continuous processing of the oculomotor system.

This article is organized as follows. In section 2, we review the principles of active inference, their application to discrete and continuous state spaces, and the relationship between the two. In section 3, we relate the computational anatomy implied by active inference to the neuroanatomy of oculomotion. In section 4, we illustrate oculomotor behavior, and its neural correlates, through simulation. Section 5 presents the discussion, and section 6 concludes.

Active inference is the process of minimizing variational free energy through action and perception (Friston, Samothrakis, & Montague, 2012; Friston, Daunizeau, Kilner, & Kiebel, 2010). The imperative to maintain a low free energy stems from the self-evidencing (Hohwy, 2016) nature of living systems. If an organism samples sensory input (e.g., blood pressure, pH, and temperature) compatible with life, this constitutes evidence for its existence as the sort of thing it is. It is intuitively sensible that living creatures should act so that they experience such sensations.

In more formal terms, the adaptive fitness of a creature's phenotypic state or trait “just is” the probability of a creature being in that state. This probability can be treated as model evidence by associating the creature with a model and its sensory exchange with the world with phenotypic or characteristic states. By casting exchange with the world in terms of a random dynamical system, it is fairly straightforward to show that the states that constitute the creature must, on average, increase model evidence; hence self-evidencing. (See Friston, 2013, for details.)

Active inference formalizes this notion by casting behavior as free-energy-minimizing or self-evidencing processes. Perception follows from recognizing or inferring the causes of sensory input by optimizing probabilistic representations with respect to free energy. Crucially, these causes include our own actions. This means action selection can be treated as “planning is inference” (Attias, 2003; Botvinick & Toussaint, 2012). In short, perceptual inference underlies action, active sensing informs perception, and both serve to minimize free energy or maximize model evidence.

This inference problem is illustrated as a Bayesian network in Figure 1. We first address inferences about the upper part of this network that selects target fixation locations. We then consider how these inform beliefs about sensory data and generate changes in the external world.

The first computations we address are those that mediate decisions about where to look. Decisions involve the selection of one from several alternatives—in our example, saccadic targets. As such, (Markov) decision processes are a natural form for the generative model because they are defined on discrete state spaces (FitzGerald, Schwartenbeck, Moutoussis, Dolan, & Friston, 2015; Friston, FitzGerald, Rigoli, Schwartenbeck, O'Doherty, et al., 2016; Friston, FitzGerald, Rigoli, Schwartenbeck, & Pezzulo, 2017; Friston et al., 2015; Friston et al., 2013; Friston et al., 2014; Friston, Lin, et al., 2017; Friston, Parr, et al., 2017; Friston, Rosch, Parr, Price, & Bowman, 2017; Mirza, Adams, Mathys, & Friston, 2016; Parr & Friston, 2017c, 2017d; Schwartenbeck, FitzGerald, Mathys, Dolan, Wurst, et al., 2015). Our latent variables are the hidden states of the world, $s$⁠, that represent fixation locations. Each policy, $π$ (course of action), that can be pursued is represented as a competing model. As time progresses, hidden states undergo transitions that depend on the state at the previous time. The transition between control states depends on the policy pursued. In our example, this means that the sequence of fixation locations is determined by the policy. Each state gives rise to an outcome, $o$⁠. We use the notation $o~=[o1,o2,…,oT]T$ to describe the sequence of outcomes through time. The outcome we are interested in here corresponds to “where I am looking.” Figure 2a shows graphically how one can factorize this kind of model.

At the level of biological effectors (glands and muscles) and sensory receptors, the variables the nervous system must deal with are continuous. In other words, to enact a selected policy, it is necessary to map categorical representations to the physical world with continuous states and time. Muscles generate continuous forces, while photoreceptors signal light intensity. This means that there must be a generative model that maps the abstract, discrete outcomes from the generative (MDP) model above, to continuous data (Friston, Parr, et al., 2017).

The ensuing location now plays the role of a latent or hidden cause that predicts the dynamics of continuous latent variables, $x˙$ (eye position, angular velocity), and observable sensory data, $y$ (vision and proprioception). The probability distributions that constitute the dynamic generative model are provided in Figure 2b. As in discrete state-space models, we use the tilde notation to indicate a trajectory. Here, however, the trajectory is represented as a vector of generalized motion (Friston, Trujillo-Barreto, & Daunizeau, 2008), $x~=[x,x',x''…]T$⁠. We will use $x[i]$ to refer to the $i$th order motion.

To recap, we have specified both discrete and continuous state-space models. We have shown how predictions of the former model can play the role of (empirical) prior beliefs in the latter. The final thing to specify is the process by which the continuous state-space model informs the discrete model. In other words, what sort of evidence is passed from the continuous to the discrete part of the (active) inference scheme? In brief, the discrete part of the generative model provides prior constraints on the continuous part, while the continuous part reciprocates with Bayesian model evidence for the discrete hypotheses entertained by the discrete part to enable Bayesian belief updating. This updating entails the selection of alternative hypotheses (outcomes) that constitute empirical priors at the continuous level.

To convert the evidence for each model back to discrete time, we integrate the model evidence (free energy) over the time period corresponding to one theta cycle. This is then combined with the prior over the model to give a vector (E) that can be passed through a softmax function to give the posterior over each outcome model, $rτ$ (Friston, Parr, et al., 2017). This plays the role of a discrete observation or outcome from the point of view of the MDP (see Figure 3).

This concludes our technical description of belief propagation between discrete and continuous parts of a generative model. A worked example of how this sort of belief propagation of message passing could work in the brain is provided in the final section (using the update equations in Figure 3). To motivate the interpretation of these simulations, we now consider the basic neurobiology of the oculomotor system and how its computational architecture could support belief propagation of this sort.

In section 2, we described the problem the brain faces in making discrete decisions about where to look and the continuous inferences required to realize and update these decisions. We outlined the computations mandated by active inference in solving these problems, with a special focus on the message passing between discrete and dynamic domains. In this section, we associate these computations with their neurobiological substrates. While this assignment is speculative, it is constrained by both the anatomy of message passing and the presence (or absence) of connections in the brain. Figure 3 shows the consistency between the computational anatomy of oculomotion and the networks known to support oculomotor function. In the following, we describe the cortical, subcortical, and brain stem components of this network (Parr & Friston, 2017a). This section concludes with an analysis of the superior colliculus, a structure uniquely placed to translate discrete decisions into target locations in a continuous state space.

The cerebral cortex is a laminar structure, with layer-specific projections and terminations (Felleman & Van Essen, 1991). The connectivity implied by inference using a Markov decision process closely resembles this pattern (Friston, Rosch, et al., 2017). Specifically, the inference scheme we have described involves several distinct types of variables that receive messages from a subset of the other variables. This implies a stereotyped pattern of connectivity between these groups (or layers) of computational units. Consistent with cortical laminae, external input targets only one layer. Outputs of different types arise from defined populations. In this section, we use known neuroanatomy to constrain the assignment of computational units to their appropriate laminae.

Layer IV of the cortex receives ascending connections from lower areas (Shipp, 2007), or from first-order thalamic nuclei. The computational units that receive this input are the error units, $ϵπτ$⁠, suggesting that these occupy this layer. This also implies that $rτ$⁠, the subcortical projection to layer IV, is likely to be represented by neurons in first-order thalamic relay nuclei, such as the lateral geniculate nucleus (Herkenham, 1980). Layer III gives rise to ascending connections. These are not shown here but would arise from neurons encoding the state $sτ$ at that hierarchical level (Friston, Rosch, et al., 2017). For simplicity, we consider a single cortical area, the frontal eye field, omitting the parietal (Corbetta et al., 1998; Gaymard, Lynch, Ploner, Condy, & Rivaud-Péchoux, 2003; Parr & Friston, 2017b; Shipp, 2004) and occipital (Bruce & Tsotsos, 2009) contributions to this system.

Layer V of the cortex has several subcortical targets (Kasper, Larkman, Lübke, & Blakemore, 1994; Ojima, Murakami, & Kishi, 1996). It is the layer that houses the pyramidal cells of Betz in the motor cortex that project to lower motor neurons in the spinal cord. In addition, layer V gives rise to projections to the second-order thalamic nuclei, such as the pulvinar, the superior colliculus, and the basal ganglia (Fries, 1985; Hübener & Bolz, 1988; Shipp, 2007). As Figure 3 shows, units encoding $oπτ$ send messages to all of these anatomical homologues. They participate in the evaluation of the expected free energy in the striatum, the model averaging of continuous time models by the superior colliculus, and Bayesian model reduction by the thalamus. That the $Eτ$ units of the thalamus receive cortical projections from layer V suggests that these neurons must be located in second-order thalamic nuclei (Crick & Koch, 1998; Rockland, 1998; Sherman, 2007).

As noted above, the basal ganglia receive input from cortical layer V, encoding predictions about discrete outcomes. This input, in addition to a signal from the error units, $ςπτ$⁠, is used to compute the expected free energy, $Gπ$⁠, of each policy. The basal ganglia are well recognized to be involved in policy evaluation (Gurney, Prescott, & Redgrave, 2001; Jahanshahi, Obeso, Rothwell, & Obeso, 2015). Most of the cortical inputs to the basal ganglia target the striatum (Alexander & Crutcher, 1990; Shipp, 2017), implying the expected free energy is represented by medium spiny neurons in this structure. These give rise to inhibitory GABAergic projections to the substantia nigra pars reticulata, which itself projects to the superior colliculus (Hikosaka & Wurtz, 1983). This direct pathway connectivity is remarkably consistent with the influence of $Gπ$ on $π$ and $π$ on $ϵ~v$⁠. The latter influence is in the Bayesian model averaging over expected outcomes to generate a prior mean, $η~$⁠, for the implementation of the policy in continuous time. The output nuclei of the basal ganglia participate in an additional Bayesian model averaging of hidden states. This is mediated by modulatory projections (via thalamic relays) to superficial layers of the cortex (Haber & Calzavara, 2009; McFarland & Haber, 2002).

The brain stem is the source of the cranial nerves to the extraocular muscles. This suggests that brain stem structures engage in continuous message passing. We have previously demonstrated that the anatomy of this message passing is not only consistent with the connectivity of the brain stem, but also that it reproduces electrophysiological responses in these structures, and the same deficits as in neurological patients when lesioned (Parr & Friston, 2018). In addition, these nerves carry proprioceptive information from the muscles (Cooper & Daniel, 1949; Cooper, Daniel, & Whitteridge, 1951), while the midbrain receives optic nerve fibers from the retinotectal pathway (Linden & Perry, 1983). Given beliefs about the current position and velocity of the eyes, $μ~x$⁠, it is possible to make predictions about the resulting sensory input (Adams, Perrinet, & Friston, 2012; Friston, Adams, Perrinet, & Breakspear, 2012; Perrinet, Adams, & Friston, 2014). This induces a sensory prediction error $ϵ~y$ that is minimized by action. This implies that the midbrain and pontine nuclei responsible for signals to the extraocular muscles must contain neurons that broadcast these errors. As the brain stem nuclei form the nodes of the network engaged in continuous inference, they must receive input from the region mapping decisions into this space. The obvious candidate for this region is the superior colliculus.

The superior colliculus is the interface between the forebrain and brain stem networks. It is the recipient of cortical (Hanes & Wurtz, 2001) and basal ganglia projections (Hikosaka & Wurtz, 1983; see Figure 4) and is intimately connected to the oculomotor system within the brain stem (Sparks, 2002). As such, it sits at the anatomical boundary between the discrete and continuous networks. It is found in the dorsal midbrain, at the same level as the oculomotor nucleus. Like the cortex, it is a laminar structure, with different electrophysiological responses in different subsets of cells. There are three broad groups of these neurons, as illustrated in Figure 4. These are the burst, fixation, and buildup cells (Ma, Graybiel, & Wurtz, 1991; Munoz & Wurtz, 1995a).

We have previously argued (Parr & Friston, 2018) that these groups correspond to three different types of computational unit. Burst cells, which fire at the start of a saccade, have the properties we would expect from neurons signaling visual prediction error, $ϵ~y$⁠. This is consistent with the fact that a subset of retinal ganglion cells synapses within the colliculus and that some collicular cells respond to visual stimuli (Mays & Sparks, 1980; Wurtz & Mohler, 1976). Fixation neurons are active during fixations, and we have associated these with the expectation neurons encoding target fixation locations, $μ~v$⁠. Consistent with the computational anatomy of Figures 3 and 4, it is this group that projects to the brain stem centers for saccade generation (Gandhi & Keller, 1997).

Buildup neurons show a pattern of activation consistent with a population encoding (Anderson, Keller, Gandhi, & Das, 1998; Lee, Rohrer, & Sparks, 1988). A traveling hill of excitation moves from a peripheral location toward the rostral pole of the colliculus during a saccade (Munoz & Wurtz, 1995b). At a population level, these neurons can be thought of as expressing a prediction error between a target fixation and the current eye position, $ϵ~v$ (Sparks, 1986). The movement toward the pole, representing the foveal location, can be thought of as encoding the reduction in prediction error as the eye moves closer to its target. That this occurs at the population level suggests that buildup neurons individually code for discrete spatial regions.

The discretized encoding of continuous variables by these units is consistent with their computational role, evaluating the difference between $η~$⁠, parameterizing competing hypothetical models, and the estimated position in continuous coordinates, $μ~v$⁠. Specifically, each neuron may encode the prediction error associated with a particular hypothesis, $η~m$⁠, with activity weighted by the prior probability of that hypothesis $om$⁠. The conversion from discrete to continuous coordinates then simply requires that the connection strengths between these neurons and the fixation neurons at the pole vary with distance. The inhibitory connections (Munoz & Istvan, 1998) from buildup to fixation neuron should be stronger if the anatomical distance between the two is greater. In summary, electrophysiological properties corroborate the neuroanatomical evidence that the superior colliculus is the discrete-continuous interface of the oculomotor system, and the topography of buildup and fixation neurons hints at the computational mechanisms that map between them. We next use the neurobiological pointers established in this section to interpret simulated oculomotor control in terms of established electrophysiological responses in the oculomotor system.

Our aim in performing these (minimal) simulations is to illustrate the interactions between the discrete and continuous domains of the oculomotor system. As such, we chose a simple behavioral paradigm with three possible fixation locations (left, right, and center). We then set the prior preferences (through the $C$ matrix) so that proprioceptive data are preferred that are consistent with central fixation initially, then with leftward fixation, rightward fixation, and finally central fixation again. This is consistent with the instruction to look at a sequence of targets at each of these locations. The structure of the model we used is as depicted in Figure 1. The continuous part employs the belief that the eyes are drawn toward an attracting location, and this is implemented by an action that has the effect of a Newtonian torque. The details of this can be found in Parr and Friston (2018) and are very closely related to those in previous formulations of oculomotion (Adams et al., 2012; Friston, Adams, et al., 2012; McSpadden, 1998; Perrinet et al., 2014). The discrete part is formally similar to that used in Mirza et al., 2016 and Parr and Friston (2017b, 2017c).

Figure 5 shows the results of simulating active vision in this model. Crucially, this type of simulation allows us to show what is happening at different neuroanatomical locations simultaneously and gives a sense of the functional interaction of these areas. The results are presented in the form of raster plots, as if we had recorded from single neurons in each of the simulated brain areas. Representations in the cortex and basal ganglia update on a theta cycle (4 Hz), while superior colliculus buildup cells translate this into continuous time for collicular fixation neurons. These induce (via brain stem circuits) changes in eye position, as shown in the simulated electro-oculography trace. Note that these neurons vary continuously with eye position, unlike the neurons in the discrete compartments, which encode the probability of an alternative fixation point.

Intuitively, we can see how the mapping from discrete to continuous occurs. At 250 ms, the cortex updates its representations. These cause the expected free energy under each policy to change, inducing updates in the striatum. Through the direct pathway, this causes inhibition in the substantia nigra pars reticulata, resulting in the selection of a new policy (“look right”). A Bayesian model average over outcomes then results in the selection of superior collicular buildup neurons that represent the error between the current belief about eye position (given by the fixation neurons) and the anticipated rightward location. These then induce changes in the fixation neuron activity. During the next 100 ms, as the eyes move to fulfill this belief, we can see the resolution of the prediction error in the superior colliculus buildup layer through the movement of the hill of activity toward the pole.

Variational message passing is not the only local message passing scheme that has been associated with neuronal signaling. Belief propagation (Pearl, 2014) is one alternative. The message passing under this approach is perfectly consistent with the free energy principle, as the requisite messages can be shown to minimize the Bethe free energy (Yedidia, Freeman, & Weiss, 2005). However, although attempts have been made to link belief propagation to neural signaling (Jardri & Denève, 2013), the resulting architectures turn out to be more complicated than the equivalent variational schemes. The primary reason for this is the recursive computation of such messages. Each message is derived from another message rather than from posterior beliefs about each variable. Variational messages passed from one population of neurons are derived from the posterior beliefs represented by these neurons, mapped through a set of synaptic weights. This means that the same set of neurons can be used to compute multiple different messages. In contrast, a recursive scheme requires streams of forward and backward messages that do not directly interact.

A further possibility is that the messages we have described might be computed through sampling, or Monte Carlo, approaches (Hastings, 1970). One argument against this is that these methods tend to take much longer to converge. In a biological setting (requiring online inference), this means that animals would spend more time in surprising states (associated with a lower model evidence). In short, while Monte Carlo approaches may achieve a greater accuracy, they induce a larger (path integral of the) free energy than variational approaches.

The approach we have described here is capable of reproducing a wide range of physiological and behavioral phenomena in the oculomotor system and more generally. In the context of the oculomotor system, we have previously shown that the signals we have simulated bear a close resemblance to those measured in brain stem nuclei (Parr & Friston, 2018). Most striking, we found that simulated collicular buildup neuron responses qualitatively reproduced single-unit recordings published in the experimental literature (Munoz & Wurtz, 1995b).

Lesions to these models induce similar behavioral syndromes to those found in neurological patients with damage to the associated neuroanatomy. By disrupting neuronal message passing (i.e. inducing disconnection syndromes; Geschwind, 1965), we have simulated visual neglect (Parr & Friston, 2017b) and internuclear ophthalmoplegia (Parr & Friston, 2018). The white matter disconnections associated with these syndromes are the superior longitudinal fasciculus (Doricchi & Tomaiuolo, 2003) and the medial longitudinal fasciculus (Virgo & Plant, 2017), respectively. The locations of these synthetic lesions constrain the computational anatomy, and their nature endorses the notion that the brain engages in variational inference.

More generally, models based on active inference have a high degree of face validity, in that they reproduce a wide range of neurobiological phenomena. These range from single cell responses, including place fields (Friston, FitzGerald, Rigoli, Schwartenbeck, & Pezzulo, 2017) and midbrain dopamine activity (Friston et al., 2014), to evoked responses, including those associated with classic working memory tasks (Parr & Friston, 2017d). They have been used to generate behaviors as diverse as exploration (Friston et al., 2015; Mirza et al., 2016), handwriting (Friston, Mattout, & Kilner, 2011), eye-blink conditioning (Friston & Herreros, 2016), habit formation (FitzGerald, Dolan, & Friston, 2014), communication (Friston & Frith, 2015), and insight (Friston, Lin, et al., 2017). In addition to these theoretical accounts, active inference has been used pragmatically to model behavior and to characterize individuals according to the parameters of their prior beliefs (Adams, Bauer, Pinotsis, & Friston, 2016; Mirza, Adams, Mathys, & Friston, 2018; Schwartenbeck & Friston, 2016).

The issues described in this article generalize beyond eye movements. Any neurobiological system that needs to make decisions and implement these via some physical effector must solve the problem we have described here. This is vital for (but not exclusive to) speech, locomotion, and autonomic regulation. Language is made up of discrete units (phonemes, words, sentences) that are expressed as continuous changes in auditory frequencies generated by contraction of the laryngeal (and pharyngeal) muscles (Simonyan & Horwitz, 2011). Walking involves taking a series of discrete steps, each of which requires a careful coordination of skeletal muscles (Ijspeert, 2008; Winter, 1984). Interoceptive states are frequently divided into discrete dichotomies including fed versus fasting (Kalsbeek, la Fleur, & Fliers, 2014; McLaughlin & McKie, 2016; Roh, Song, & Kim, 2016), diastole versus systole, and sympathetic versus parasympathetic (McDougall, Münzberg, Derbenev, & Zsombok, 2014; Owens, Friston, Low, Mathias, & Critchley, 2018). Each of these induces continuous changes in enzyme activity, blood pressure, or smooth muscle contractions. The form of the variational message passing will be very similar for each of these processes, but the variables represented will differ. This suggests a similar pattern of cortico-subcortical connectivity, but differing regions of cortex, and different subcortical components.

In this article, we have chosen to focus on a fairly concrete problem: deciding where to look and how to do this. For more abstract decisions, perhaps at higher hierarchical levels in the brain (Badre, 2008; Badre & D'Esposito, 2009; Christoff, Keramatian, Gordon, Smith, & Mädler, 2009; Rasmussen, 1985), it may be necessary to integrate beliefs across multiple modalities. A challenge for future work is to incorporate the set of beliefs that constitute an emotional state, as emotions are often thought to contribute to “irrational” behaviors. It is not always easy to intuit how such behaviors might be Bayes optimal.

One line of research into these issues frames them as questions about interoceptive inference (Ondobaka, Kilner, & Friston, 2017; Seth, 2013; Seth & Friston, 2016). Given beliefs about (abstract) variables that have both interoceptive and exteroceptive sensory consequences, it becomes clear that policies must minimize expected free energy in both domains. For example, a belief that a predatory animal is present implies that the sympathetic nervous system should be active, but also that visual data are consistent with the presence of said animal. Anatomically, these dependencies are consistent with the sensory and autonomic targets of the amygdala (LeDoux, Iwata, Cicchetti, & Reis, 1988; Ressler, 2010). A tachycardia then carries (weak) evidence for the presence of a scary animal and could influence policy selection even in the absence of exteroceptive evidence. This suggests a framework in which an emotional state may influence decision making in an apparently irrational way that is entirely compatible with the formulation we have described here.

We hope to further the ideas in this work both theoretically and empirically. Two important theoretical issues need to be addressed in greater depth than we have space for here. The first is a thorough comparison of alternative message-passing schemes in relation to their anatomical and physiological plausibility. The second is a generalization of the inferences required for oculomotor decisions (and their motoric implementation) to other systems. While the ideas we have presented are generally applicable, it will be necessary to specify the generative models required to solve locomotive, autonomic, and abstract decision-making problems. Finally, although the computational anatomy we have proposed has a high degree of face validity, it will be necessary to establish its predictive validity.

One way to do so would be to use computational fMRI (Schwartenbeck, FitzGerald, Mathys, Dolan, & Friston, 2015), fitting this model to oculomotor behavior, and looking for brain regions that show activity patterns consistent with the simulated neuronal responses. We hypothesize that these regions will match the computational anatomy illustrated in Figure 3. An alternative would be to use single-unit responses from each brain area recorded during an oculomotor task. One could then compare each simulated neuronal response to each recording and construct a confusion matrix of the evidence for each synthetic signal in each region. We would expect a greater evidence for each signal that we have associated with each region above.

In this article, we have described the discrete and continuous message passing that must be performed in an oculomotor system that realizes a sequence of saccadic fixations. We have illustrated the remarkable consistency between the belief propagation or message passing implied by active inference and the anatomy of the oculomotor system. This accounts for several neuroanatomical observations, including the dual input from frontal eye fields and the substantia nigra to the superior colliculus, and the common laminar origin of axons that target the striatum, second-order thalamus, and midbrain tectum. Finally, we simulated electrophysiological responses as saccadic targets are selected and as the eyes move to implement that saccade. This shows, functionally, how the superior colliculus is uniquely positioned to act as the interface between the discrete and continuous oculomotor systems.

Although the generative model changes from application to application, the belief updates described in this article are generic and can be implemented using standard routines (here spm_MDP_VB_X.m). These routines are available as Matlab code in the SPM academic software: http://www.fil.ion.ucl.ac.uk/spm/. Simulations of the sort reported above can be reproduced (and customized) via a graphical user interface by typing in $>>$ DEM and selecting the ‘visual foraging’ demo.

T.P. is supported by the Rosetrees Trust (award 173346). K.J.F. is a Wellcome Principal Research Fellow (088130/Z/09/Z). We are grateful to two anonymous reviewers for their helpful suggestions.

We have no disclosures or conflict of interest.