A Cerebellar Computational Mechanism for Delay Conditioning at Precise Time Intervals

With the evolution of vertebrates came the problem of combination of sensory signals and coordination of motor commands that arrive from and are destined for distant parts of the body. Different body regions are connected to the central nervous system by neurons whose axons are responsible for differing delays in transmission. This creates the need for recognition and generation of signals with precise time delays. While some delays are determined by neuroanatomy and therefore unchanging, the need to interact with a dynamic environment requires the ability to learn and replicate arbitrary time intervals (Molinari, Leggio, & Thaut, 2007). This function is often attributed to the cerebellum (Bareš et al., 2019), and cerebellar injury impairs the estimation and production of millisecond-precision time intervals up to about 1 to 2 seconds in duration (Buhusi & Meck, 2005; Ivry & Spencer, 2004; Schwartze, Keller, & Kotz, 2016). A common experimental paradigm is eyeblink conditioning, in which a behaviorally irrelevant conditioned stimulus (CS) is repeatedly followed at a predictable interval by a puff of air on the cornea (unconditioned stimulus, US). After training, presentation of the CS without the US leads to an eyeblink at the precise interval at which the US would normally have occurred (Attwell, Ivarsson, Millar, & Yeo, 2002; Gerwig, Kolb, & Timmann, 2007; Kim & Thompson, 1997; Kotani, Kawahara, & Kirino, 2002; Raymond & Medina, 2018).

The mechanism by which time intervals can be learned and replicated is the subject of multiple models (Yamazaki & Tanaka, 2009). Proposed mechanisms include delays implemented by the time required for axonal conduction or conduction through neural circuits (Buonomano, 2003; Braitenberg, 1967), rhythmic pattern generators with a known period (Miall, 1989; Panzeri, Ince, Diamond, & Kayser, 2014), beat frequency between oscillators at differing frequencies (Buhusi & Meck, 2005), synchronized firing of groups of neurons (Hopfield & Brody, 2001), or superimposed time-varying waveforms that can be combined to produce temporal sequences (Medina, Garcia, Nores, Taylor, & Mauk, 2000; Narain, Remington, De Zeeuw, & Jazayeri, 2018). A particularly interesting group of models suggests that random but repeatable dynamics can be generated by feedback loops between granule cells and Golgi cells, and the resulting time-varying patterns triggered by the conditioned stimulus allow determination of time intervals (Roössert, Dean, & Porrill, 2015; Yamazaki & Tanaka, 2007a, 2007b). While most of these mechanisms are able to approximate smooth temporal sequences, they do not have the ability to learn a rapid eyeblink at an arbitrary time interval with millisecond precision because they are based on average firing rates rather than instantaneous spikes (Karmarkar & Buonomano, 2007). The precision, repeatability, and stability of rate-based models may not reflect the true challenges of achieving millisecond precision timing with interconnected populations of spiking neurons Even in a model based on simulation of populations of spiking neurons, time-varying patterns in granule cells are expressed as relatively slow variations in firing rates that cannot be identified at millisecond precision (Yamazaki & Tanaka, 2007b).

We propose a new mechanism for learning time intervals with millisecond precision. This mechanism is based on patterns of activity within populations of granule cells, but the patterns are determined by predictable and repeatable (although random-appearing) sequences of individual spikes rather than by slower temporal patterns of spike rates, oscillations, or synchrony. The mechanism requires that cerebellar granule cells are able to produce nearly identical spike trains for similar inputs and that the Purkinje cells can recognize specific patterns in the population of granule cells. If these two conditions are met, then the pattern of spikes in the population provides a unique and precise time stamp at any desired time interval, and the association of a training signal in the climbing fibers at a particular time interval after the stimulus allows plasticity at the parallel fiber to Purkinje cell synapses to respond to the pattern of activity at that specific time. There is accumulating evidence that even though cell spike sequences appear to have Poisson statistics, they may not be random and often have the repeated sequences required for this mechanism (Alstrøm, Beierholm, Nielsen, Ryge, & Kiehn, 2002; Hires, Gutnisky, Yu, O'Connor, & Svoboda, 2015; Kayser, Logothetis, & Panzeri, 2010; Mainen & Sejnowski, 1995; Masquelier, 2013; Nolte, Reimann, King, Markram, & Muller, 2019; Petersen, Panzeri, & Diamond, 2001, 2002; Shmiel et al., 2005). This is consistent with theoretical results showing that for certain classes of nonlinear systems, apparently chaotic but nevertheless repeatable sequences can be achieved (Rössert et al., 2015; Uchida, McAllister, & Roy, 2004). Although this phenomenon has not yet been tested in granule cells, we suggest that the identification of time-stamped patterns of neural activity may provide a widespread mechanism for the representation of time intervals and the synchronization of events with repeatable time delays within the cerebellum, as well as elsewhere in the brain.

The model and simulations we propose we have here are intended to demonstrate the potential for this computational mechanism to generate and recognize precise time intervals in response to pairings of CS and US. Although this can be perhaps interpreted as a model for certain behavioral phenomena including eyeblink conditioning, we emphasize that we do not attempt to emulate many of the more complex and interesting behavioral phenomena associated with eyeblink or other conditioned reflexes, we do not attempt to match the empirically observed firing patterns of granule or Purkinje cells, and the time intervals we have chosen are intended for illustration rather than as a specific instance of conditioning. Our intent is to show in a very simplified model that there is a previously undescribed computational mechanism based on spike timing that has the potential to be used to generate time intervals and response patterns with millisecond precision after relatively short periods of training.

We emulate a delay conditioning paradigm in which the conditioned stimulus (CS) and unconditioned stimulus (US) overlap in time. This is because in the absence of overlap (trace conditioning), the response may depend on other brain regions, including the hippocampus (Beylin et al., 2001; Kim, Clark, & Thompson, 1995; Weiss, Bouwmeester, Power, & Disterhoft, 1999), although more recent results suggest that trace and delay conditioning may share mechanisms at the Purkinje cell (Halverson, Khilkevich, & Mauk, 2018). In our model as well as most other models that depend on temporal patterns of granule cell activity, in the absence of overlap between CS and US, the CS input drive from the mossy fibers to the granule cells is absent at the time of the US, so the parallel fibers to Purkinje cell synapses are not activated and learning cannot be performed.

To estimate the probability that an untrained pattern could be mistaken for a pattern present during the US, we calculate the cross-correlation matrix between each different pattern. We compare the magnitude of the diagonal elements of the cross-correlation matrix (responding to the correct pattern) with the magnitude of the off-diagonal elements (responding to an incorrect pattern).

Because neither the granule cell to Purkinje cell synaptic weights nor the granule cell inputs can represent negative numbers, orthogonality of different granule cell patterns occurs only when there is no overlap between patterns. This substantially reduces the number of possible orthogonal patterns representable in any population. At least 2000 granule cells appear to be needed for reliable results in our simulation. Since those cells are driven by only 100 mossy fiber inputs, there needs to be a method to distribute the inputs randomly across the granule cells. Expansion coding is necessary in our simulation in order to allow a rich enough representation to achieve a unique pattern on the parallel fibers for each stimulus at each time.

The very simple LTD/LTP learning algorithm emulated here effectively “deletes” the response to granule cells that tend to be on during the US. However, once deleted, any subset of the deleted set will also lead to a fall in Purkinje cell firing. Thus normalization is necessary for the small numbers of granule cells in our simulation to ensure that random coincidences of pauses in granule cell firing do not automatically lead to low Purkinje cell firing that would be misinterpreted as a conditioned response (CR). This mechanism predicts dense granule cell firing (Badura & De Zeeuw, 2017), since a sparse code will frequently have very few or absent input firing, which would lead to lack of Purkinje cell input that would be mistaken for the decreased firing associated with the CR. Similar to other algorithms, this mechanism is not a model for trace conditioning, since if the CR occurs following the end of the CS, then the granule cells would not be driven by the CS and both granule cells and Purkinje cells would cease firing.

A reasonable analogy is a lock-and-key mechanism. The Purkinje cell synapses are a lock that causes the Purkinje cell to fire for almost all incoming patterns of granule cells. Only when the particular incoming pattern is the key that exactly matches the synapses that have been silenced by LTD does the Purkinje cell stop firing. Normalization (perhaps performed by Golgi cells or other cerebellar interneurons) is essential to ensure that a blank “skeleton” key with little or no granule cell activity would not be able to emulate a truly matching key.

Successful function of the mechanism we propose depends on several details of the network, without which the conditioned response can be incorrect or absent:

1. The pattern of granule cell spike timing during the CS is similar on each repetition of the CS.

2. There is high variation between the spike time patterns of different granule cells.

3. Purkinje cell inputs are normalized.

In order to achieve the first detail, the mossy fiber input needs to have a reliably repeatable pattern for each repetition, there must be no other noise input to the granule cells, and there must be a sufficient intertrial interval that the granule cell membrane potentials can return to their original resting state. In order to achieve the second one, expansion recoding of the mossy fiber input is needed to create a sufficient number of differing spike trains from the mossy fiber input, and the US must occur at a sufficient time interval following onset of the CS that initial transient-correlated responses in the granule cell population have abated. In order to achieve the final detail, there must be either a very large number of granule cells so that in the absence of the US, the mean Purkinje cell EPSP has low variance, or there must be feedback onto the granule cells that maintains the total granule cell activity approximately constant.

The algorithm proposed here shares important features with prior timing algorithms based on extraction of information from time-varying population responses to the CS input. Tripp and Eliasmith (2006) show that arbitrary output patterns can be computed based on apparently random input patterns similar to those used in our simulations, but they did not implement a cerebellar-like learning rule, and although their results depend on reliable spike timing within the patterns, they do not discuss or simulate mechanisms that permit such timing. Medina et al. (2000) provide a detailed simulation of cerebellar circuitry that estimates temporal patterns as linear combinations of the granule cell patterns. Repeatable patterns of granule cell average firing rates (but not precise spike timing) are induced by variations in network properties within the cerebellar granule and molecular layers. Yamazaki and Tanaka (2007b) demonstrate a spiking neural model with random interconnectivity between granule and Golgi cells that leads to a repeatable but random pattern of firing rates, and they show that a spike-timing-dependent plasticity rule at the parallel fiber to Purkinje cell synapse leads to the ability to learn time intervals in a delay conditioning task. The major difference from the work presented here is that these prior models depend on Purkinje cells detecting (random or known) temporal patterns of spike rates but not the spikes themselves (Rössert et al., 2015). Therefore, the temporal precision of such models is limited because determination of a spike rate requires the generation of multiple spikes over time and cannot respond at the millisecond level of precision. For example, rate-dependent models cannot generally model the ability of Purkinje cells to respond to two different US timings (Jirenhed et al., 2017), whereas we have shown that our model can yield both multiple responses as well as patterned responses at precise intervals (see Figure 6).

Although the granule cell spike trains are repeatable, they appear to have Poisson statistics. A simple rate code that looks at average firing but ignores the temporal pattern would identify the average spatial pattern of activity on the mossy fibers, and this could potentially confuse different conditioned stimulus inputs. By responding to the precise pattern of spikes, the Purkinje cell has the ability to recognize not only the pattern of the CS but the timing as well. Thus, the temporal pattern of spikes on the granule cells identifies both the what and the when of the incoming input data stream.

The model presented here is not intended to be a complete model of cerebellar function or the behavior of eyeblink delayed conditioning tasks. For instance, known phenomena, including the effect of interstimulus interval on learning, extinction of learned eyeblink responses, eligibility traces, and changes in amplitude and reliability, are not modeled (Bareš et al., 2019). The possibility that Purkinje cells have intrinsic time discrimination properties is not modeled (Johansson, Jirenhed, Rasmussen, Zucca, & Hesslow, 2014). The time intervals chosen (100 msec CS with ISIs of 40 or 70 msec) are not intended to be representative of the values seen in eyeblink conditioning; rather, they are intended as examples of high-precision timing responses that can be achieved by this mechanism. Our purpose is to demonstrate the feasibility, precision, and capability of this previously undescribed mechanism without proposing that it is the sole explanation for the complexity of observed learning behaviors.

This model depends critically on the presence of repeatable spike patterns in response to repeatable inputs. Although we believe that individual cells do not have true random effects, there are sufficient numbers of inputs to many cells in the central nervous system that truly repeatable spike patterns may not occur. Despite this concern, there is emerging evidence of specific instances in which repeatable patterns have been found. Our results suggest that whenever this occurs, whether within or outside the cerebellum, there is the potential ability to identify and respond to precise time intervals identified by the specific patterns of spikes that occur over the population at each time. Therefore, this mechanism has the potential to form a more widespread interval timer that could be used for synchronization, sequence generation, recognition and matching of intervals in sensory data, and production of rhythmic sequences. It is tempting to speculate that such a mechanism could be used to fill in the gaps between successive peaks in the theta rhythms generated by thalamus that are thought to provide a global synchronization signal. This would allow the relatively low-frequency theta rhythm to provide a global clock that could be subdivided locally to provide a high-resolution synchronization signal suitable for sensory-motor responses of animals with large bodies and variable peripheral transmission delays.

Support for this project was provided by Riken AIP, ATR, and the USC Department of Biomedical Engineering.