Spike synchrony, which occurs in various cortical areas in response to specific perception, action, and memory tasks, has sparked a long-standing debate on the nature of temporal organization in cortex. One prominent view is that this type of synchrony facilitates the binding or grouping of separate stimulus components. We argue instead for a more general function: a measure of the prior probability of incoming stimuli, implemented by long-range, horizontal, intracortical connections. We show that networks of this kind—pulse-coupled excitatory spiking networks in a noisy environment—can provide a sufficient substrate for stimulus-dependent spike synchrony. This allows for a quick (few spikes) estimate of the match between inputs and the input history as encoded in the network structure. Given the ubiquity of small, strongly excitatory subnetworks in cortex, we thus propose that many experimental observations of spike synchrony can be viewed as signs of input patterns that resemble long-term experience—that is, of patterns with high prior probability.
Depending on the behavioral context, specific cortical neurons fire in synchrony. In the sensory cortex, this depends on qualities of the sensory input: sounds evoke simultaneous activity in auditory cortex cells with matching receptive fields (Brosch, Budinger, & Scheich, 2002; Atencio & Schreiner, 2013), distant cells in somatosensory cortex synchronize when particular skin regions are stimulated (Reed et al., 2008), and synchrony in the primary visual cortex (V1) varies with geometrical stimulus features such as spatial continuity (Gray, König, Engel, & Singer, 1989; Livingstone, 1996) or similarity of orientation (Kohn & Smith, 2005). This can be observed as soon as 30 ms after a stimulus change (Maldonado et al., 2008), that is, within only a few spikes. Beyond sensory cortex, spike synchrony in the primary motor cortex varies with either the performed (Jackson, Gee, Baker, & Lemon, 2003) or the intended action (Riehle, Grün, Diesmann, & Aertsen, 1997; Pipa, Riehle, & Grün, 2007), and it has has been shown that distant prefrontal cortex cells synchronize selectively during memory tasks (Pipa & Munk, 2011).
Interpretations of such results have been conflicting. They have either been seen as an (inconsequential) epiphenomenon of cortical connectivity (Shadlen & Movshon, 1999) or as evidence for a synchrony-based mechanism to quickly assemble and group different sources of currently relevant information (Roelfsema, Engel, König, & Singer, 1996; Gray, 1999). Regardless of their function, the neural basis of these effects is mostly sought in long-range, horizontal, intracortical connections (Stettler, Das, Bennett, & Gilbert, 2002; Li & Gilbert, 2002). Horizontal connections are known to adapt to experience during development (Galuske & Singer, 1996; Schmidt, Galuske, & Singer, 1999) and in adult learning (Rioult-Pedotti, Friedman, Hess, & Donoghue, 1998). In the case of V1, their structure has been found to reflect the aggregate statistics of natural visual scenes (Onat, Jancke, & König, 2013). Perhaps as a result of such adaptations, the fine-scale topology of these networks is complex and functionally heterogenous (Martin, Roth, & Rusch, 2014; Rothschild, Nelken, & Mizrahi, 2010); connections between cells with differing response properties are common. However, one of the few broad regularities in these networks appears to be that cells that respond in similar contexts tend to be well connected. This holds across many cortical areas. For example, direct horizontal connections are found between distant auditory cortex cells with similar response selectivity (Read, Winer, & Schreiner, 2001) and between primary motor cortex cells representing related muscle groups (Weiss & Keller, 1994). V1 cells with nearby receptive fields are preferentially connected, and even more so where they select for similar visual orientations (Ts'o, Gilbert, & Wiesel, 1986; Stettler et al., 2002). V1 connectivity may further favor cells whose receptive fields fall in line with the axis given by their orientation tuning (Bosking, Zhang, Schofield, & Fitzpatrick, 1997, but see Martin et al., 2014). In somatosensory cortex, horizontal connections occur between distant cells that respond to sensations at opposing fingertips (Négyessy et al., 2013). This illustrates that horizontal connectivity is not simply determined by receptive field similarity, but more generally seems to favor cells that are activated jointly in common sensorimotor contexts (such as handling an object between two fingers or seeing a continuous line). Conversely, these are contexts that activate well-connected groups of cells.
To provide a concrete illustration, we briefly turn to the visual cortex. V1 cells are known to respond selectively to stimuli with retinal coordinates matching their cortical position (retinotopy) and with a particular angle (orientation tuning). Consider a visual pattern formed by various short line segments. In case these elements are scattered across the visual field, retinotopy implies that a spatially scattered set of cells is activated. Few of these cells will have direct connections, since horizontal connections preferably connect cells with nearby receptive fields. More generally thus, the shortest path between any two responding cells will likely be longer, on average, for such a scattered stimulus than for a more compact stimulus. A similar illustration can be made with respect to orientation. Consider a chain of line segments. In case their orientations are aligned, a spatially neighboring set of cells with similar orientation tuning responds. The cited patterns in V1 connectivity suggest denser connectivity between these cells than between a more heterogenously tuned set of cells that would respond to a chain of heterogenously oriented segments. Again, depending on the distances, this may translate to a difference in the length of paths between the activated cells. A much more complete, idealized, geometrical model of V1 horizontal connection patterns in relation to visual grouping has been provided by Ben-Shahar and Zucker (2004).
In summary, it appears that the cortex in certain situations receives activation patterns that match the existing lateral network structure particularly well, in the sense that they activate closely connected groups of cells. Further, the network structure appears to reflect experience in the sense that commonly coactivated units are better connected. Thus, to show that synchrony reflects the similarity of an incoming spatial pattern to the patterns commonly seen in the past, one primarily needs to show that synchrony reflects the degree to which the current network connects the cells activated by that pattern. The question, therefore, is why, in this case, there is an increase in spike synchrony.
It seems unlikely at first sight that the discussed horizontal network by itself explains the synchrony effect at hand: These connections establish an excitatory (Ts'o et al., 1986; Sincich & Blasdel, 2001), pulsed (chemical) coupling of spiking cells (McGuire, Gilbert, Rivlin, & Wiesel, 1991). In theoretical models of such networks, synchrony, if possible at all, has been found to be unstable in the sense that the phases of different cells spontaneously align and disperse in reaction even to small disturbances (Ernst, Pawelzik, & Geisel, 1998), depending on delay durations, connection strengths, and the network topology (Pérez et al., 2011).
Consequently, at the heart of all existing spiking network models of cortical synchrony are oscillators formed by the interaction of excitatory and inhibitory cells, in which the latter counterbalance the destabilizing effects of excitation (see Wilson & Bower, 1991; Börgers & Kopell, 2005). Here, we argue that the reported findings of cortical synchrony by themselves do not require or imply the presence of excitatory-inhibitory oscillators.
We mentioned that an important aspect of cortical synchrony is its dependence on spatial features of incoming stimulus patterns. The model by Wilson and Bower (1991) is notable for replicating one such effect (Gray et al., 1989) in an excitatory-inhibitory spike-based model. Richer stimulus-dependent synchrony effects are found in a group of models that abstract away from the (spiking) dynamics of biological networks, aiming for an interpretation of cortical synchrony as a binding mechanism. Specifically, these models demonstrate the emergence of synchronous cell assemblies that reflect large-scale, spatial relations within a stimulus, which, for example, leads to an elegant, self-organized solution for visual grouping or image segmentation problems (Li, 1998; Yen & Finkel, 1998; Schillen & König, 1994; Wang, 1995; Wang & Terman, 1995; Finger & König, 2013). These synchronization effects seem to rely on properties of continuously coupled rate or phase models, as they have not yet been demonstrated in pulse-coupled networks of spiking cells. A more exotic approach to achieve synchronous cell assemblies is to reconfigure the network structure for each new stimulus pattern (Yen & Finkel, 1998; Wang, 1995; Yu & Slotine, 2009), though this is biologically motivated in only one case (Wang, 1995) to a certain degree. Finally, it is interesting to note that VanRullen, Delorme, and Thorpe (2001) criticized the whole category of models cited here as implausibly slow for the purpose of stimulus grouping and proposed a fast, feedforward, spike-based grouping model without regard to synchronization.
Here, in an attempt to find a minimal model that has the required computational properties, we provide conditions under which fixed, purely excitatory, chemically coupled spiking networks alone exhibit the required, fast stimulus-dependent synchrony response. One relevant property of such networks is that they can produce temporally ordered spike responses in the presence of noise. Perhaps counterintuitively, noise appears as a beneficial factor in various biological contexts, such as improved neural signal transmission or increased spike time reliability (Ermentrout, Galán, & Urban, 2008; McDonnell & Abbott, 2009). In particular, excitable systems—systems such as spiking neurons, whose response to a sufficient input is stereotypical and followed by a refractory period—are known to engage in periodic oscillations when they receive noise input of some optimal, nonzero amplitude (Lindner, García-Ojalvo, Neiman, & Schimansky-Geier, 2004). This effect (coherence resonance or autonomous stochastic resonance) has been shown in the FitzHugh-Nagumo (Pikovsky & Kurths, 1997), Hodgkin-Huxley (Lee, Neiman, & Kim, 1998), and Morris-Lecar (Balenzuela & García-Ojalvo, 2005) neuron models, and it extends to arrays of multiple, sufficiently coupled spiking neurons—that is, to excitable media (Neiman, Schimansky-Geier, Cornell-Bell, & Moss, 1999). Such networks, when driven by random input, can display regular traveling waves, among other spatially organized activation patterns. We build on these findings and show that this type of emergent, spatial organization of synchronous activity in excitatory networks allows a fast synchrony-based measure of the match between incoming spatial activation patterns and the current network topology.
This letter is structured as follows. We first show that random spike input is transformed to synchronous responses in various excitable, pulse-coupled networks, in particular when stimulating well-connected subpopulations. This basic effect is then used to demonstrate synchrony as a measure of similarity between incoming stimuli and the long-term stimulus history as reflected in current network structures (“familiarity”). Additionally, we show that the effect is compatible with Gray et al.'s (1989) observation of stimulus dependent synchrony and provide some conditions for its occurrence, and we close with a comparison to known cortical dynamics.
We study networks of single-compartment Izhikevich spiking neurons (Izhikevich, 2003) with excitatory chemical (pulsed) coupling. A subpopulation receives uncorrelated, stationary, random (Poisson) input spike trains on excitatory and, optionally, inhibitory synapses. As the network responds, the degree of zero time lag spike synchrony of a particular group of cells is measured. This basic setup is shown in Figure 1.
The model neuron is set to a regime of integrating, class 1 behavior, firing at a much lower rate than the rate of external input pulses (a feature shared by many cortical neurons; Shadlen & Newsome, 1998). Depending on connectivity, networks of such neurons can generate synchronous responses. Figure 2 illustrates this by showing the spike activity in a 4 10 lattice network with eight-nearest-neighbor connectivity, receiving random input in a central, fixed subpopulation of 12 units. The different panels show the network's response given increasing lateral synapse conductances . Coherent spiking already emerges at lower conductances, followed by a regime of coherent chattering, followed at higher conductances by a regime of spreading activity beyond the input-receiving population. The first two regimes are considered in the following. Animations of network activity over time (see the supplementary material M1, M2, M3 or online1) show that a single oscillation cycle is characterized by one or more waves of activation spreading out from one or more (roughly simultaneously appearing) origin points, passing through the input-receiving region until extinction occurs by either collision with another wave or reaching the stimulus boundary. Fewer, broader waves are observed in networks with strong lateral connectivity. In short, the input-receiving region appears to behave as an excitable medium during the presence of a stimulus.
In general, different stimulus patterns—different choices of input-receiving cells—lead to different degrees of average zero time lag synchrony () among these cells, depending on the connections that exist between them. This is best visualized on a lattice network. Figure 3a shows such a network set in the chattering regime (). For a stimulus that activates cells connected by short paths, these cells fire more synchronously than the more dispersed cells activated by a scattered version of the stimulus. Synchrony differences between different stimuli are discernible after a few spikes. Figure 3b shows synchrony measurements taken over increasingly long time windows (each starting from stimulus onset at ).
To quantify the information content of such measurements, we compute the length of time for which the network needs to be observed until a naive Bayes classifier correctly infers which stimulus was presented, based on the measured synchrony values. (See section 4 for details.) We report the number of spikes that need to occur, on average over the measured set of cells, until such a classification succeeds (see the histogram panels in Figures 3 and 5). For example, in case of the three stimuli in Figure 3, it is in almost all trials sufficient to observe the degree of synchrony of one or two spike waves. While simple grid networks as shown have the advantage of allowing an easy visualization of topological distances, we note that the same effect also occurs on other network topologies, such as networks where connection probability diminishes with spatial distance and “small world” networks with power-law degree distribution (see the supplemental material).
We showed excitatory spiking networks in which well-connected subpopulations fire synchronously when they receive (noisy) external drive. We have already argued that behavior like this is pivotal in explaining the interplay of cortical response selectivities, learned excitatory horizontal network structures, and experimental observations of synchrony. Further, a connection to stimulus familiarity is implied. Since cortical horizontal connections concentrate between cells that have often been coactivated by previous stimuli, a stimulus that activates well-connected cells is likely similar to (parts of) these past stimuli; they are thus familiar. To make this connection somewhat more explicit, in Figure 4 we turn to networks with a more heterogenous structure, shaped by stimulus patterns. We sample networks with random, local connectivity. Specifically, the probability that two different cells on a integer lattice are connected is set to fall with their Euclidean distance, . The cutoff varies between cell pairs. For each network, a set of random stimulus patterns plays the part of long-term stimulus history; these patterns are taken to have high prior probability (see section 4). In sampling the network, is then relaxed (from to ) between any cells that co-occur in such a pattern (allowing slightly longer links between such cells). These connections are also stronger ( versus 1 elsewhere). As a result, several of the cell pairs coactivated in such patterns end up being directly and strongly connected. In other words, a number of strong excitatory subnetworks have been embedded in the larger network structure (concentrated around certain stimulus configurations), leading to a network that, under suitable external input, can partially behave as an excitable medium. The network structure is then kept fixed, since we are concerned only with network dynamics during short stimulus presentations. A model of the origin of such clustered network structures in terms of an interaction of plasticity rules and axonal delays has been proposed by Izhikevich, Gally, and Edelman (2004).
The stimulus patterns are random and local, in the sense that the probability that a cell is counted as active in a certain pattern falls with the distance (like above) from a randomly chosen center point. After generating the network, additional patterns are sampled to serve as new, incoming stimuli. We define the familiarity of such a new pattern as the fraction of its cells shared with the initial set of patterns that has shaped the network (its similarity to these patterns). This can also be expressed as a prior probability (see section 4). We define 10 intervals (bins) for this familiarity value and, by rejection sampling, generate a number of stimulus patterns in each bin. For generality, the whole sampling procedure of networks and patterns is repeated until 100 examples (different networks with different input patterns) have been drawn in each bin. There are thus three main sources of variance in the synchrony measurements displayed in Figure 4. First, the network sampling procedure is stochastic, meaning that nominally familiar patterns do not necessarily materialize in the form of a well-connected network. To emphasize this factor, the last bar in Figure 4 shows the subset of trials in which at least two connections have been sampled per stimulated cell, that is, the subset of trials in which the network clearly reflects the pattern. Second, variance is introduced by the random initial patterns that define the network structure, which may, for example, overlap. Third, there is some degree of trial-by-trial variability of any single pattern on a single network, as seen in Figure 3.
Despite these factors, an increase in synchrony is apparent as we increase the similarity of incoming stimulus patterns to the patterns embodied in the network structure. This result can be explained in terms of the effect shown before. When an incoming pattern happens to hit an excitable (strongly connected) subnetwork, dynamics as in Figure 3 play out. There, we have shown that the better a group of activated cells is connected by such strong connections, the higher its synchrony is. Here, as in the cortex, these connections are clustered around certain familiar stimulus patterns; hence, stimuli that resemble these patterns produce stronger synchrony. Note that this is not simply a matter of activating cells in the vicinity of strong connections. In Figure 4, the pattern with similarity index 0.56 (bottom row) is located in a relatively strongly connected region but evidently misses part of the relevant subnetwork.
In the cited experimental reports of cortical synchrony, only a few (multi-)electrode recording sites are usually set and stay fixed throughout the various stimulus presentations. Mimicking this scenario, we find that two fixed, small groups of cells fire with increased synchrony if a stimulus activates cells on the direct path between those groups (see Figure 5). This network was generated by the same procedure used in Figure 4, with increased connectivity concentrated in a horizontal line (as would be expected, for example, in a group of similarly orientation-tuned V1 cells often exposed to continuous contours). Lateral conductances are (see panel a) and (see panel b).
Under uncorrelated inhibition, the stimulus-driven cells receive random inhibitory input spikes in addition to the excitatory drive. As the conductance of inhibitory input synapses increases beyond that of the lateral excitatory connections, synchrony begins to decline gradually. Under pure feedback inhibition (in which each excitatory cell projects to an additional inhibitory interneuron that projects directly back to it), even strong inhibitory conductances have little effect on synchrony. This changes with the addition of lateral inhibition (in which the inhibitory interneuron projects not only back to its driving excitatory cell, but also to the neighbors of that cell). Here again, we see a gradual decline in synchrony as inhibitory conductances increase beyond the strength of the lateral excitatory network. In sum, we find that the presence of moderately strong inhibition of various temporal structures is not sufficient to destabilize the discussed synchronous dynamics but offers a mechanism to dampen them gradually.
We showed examples of noise-driven, pulse-coupled spiking networks that display synchronous responses to random pulse inputs. By directing this input to cells that are closely and strongly connected (or located on a direct strong path between measured network locations), synchrony is increased. In accordance with experiments (Maldonado et al., 2008), synchrony diverges quickly between different stimuli (i.e., within few spikes).
Our motivation for our choice of network model was the properties of cortical long-range horizontal connections. These are distance-limited, excitatory, chemical (pulsed) couplings between spiking cells, with a topology shaped by long-term experience in the sense that commonly coactivated cells are connected more closely and much more strongly (Cossell et al., 2015). But independent, excitatory subnetworks of functionally related cells also exist at smaller scales (Yoshimura, Dantzker, & Callaway, 2005). In fact, since plasticity in cortical networks seems to encourage cluster formation (Izhikevich et al., 2004), such subnetworks may be ubiquitous. For instance, layer 5 cortical microcircuits appear to form a network of small, strongly excitatory subnetworks (Song et al., 2005). The presented dynamics may therefore be found in structures other than the horizontal connections discussed, the more so as these dynamics occur across a broad range of synapse strengths, noise rates, and diversity of network topologies.
The effect appears to be linked to the presence of traveling spike waves within the input-receiving population. This does not imply that the model predicts large wave fronts traveling across the cortex. Given that strong synaptic connections (on which such waves depend) tend to be highly clustered, spreading waves may play out entirely within relatively confined subnetworks, undetectable at the mesoscopic scale. In other words, the proposed excitable dynamics are compatible with situations where no large-scale traveling fronts are observed. Such fronts do, however, occur in certain situations (Sato, Nauhaus, & Carandini, 2012; Takahashi et al., 2015): In visual cortex, they are mostly found during presentations of small, isolated stimuli (Sato et al., 2012; Nauhaus et al., 2009), and their size and number appear to peak directly following a stimulus change, after which they ease off progressively (Shew et al., 2015).
Inhibitory interneurons play no part in creating the presented synchrony effects; conversely, the effect is robust to moderate levels of interfering inhibitory input, which gradually attenuates it. Reports of cortical stimulus-dependent spike synchrony thus do not by themselves imply the presence of excitatory-inhibitory oscillator pairs.
We have isolated excitable dynamics as a common mechanism to explain a number of reports of cortical spike synchrony, including some of the experiments that sparked the binding-by-synchrony debate. Functionally, this leads to a quick, locally computable, spike timing–encoded measure of stimulus familiarity (or, more generally, beyond sensory areas, of the familiarity of a given spatial activation pattern). As noted, horizontal connectivity patterns across the cortex reflect long-term experience in the sense that commonly coactivated units are particularly well connected. In an excitable dynamic regime as presented here, higher spike synchrony is thus expected for input patterns that resemble experience. Hence, for example, synchrony may increase during perceptions of coherent visual stimuli such as connected lines simply because these are often experienced structures in our visual environment, which the network structure reflects. In such early sensory contexts, familiarity may thus express the degree to which a stimulus is structure or noise. Most behaviorally relevant objects have a highly structured appearance (consisting, for example, of Gestalt-like unbroken lines), while other, noisier signals (with fewer spatial correlations) are typically found in less urgently important phenomena, such as background textures. In other words, often experienced, behaviorally relevant stimuli are characterized by high mutual predictability of their different constituent parts, and synchrony appears to signal this (Vinck & Bosman, 2016). The result is a receptive field–like effect on the network level, in which a given group of cells is tuned to certain, well-correlated spatial arrangements of incoming activity and responds with a particular activation signature to these patterns but not to others, constituting a feature extraction step that operates on the level of groups of activated cells.
In more abstract terms, we have proposed that various cortical networks have access to a synchrony-encoded estimate of the prior probability of observing the current input pattern. A first estimate is available directly after the onset of the pattern (since synchrony in the first few spike waves is often already informative), after which precision continually improves. Hence, such a signal could be used early after input onset in a feedforward fashion—for example, to guide attention toward stimuli composed of plausible parts. More generally, estimates of prior probabilities are a prerequisite in Bayesian accounts of perception and learning, but it is unclear how such probabilities are represented neurally (Fiser, Berkes, Orbán, & Lengyel, 2010). We suggest that a spike-based encoding with the presented mechanism allows to rapid generation and transmission of such signals.
4.1 Network Model
4.2 Measuring Synchrony
Intuitively, if all members of the measured population fire strictly simultaneously, the mean activity of these cells fluctuates just as strongly as each of their individual activities (producing a value of ). If cells fire out of phase, their mean activity is comparably stable, while each individual cell still fluctuates as much as before, leading to a lower, though nonzero, value.
4.3 Measuring Synchrony Differences over Time
To quantify for how long a network needs to be observed until a difference in synchrony between stimuli becomes apparent, we take synchrony measurements in increasingly long time intervals. The earlier a correct classification by stimulus is possible, the more quickly the degrees of synchrony of these stimuli must have diverged. In more technical terms, we perform repeated, independent gaussian naive Baye's classifications on synchrony values measured in growing window increments, as follows.
4.4 Probabilistic Formulation of Familiarity
4.5 Code Availability
Annotated source files to reproduce all presented results are found at https://github.com/cknd/synchrony.
J.G.O. was supported by the Ministerio de Economia y Competividad and FEDER (Spain, project FIS2015-66503-C3-1-P) and the ICREA Academia programme. E.U. acknowledges support from the Scottish Universities Life Sciences Alliance (SULSA) and HPC-Europa2.