It has been established that homeostatic synaptic scaling plasticity can maintain neural network activity in a stable regime. However, the underlying learning rule for this mechanism is still unclear. Whether it is dependent on the presynaptic site remains a topic of debate. Here we focus on two forms of learning rules: traditional synaptic scaling (SS) without presynaptic effect and presynaptic-dependent synaptic scaling (PSD). Analysis of the synaptic matrices reveals that transition matrices between consecutive synaptic matrices are distinct: they are diagonal and linear to neural activity under SS, but become nondiagonal and nonlinear under PSD. These differences produce different dynamics in recurrent neural networks. Numerical simulations show that network dynamics are stable under PSD but not SS, which suggests that PSD is a better form to describe homeostatic synaptic scaling plasticity. Matrix analysis used in the study may provide a novel way to examine the stability of learning dynamics.
It is well established that neurons undergo homeostatic forms of plasticity (Ramakers, Corner, & Habets, 1990; Karmarkar & Dan, 2006; Nelson & Turrigiano, 2008; Pozo & Goda, 2010), in which neurons up- or downregulate different neuronal or synaptic properties in response to changes in overall levels of neural activity. In recent years, one form of homeostatic plasticity, called homeostatic synaptic scaling, has been of particular interest to both experimental and theoretical neuroscientists, since it has been proposed to stabilize the positive feedback of the Hebbian rule (van Rossum, Bi, & Turrigiano, 2000; Houweling, Bazhenov, Timofeev, Steriade, & Sejnowski, 2005; Renart, Song, & Wang, 2003; Frohlich, Bazhenov, & Sejnowski, 2008; Turrigiano, 2007).
The original study revealed uniformly multiplicative synaptic scaling (Turrigiano, Leslie, Desai, Rutherford, & Nelson, 1998) in which all the synapses onto a postsynaptic neuron are scaled by the same factor according to the difference between desired and actual average activity levels in neurons. However, recent experimental results show that synaptic scaling is observed in some situations but not in others (Goel & Lee, 2007; Turrigiano, 2007; Kim & Tsien, 2008). These results suggest that synaptic scaling is not always uniform and that changes in network activity do not equally affect all presynaptic inputs onto a given neuron (Pozo & Goda, 2010). Therefore, the traditional synaptic scaling (SS) learning rule formulated with one uniform scale, which is dependent only on postsynaptic neural activity (van Rossum et al., 2000), may not be correct. Indeed, it has been shown that in recurrent networks, SS is unstable (Buonomano, 2005; Houweling et al., 2005; Frohlich et al., 2008; Liu & Buonomano, 2009; Liu & She, 2009). Another type of learning rule, termed presynaptic-dependent synaptic scaling (PSD), has been proposed (Buonomano, 2005; Liu & Buonomano, 2009). PSD is a variation of synaptic scaling that takes into account the average levels of activity of the presynaptic neurons. Computer simulations have established that PSD can stabilize multiple network-wide trajectories in recurrent networks (Liu & Buonomano, 2009).
In this study, we mathematically analyze the learning rules in both SS and PSD forms. We show that the mathematical structures of these learning rules are dramatically different. The transition matrix, which is defined as the synaptic scaling factor between synaptic matrices of two consecutive time steps, is diagonal and linearly proportional to neural activity under usual matrix multiplication under SS, while under PSD, it is nondiagonal and nonlinear under the Hadamard (entrywise) product. Through simulating recurrent networks with these two rules, we find that SS produces unstable dynamics with excitation explosion—runaway activity of high firing rates in all excitatory neurons of the network. However, PSD provides stable dynamics with self-organized neural trajectories that resemble behaviorally relevant spatiotemporal patterns of activity for sensory inputs (Broome, Jayaraman, & Laurent, 2006), motor behaviors (Hahnloser, Kozhevnikov, & Fee, 2002), as well as memory and planning (Pastalkova, Itskov, Amarasingham, & Buzsaki, 2008). Furthermore, the maximal eigenvalue of the synaptic matrix converges to a steady and larger-than-one value under PSD, but it does not converge under SS. Although the learning rules, whose synaptic matrices have all eigenvalues less than 1, have been shown to be sufficient for network stability (Rajan & Abbott, 2006; Siri, Quoy, Delord, Cessac, & Berry, 2007; Siri, Berry, Cessac, Delord, & Quoy, 2008; Goldman, 2009), our results indicate that this condition is not necessary; the maximal eigenvalue of synaptic matrix under stable learning dynamics can be larger than 1, which is consistent with a recent study (Sussillo & Abbott, 2009). Together with these results, we provide a new way to investigate the stability of learning rules and extend the previous view on the relationship between stable learning dynamics and synaptic matrix eigenvalues. Our results suggest that the learning rule of homeostatic synaptic scaling depends not only on postsynaptic but also on presynaptic neural activity.
2. Formulation and Simulation of Synaptic Scaling
2.1. Traditional Synaptic Scaling.
2.2. Presynaptic-Dependent Synaptic Scaling.
Although the learning dynamics, equations 2.1 and 2.2, and the neural and synaptic dynamics form a closed system, this system is difficult to analyze mathematically. Instead, we focus on the learning dynamics, equations 2.3 and 2.5, and show that network dynamics depend only on the learning rule chosen, not on the specific neural dynamics and synaptic kinetics.
2.3. Simulated Network Dynamics Is Stable with PSD, Not SS.
We conducted a series of numerical simulations to study learning dynamics with a recurrent neural network consisting of 500 (400 excitatory and 100 inhibitory) integrate-and-fire neurons and excitatory AMPA and NMDA and inhibitory GABAA kinetic synapses. (Neural network model and simulation details can be found in the appendix.)
Neural activity patterns under SS are presented in Figure 1A. At τ=1, only the stimulated neurons are active. As learning proceeds, more neurons fire, and at τ=169, approximately all neurons fire. However, excitation explodes only one trial later, at τ=170. Such a sharp transition indicates excitation explosion, or what has been termed a synfire explosion (Mehring, Hehl, Kubo, Diesmann, & Aertsen, 2003; Vogels, Rajan, & Abbott, 2005; Destexhe & Contreras, 2006). At τ=173, activity decreases quickly. In contrast, activity patterns under PSD are significantly different (see Figure 1 B), where neurons fire in a stable manner as learning proceeds. Network activity at τ=200, 300, and 500 represents typical patterns of different learning phases. In contrast with SS, the final neural activity is stable, and no excitation explosion is observed at any point during the simulation.
Pathological excitation explosion can be formalized as a sharp discontinuity, in a mathematical sense, since it exhibits the large jump of neural activity between two consecutive trials or within a short time period (Mehring et al., 2003), which indicates the instability of the learning rule. Stability can therefore be defined as that is, the discrete derivative of the average firing rate over the whole network is less than a small number (ε≪1). Using equation 2.4, this condition becomes Given that is small and N is large, when network dynamics are stable, one expects to be small with ε≪1.
Stability is visualized in the plot of the mean firing rate averaged over all neurons as a function of learning trial τ. As shown in Figure 1C, under SS is oscillating with jumps, whereas under PSD, develops stably and converges gradually to the target firing rate, νgoal=1. Learning dynamics can also be described by the derivative of the curve, . Figure 1D shows a clear jump discontinuity under SS, with a larger bound ε<0.5, but this discontinuity is prevented by PSD, with a small bound ε<0.01.
Note that in Figure 1C, the activity goes up quickly and then goes down slowly. Such a sharp upstroke is a signature of SS. As shown in equation 2.4, the change of synaptic weights under SS is dependent on only postsynaptic neural activity. Therefore, under SS, all synaptic weights onto a postsynaptic neuron are scaled up by the same amount, which corresponds to the amount of excitatory postsynaptic potential (EPSP, denoted as Vss) induced by this increased weight. A large number of synaptic weights (denoted as Nss) are on the same postsynaptic neuron, and they all induce an amount of EPSP, Vss. There is no competition among these synapses, and the ratios between these synaptic weights remain unchanged. Thus, the overall strength of this postsynaptic neuron will be Nss×Vss, which is large enough to make this neuron fire much faster. Overall, at the network level, the activity increases quickly, which results in a sharp upstroke or excitation explosion. Changing parameters alters only the magnitude and period of oscillations, but the excitation explosion remains unchanged. The instability of SS is unchanged when noise is considered, as shown in Liu and Buonomano (2009).
2.4. Homeostatic Control Realized by PSD, Not SS.
The reason for this instability of SS is that the presynaptic index j is free from the change of synaptic weights—the contribution of Δw is dependent on only postsynaptic neural activity. Therefore, if the synaptic matrix is summed over all presynaptic elements for each postsynaptic index, a vector can be defined as sw(τ)i≔∑jw(τ)ij, which is denoted as the prestrength vector. One expects that sw(τ)i changes uniformly over learning trials by the same scale without changing the inner ratio of presynaptic strengths converging to a postsynaptic neuron i. Essentially the competition of synapses is absent under SS. Similarly, one can define a vector sw(τ)j≔∑iw(τ)ij as the poststrength vector.
Figure 2A shows that the elements of the prestrength vector swi under SS are distributed uniformly within one trial except for the first 24 neurons, which are stimulated as an input. The same panel also shows that swi are scaled globally across different trials τ=1, 100, and 300. This global scaling is a typical signature of SS due to no synaptic competition among all synapses projecting onto a postsynaptic neuron. However, in Figure 2B swi under PSD are distributed and scaled heterogeneously, particularly at trial τ=300. The absence of single global scaling across trials under PSD stems from the strong synaptic competition among all synapses of the network. Similarly, in Figure 2C, pos-strengths swj under SS are scaled globally across trials, even though they are distributed less uniformly within one trial. In Figure 2D, swj under PSD are heterogeneous both within one trial and across trials. Note that stimulated neurons have larger values of swj because they fire all the time during learning, and synapses from them are preferentially potentiated under PSD. Further examination of the standard deviation σ of distributions, in Figure 2E, shows that they are significantly different and separated under PSD and nearly overlapping under SS, which indicates that synaptic competition is missing under SS and present under PSD.
3. Matrix Analysis of Learning Dynamics
3.1. Matrix Form of SS.
3.2. Matrix Form of PSD.
3.3. Synaptic Matrix Converged Under PSD, Not SS.
Now we analyze synaptic matrices of SS and PSD. In Figure 3A (the gray lines), the ratios r1 and r2 shaped by SS are close to 1, the theoretical upper-bound. We find that the largest eigenvalue of the synaptic matrix, ρ(W(τ)) in Figure 3C (the gray lines), the all-step transition matrix, in Figure 3B (the gray lines), and the one-step transition matrix, ρ(D(τ)), Figure 3D (the gray lines) are always larger than 1. In particular, ρ(W(τ)) and are increasing without an upper bound under SS. In contrast, the matrices shaped by PSD (see Figure 3, the black lines) are different from those shaped by SS (see Figure 3 the gray lines). The ratios are significantly less than 1: r1≪1 and r2≪1. The largest eigenvalue of the synaptic matrix under PSD (see Figure 3C the black lines) is also larger than 1, which violates the sufficient condition where it should be less than 1. Because the singular values of all-step and one-step transition matrices under PSD are not their largest eigenvalues, they are much larger than those of SS as in Figures 3B and 3D (the black lines). Under both SS and PSD, results with the larger-than-1 largest eigenvalues for synaptic matrix W are in contrast to the traditional viewpoint.
In this work, we have studied analytically and numerically two types of homeostatic synaptic scaling learning rules in recurrent neural networks. In particular, the underlying mathematical structures of learning rules are identified. The difference is captured by the transition matrix between synaptic matrices, which is diagonal and linear under SS but nondiagonal and nonlinear under the Hadamard product under PSD. Through numerical simulations, we have confirmed that SS generates an unstable excitation explosion, and PSD gives stable network dynamics. Furthermore, the stable PSD learning produces a synaptic matrix in which the largest eigenvalue is larger than 1. These results, together with recent experiments (Goel & Lee, 2007; Kim & Tsien, 2008), suggest that homeostatic synaptic scaling is dependent on both pre- and postsynaptic neural activity.
Note that the above analysis of learning rules is independent of the underlying neural dynamics. To further confirm that our results are unrelated to specific dynamics of neurons and synapses, we simulated a neural network with binary excitatory neurons without synaptic decaying dynamics and obtained qualitatively similar results (see the supplemental material available online at http://www.mitpressjournals.org/doi/suppl/10.1162/NECO_a_00210).
4.1. Stability of Learning Dynamics.
Biologically, the question of how recurrent networks develop functional dynamics and avoid excitation explosion is critical to understanding cortical function. The stable learning rule should generate the convergent dynamics within a neural network without pathological activity (Vogels et al., 2005; Destexhe & Contreras, 2006; Frohlich et al., 2008). The most straightforward way to describe excitation explosion over the course of learning is to use the derivative of the average firing rate curve, . The different values of ϵ characterize the degree of excitation explosion. Therefore, stability is obtained when ϵ≪1, which occurs under PSD learning. The instability with a sharp discontinuity and excitation explosion is observed under SS learning.
Stability analysis has been intensively studied in the literature of artificial neural networks and machine learning (Hertz, Krogh, & Palmer, 1991) with a goal of controlling the stability of the synaptic matrix. Here we have focused on what would correspond to the development of a cortical network. In our simulation, the network develops from an initial state, in which all synapses are weak and activity does not propagate, to the one in which stimuli elicit network-wide activity. This scenario is observed in cortical networks in vitro (Johnson & Buonomano, 2007), where the underlying synaptic matrix may be shaped by learning dynamics to avoid excitation explosion.
When a stimulus is presented, it is the learning dynamics that make the activity develop in a stable manner and the synaptic matrix converge to a stable state. The stability condition of learning rules may be more complicated than what is expected by controlling the largest eigenvalue. We find that the ratio r1 is close to the theoretical upper bound 1 under SS but is much less than 1 under PSD. The classical analysis of the Hebbian rule requires the synaptic matrix to be controlled with ρ(W)<1, which is a sufficient condition. However, this condition fails in our results. The largest eigenvalues of all matrices under SS and PSD are larger than 1. We speculate that r1≪1 is a necessary and sufficient condition for the stability of homeostatic synaptic scaling, and we suggest that r1 is an important diagnostic variable for the stability. r1 may play the role of order parameter as in a phase transition in statistical physics. It is likely that calculating r1 for a number of learning rules and plotting them all together can generate a phase diagram of stability of learning rules, in which stable rules have r1≪1 and unstable rules have larger r1. Then in this phase diagram with r1 as an order parameter, PSD resides nearly at the boundary point 0, and SS is close to the boundary point 1. In this way, each learning rule has a unique r1 associated with its stability property.
Interestingly, consistent with our results, a recent study shows that chaotic neural networks can generate coherent patterns of activity, even though real parts of many eigenvalues are greater than 1, both before and after training (Sussillo & Abbott, 2009). During training with an unchaotic neural network, there exist eigenvalues with real parts greater than 1 after training. These results suggest that generating stable network dynamics with different learning rules yields different solutions, in which the synaptic matrix, with or without larger-than-1 eigenvalues, can be shaped by learning dynamics in a stable manner.
4.2. Generalizing to Other Learning Rules.
In the classical models of Hebbian learning, Hebb's postulate is rephrased as modifications of synaptic weights driven by correlations in the firing activity of pre- and postsynaptic neurons, which is often taken as an additive form without the scaling factor, Δwij∼F(νi×νj). Most classic theoretical studies represent the activity of pre- and postsynaptic neurons in terms of firing rates with the different functions of form F(·) (Gerstner & Kistler, 2002). However, the unique feature of homeostatic synaptic scaling is that the change of weights has a multiplication form with Δwij∼F(νi×νj)wij.
In general, synaptic learning rules can be classified into two categories: multiplicative and additive rules dependent on whether there is a scale factor in the function F or firing rate and spike timing rules dependent on which type of F is used. Combinations of these two categories give four specific types of learning rules. Synaptic scaling is a form of a multiplicative firing-rate rule, in which F of the SS rule is dependent only on postsynaptic neural firing rate, whereas F of the PSD rule is dependent on both pre- and postsynaptic neural firing rates. The matrix analysis conducted in this study can be applied to other learning rules even when spike timing is considered.
As long as presynaptic activity (firing rate or spike timing) is considered in any particular learning rule, the matrix structure of this rule also uses the Hadamard product. In recent years, spike-timing-dependent plasticity (STDP) has been identified experimentally and studied intensively (Bi & Poo, 2001; Morrison, Diesmann, & Gerstner, 2008). Since STDP needs information from both pre- and postsynaptic spike times, the matrix analysis we have explored can be applied. Other presynaptic dependent rules have been proposed in the literature, such as heterosynaptic depression among all input synapses, which has been shown to generate stable activity sequences within recurrent networks (Fiete, Senn, Wang, & Hahnloser, 2010). However, there is no systematic way to study the stability of these rules. Furthermore, it will be interesting to study the case where two or more learning rules are used together (Liu & Buonomano, 2009; Fiete et al., 2010; Clopath, Bsing, Vasilaki, & Gerstner, 2010); in this case the order parameter r1 may have different values, and the stability of combined learning rules may have a different stable phase. Our analysis may give a clue on this issue, although further studies are needed.
Appendix: Simulation of Spiking Neural Network
We used the same neural network model as described by Liu and Buonomano (2009), where simulations were performed using NEURON (Hines & Carnevale, 1997) with a time step t=0.1 ms. The codes programmed with C++ generated similar results (Liu & She, 2009). The code can be downloaded from the author's home page.
A.1. Neural Dynamics.
A.2. Synaptic Dynamics.
A.3. Network Topology and Parameters.
All simulations are performed using a network with 400 excitatory (E) and 100 inhibitory (I) neurons connected with a probability 0.12 for E→E, and 0.2 for both E→I and I→E, which results in each postsynaptic E-neuron receiving 48 inputs from other E-neurons and 20 inputs from I-neurons; each postsynaptic I-neuron receives 80 inputs from E-neurons. Initial synaptic weights are chosen from a normal distribution with mean WEE=2/48 nS, WEI=1/80 nS, and WIE=2/20 nS, respectively, and SD σEE=2WEE, σEI=8WEI, and σIE=8WIE. If the initial weights are nonpositive, they are reset to a uniform distribution from 0 to twice the mean. To avoid the induction of unphysiological states in which a single presynaptic neuron fires a postsynaptic neuron, the maximal E→E AMPA synaptic weights are WEEmax=1.5 nS. The maximal E→I AMPA synaptic weights are set to WEImax=0.4 nS. All inhibitory synaptic weights are fixed. For learning rules, αw=0.01 and . νgoal is the target activity set to 1(2)Hz for E(I)-cells. A stimulus is composed by randomly selected 24 E- and 12 I-cells that fire at 1 Hz. The input spiking timings are assigned to 10±1 ms (mean ± SD) following a normal distribution relative to the onset of each period of 1 s, thus, one subset of cells fires at the beginning of each period. Selected input cells are activated by a 1 Hz excitatory postsynaptic current.
We thank Dean Buonomano, Tiago Carvacrol, and Tyler Lee for helpful discussions, and Nicolas Brunel, Claudia Clopath, Omri Harish, and David Higgins for comments and careful reading of the manuscript. This work was partially supported by the ANR-BBSRC Grant VESTICODE.