## Abstract

It has been established that homeostatic synaptic scaling plasticity can maintain neural network activity in a stable regime. However, the underlying learning rule for this mechanism is still unclear. Whether it is dependent on the presynaptic site remains a topic of debate. Here we focus on two forms of learning rules: traditional synaptic scaling (SS) without presynaptic effect and presynaptic-dependent synaptic scaling (PSD). Analysis of the synaptic matrices reveals that transition matrices between consecutive synaptic matrices are distinct: they are diagonal and linear to neural activity under SS, but become nondiagonal and nonlinear under PSD. These differences produce different dynamics in recurrent neural networks. Numerical simulations show that network dynamics are stable under PSD but not SS, which suggests that PSD is a better form to describe homeostatic synaptic scaling plasticity. Matrix analysis used in the study may provide a novel way to examine the stability of learning dynamics.

## 1. Introduction

It is well established that neurons undergo homeostatic forms of plasticity (Ramakers, Corner, & Habets, 1990; Karmarkar & Dan, 2006; Nelson & Turrigiano, 2008; Pozo & Goda, 2010), in which neurons up- or downregulate different neuronal or synaptic properties in response to changes in overall levels of neural activity. In recent years, one form of homeostatic plasticity, called homeostatic synaptic scaling, has been of particular interest to both experimental and theoretical neuroscientists, since it has been proposed to stabilize the positive feedback of the Hebbian rule (van Rossum, Bi, & Turrigiano, 2000; Houweling, Bazhenov, Timofeev, Steriade, & Sejnowski, 2005; Renart, Song, & Wang, 2003; Frohlich, Bazhenov, & Sejnowski, 2008; Turrigiano, 2007).

The original study revealed uniformly multiplicative synaptic scaling (Turrigiano, Leslie, Desai, Rutherford, & Nelson, 1998) in which all the synapses onto a postsynaptic neuron are scaled by the same factor according to the difference between desired and actual average activity levels in neurons. However, recent experimental results show that synaptic scaling is observed in some situations but not in others (Goel & Lee, 2007; Turrigiano, 2007; Kim & Tsien, 2008). These results suggest that synaptic scaling is not always uniform and that changes in network activity do not equally affect all presynaptic inputs onto a given neuron (Pozo & Goda, 2010). Therefore, the traditional synaptic scaling (SS) learning rule formulated with one uniform scale, which is dependent only on postsynaptic neural activity (van Rossum et al., 2000), may not be correct. Indeed, it has been shown that in recurrent networks, SS is unstable (Buonomano, 2005; Houweling et al., 2005; Frohlich et al., 2008; Liu & Buonomano, 2009; Liu & She, 2009). Another type of learning rule, termed presynaptic-dependent synaptic scaling (PSD), has been proposed (Buonomano, 2005; Liu & Buonomano, 2009). PSD is a variation of synaptic scaling that takes into account the average levels of activity of the presynaptic neurons. Computer simulations have established that PSD can stabilize multiple network-wide trajectories in recurrent networks (Liu & Buonomano, 2009).

In this study, we mathematically analyze the learning rules in both SS and PSD forms. We show that the mathematical structures of these learning rules are dramatically different. The transition matrix, which is defined as the synaptic scaling factor between synaptic matrices of two consecutive time steps, is diagonal and linearly proportional to neural activity under usual matrix multiplication under SS, while under PSD, it is nondiagonal and nonlinear under the Hadamard (entrywise) product. Through simulating recurrent networks with these two rules, we find that SS produces unstable dynamics with excitation explosion—runaway activity of high firing rates in all excitatory neurons of the network. However, PSD provides stable dynamics with self-organized neural trajectories that resemble behaviorally relevant spatiotemporal patterns of activity for sensory inputs (Broome, Jayaraman, & Laurent, 2006), motor behaviors (Hahnloser, Kozhevnikov, & Fee, 2002), as well as memory and planning (Pastalkova, Itskov, Amarasingham, & Buzsaki, 2008). Furthermore, the maximal eigenvalue of the synaptic matrix converges to a steady and larger-than-one value under PSD, but it does not converge under SS. Although the learning rules, whose synaptic matrices have all eigenvalues less than 1, have been shown to be sufficient for network stability (Rajan & Abbott, 2006; Siri, Quoy, Delord, Cessac, & Berry, 2007; Siri, Berry, Cessac, Delord, & Quoy, 2008; Goldman, 2009), our results indicate that this condition is not necessary; the maximal eigenvalue of synaptic matrix under stable learning dynamics can be larger than 1, which is consistent with a recent study (Sussillo & Abbott, 2009). Together with these results, we provide a new way to investigate the stability of learning rules and extend the previous view on the relationship between stable learning dynamics and synaptic matrix eigenvalues. Our results suggest that the learning rule of homeostatic synaptic scaling depends not only on postsynaptic but also on presynaptic neural activity.

## 2. Formulation and Simulation of Synaptic Scaling

### 2.1. Traditional Synaptic Scaling.

*w*is the synaptic strength from presynaptic neuron

_{ij}*j*to postsynaptic neuron

*i*, and ν

_{goal}is the target firing rate. is the average activity (firing rate) of neuron

*i*given by where δ is the Dirac function,

*t*

_{max}is the time window for the spike count, and

*t*,

^{k}*k*=1, 2, …, are the spike times relative to the onset of stimulus. The characteristic timescale of synaptic scaling is long, and τ

_{w}and are large (Turrigiano, 2007). As in the previous study (Frohlich et al., 2008), we separate the neural network dynamics into two timescales and approximate slow synaptic regulation with a discrete time update scheme.

*w*

^{(τ)}

_{ij}. Then equations 2.1 and 2.2 can be written as where and α

_{w}=1/τ

_{w}. Learning dynamics and neural activity are coupled via , that is, the firing rate of neuron

*i*at the τth trial. The mismatch between the instantaneous and the average firing rates adjusts the interaction between neural activity and learning dynamics until the network reaches a stable state with .

### 2.2. Presynaptic-Dependent Synaptic Scaling.

*w*: Under this PSD rule, a postsynaptic neuron will preferentially potentiate the synapses from more active neurons.

_{ij}Although the learning dynamics, equations 2.1 and 2.2, and the neural and synaptic dynamics form a closed system, this system is difficult to analyze mathematically. Instead, we focus on the learning dynamics, equations 2.3 and 2.5, and show that network dynamics depend only on the learning rule chosen, not on the specific neural dynamics and synaptic kinetics.

### 2.3. Simulated Network Dynamics Is Stable with PSD, Not SS.

We conducted a series of numerical simulations to study learning dynamics with a
recurrent neural network consisting of 500 (400 excitatory and 100 inhibitory)
integrate-and-fire neurons and excitatory AMPA and NMDA and inhibitory
GABA_{A} kinetic synapses. (Neural network model and simulation
details can be found in the appendix.)

Neural activity patterns under SS are presented in Figure 1A. At τ=1, only the stimulated neurons are active. As learning proceeds, more neurons fire, and at τ=169, approximately all neurons fire. However, excitation explodes only one trial later, at τ=170. Such a sharp transition indicates excitation explosion, or what has been termed a synfire explosion (Mehring, Hehl, Kubo, Diesmann, & Aertsen, 2003; Vogels, Rajan, & Abbott, 2005; Destexhe & Contreras, 2006). At τ=173, activity decreases quickly. In contrast, activity patterns under PSD are significantly different (see Figure 1 B), where neurons fire in a stable manner as learning proceeds. Network activity at τ=200, 300, and 500 represents typical patterns of different learning phases. In contrast with SS, the final neural activity is stable, and no excitation explosion is observed at any point during the simulation.

Pathological excitation explosion can be formalized as a sharp discontinuity, in
a mathematical sense, since it exhibits the large jump of neural activity
between two consecutive trials or within a short time period (Mehring et al., 2003), which indicates the
instability of the learning rule. Stability can therefore be defined as that is, the discrete derivative of the average firing rate
over the whole network is less than a small number (ε≪1). Using
equation 2.4, this condition
becomes Given that is small and *N* is large, when network
dynamics are stable, one expects to be small with ε≪1.

Stability is visualized in the plot of the mean firing rate averaged over all neurons as a function of learning trial
τ. As shown in Figure 1C, under SS is oscillating with jumps, whereas under PSD, develops stably and converges gradually to the target firing
rate, ν_{goal}=1. Learning dynamics can also be described by the derivative of
the curve, . Figure 1D shows a clear
jump discontinuity under SS, with a larger bound ε<0.5, but this
discontinuity is prevented by PSD, with a small bound ε<0.01.

Note that in Figure 1C, the activity goes up
quickly and then goes down slowly. Such a sharp upstroke is a signature of SS.
As shown in equation 2.4, the
change of synaptic weights under SS is dependent on only postsynaptic neural
activity. Therefore, under SS, all synaptic weights onto a postsynaptic neuron
are scaled up by the same amount, which corresponds to the amount of excitatory
postsynaptic potential (EPSP, denoted as *V _{ss}*) induced by this increased weight. A large number of synaptic weights
(denoted as

*N*) are on the same postsynaptic neuron, and they all induce an amount of EPSP,

_{ss}*V*. There is no competition among these synapses, and the ratios between these synaptic weights remain unchanged. Thus, the overall strength of this postsynaptic neuron will be

_{ss}*N*×

_{ss}*V*, which is large enough to make this neuron fire much faster. Overall, at the network level, the activity increases quickly, which results in a sharp upstroke or excitation explosion. Changing parameters alters only the magnitude and period of oscillations, but the excitation explosion remains unchanged. The instability of SS is unchanged when noise is considered, as shown in Liu and Buonomano (2009).

_{ss}### 2.4. Homeostatic Control Realized by PSD, Not SS.

The reason for this instability of SS is that the presynaptic index *j* is free from the change of synaptic weights—the
contribution of Δ*w* is dependent on only postsynaptic
neural activity. Therefore, if the synaptic matrix is summed over all
presynaptic elements for each postsynaptic index, a vector can be defined as *sw*^{(τ)}_{i}≔∑_{j}*w*^{(τ)}_{ij}, which is denoted as the prestrength vector. One expects that *sw*^{(τ)}_{i} changes uniformly over learning trials by the same scale without changing
the inner ratio of presynaptic strengths converging to a postsynaptic neuron *i*. Essentially the competition of synapses is absent under
SS. Similarly, one can define a vector *sw*^{(τ)}_{j}≔∑_{i}*w*^{(τ)}_{ij} as the poststrength vector.

Figure 2A shows that the elements of the
prestrength vector *sw _{i}* under SS are distributed uniformly within one trial except for the
first 24 neurons, which are stimulated as an input. The same panel also shows
that

*sw*are scaled globally across different trials τ=1, 100, and 300. This global scaling is a typical signature of SS due to no synaptic competition among all synapses projecting onto a postsynaptic neuron. However, in Figure 2B

_{i}*sw*under PSD are distributed and scaled heterogeneously, particularly at trial τ=300. The absence of single global scaling across trials under PSD stems from the strong synaptic competition among all synapses of the network. Similarly, in Figure 2C, pos-strengths

_{i}*sw*under SS are scaled globally across trials, even though they are distributed less uniformly within one trial. In Figure 2D,

_{j}*sw*under PSD are heterogeneous both within one trial and across trials. Note that stimulated neurons have larger values of

_{j}*sw*because they fire all the time during learning, and synapses from them are preferentially potentiated under PSD. Further examination of the standard deviation σ of distributions, in Figure 2E, shows that they are significantly different and separated under PSD and nearly overlapping under SS, which indicates that synaptic competition is missing under SS and present under PSD.

_{j}## 3. Matrix Analysis of Learning Dynamics

### 3.1. Matrix Form of SS.

**W**

^{(τ+1)}is the synaptic matrix. The matrix

**D**

^{(τ)}with diagonal elements , which describes the change of synaptic weights due to learning, can be defined as a one-step transition matrix between two consecutive time steps. Note that this matrix yields a simple diagonal structure with each element linearly proportional to the average activity of the postsynaptic neuron. It is easy to show by recurrence that equation 3.1 becomes where the is diagonal with elements , which can be defined as an all-step transition matrix, since it includes the whole history of learning process. Thus, SS is essentially governed by equations 3.1 and 3.2, which satisfy where the spectral norm ‖·‖

_{2}is used since all matrices are square. By definition, the spectral norm of

**D**

^{(τ)}is the largest singular value of the matrix: where σ

_{1}(·) is the largest singular value, and ρ(·) is the largest eigenvalue—the spectral radius. These two are equal since

**D**

^{(τ)}is diagonal. Equation 3.5 can be tested by calculating the ratios which can be used to examine numerical results and find the specific values of

*r*

_{1},

*r*

_{2}.

### 3.2. Matrix Form of PSD.

*w*is dependent on both pre- and postsynaptic neural activity. As a result, PSD is more complicated than SS due to this nonlinear interaction , which induces a more complex mathematical structure. To see this, one can write the matrix equation as where ○ denotes the Hadamard product, a special product of matrices with entrywise instead of row- and column-wise multiplication, and

_{ij}**T**

^{(τ)}=

**J**+α

_{w}

**Γ**, in which

^{(τ)}**J**is the identity matrix under the Hadamard product with all entries equal to 1, and

**Γ**denotes the activity matrix with each element as . Rewriting this equation by recurrence, we get where has elements as . Note that

^{(τ)}**T**is nonlinear with respect to neural activity, and the Hadamard product is an unusual matrix product. Given these difficulties and the fact that

^{(τ)}**T**is not symmetric since synapses are directional with respect to neurons, there is one useful property of the Hadamard product (Horn & Johnson, 1994), which yields To compare with SS, we write the ratios

^{(τ)}*r*

_{1}and

*r*

_{2}as Note that under PSD, the largest singular value σ

_{1}(·) is not the largest eigenvalue ρ(·). The behavior of

*r*

_{2}is related to

*r*

_{1}, but it is not a simple linear relationship, since

*r*

_{2}is from the multiplication of all transition matrices with the whole history of learning trials. As a result,

*r*

_{2}reflects the mutual coupling of neural and learning dynamics. Since matrix structures are different between two learning rules, one expects the behaviors of

*r*

_{1}and

*r*

_{2}to be different as well.

### 3.3. Synaptic Matrix Converged Under PSD, Not SS.

**D**

^{(τ)}of SS be less than 1. Given that under SS , and supposing that the maximal eigenvalue is reached at the index

*m*, we have, after simple calculations, It is a sufficient condition to achieve the stability, for the synaptic weight equation, as discussed previously (Rajan & Abbott, 2006; Siri et al., 2007, 2008; Goldman, 2009). However, is the average firing rate starting from 0 and converging to the equilibrium firing rate ν

_{goal}. Ideally, we have without oscillations, or with decaying oscillations. Therefore, there is no index

*m*satisfying the sufficient condition, and this condition cannot hold in our case. This prevents any analysis of the synaptic matrix such that ρ(

**W**

^{(τ)})<1 during and after learning. It may turn out that ρ(

**W**

^{(τ)})<1 may be less realistic or not necessary. Instead, one expects the ratios

*r*

_{1}and

*r*

_{2}to include useful information for stability, although their actual values may vary from case to case.

Now we analyze synaptic matrices of SS and PSD. In Figure 3A (the gray lines), the ratios *r*_{1} and *r*_{2} shaped by SS are close to 1, the theoretical upper-bound. We find
that the largest eigenvalue of the synaptic matrix, ρ(**W**^{(τ)}) in Figure 3C (the
gray lines), the all-step transition matrix, in Figure 3B (the gray
lines), and the one-step transition matrix, ρ(**D**^{(τ)}), Figure 3D (the gray
lines) are always larger than 1. In particular, ρ(**W**^{(τ)}) and are increasing without an upper bound under SS. In contrast,
the matrices shaped by PSD (see Figure 3,
the black lines) are different from those shaped by SS (see Figure 3 the gray lines). The ratios are
significantly less than 1: *r*_{1}≪1 and *r*_{2}≪1. The largest eigenvalue of the synaptic matrix under PSD
(see Figure 3C the black lines) is also
larger than 1, which violates the sufficient condition where it should be less
than 1. Because the singular values of all-step and one-step transition matrices
under PSD are not their largest eigenvalues, they are much larger than those of
SS as in Figures 3B and 3D (the black lines). Under both SS and PSD, results
with the larger-than-1 largest eigenvalues for synaptic matrix **W** are in contrast to the traditional viewpoint.

## 4. Discussion

In this work, we have studied analytically and numerically two types of homeostatic synaptic scaling learning rules in recurrent neural networks. In particular, the underlying mathematical structures of learning rules are identified. The difference is captured by the transition matrix between synaptic matrices, which is diagonal and linear under SS but nondiagonal and nonlinear under the Hadamard product under PSD. Through numerical simulations, we have confirmed that SS generates an unstable excitation explosion, and PSD gives stable network dynamics. Furthermore, the stable PSD learning produces a synaptic matrix in which the largest eigenvalue is larger than 1. These results, together with recent experiments (Goel & Lee, 2007; Kim & Tsien, 2008), suggest that homeostatic synaptic scaling is dependent on both pre- and postsynaptic neural activity.

Note that the above analysis of learning rules is independent of the underlying neural dynamics. To further confirm that our results are unrelated to specific dynamics of neurons and synapses, we simulated a neural network with binary excitatory neurons without synaptic decaying dynamics and obtained qualitatively similar results (see the supplemental material available online at http://www.mitpressjournals.org/doi/suppl/10.1162/NECO_a_00210).

### 4.1. Stability of Learning Dynamics.

Biologically, the question of how recurrent networks develop functional dynamics and avoid excitation explosion is critical to understanding cortical function. The stable learning rule should generate the convergent dynamics within a neural network without pathological activity (Vogels et al., 2005; Destexhe & Contreras, 2006; Frohlich et al., 2008). The most straightforward way to describe excitation explosion over the course of learning is to use the derivative of the average firing rate curve, . The different values of ϵ characterize the degree of excitation explosion. Therefore, stability is obtained when ϵ≪1, which occurs under PSD learning. The instability with a sharp discontinuity and excitation explosion is observed under SS learning.

Stability analysis has been intensively studied in the literature of artificial neural networks and machine learning (Hertz, Krogh, & Palmer, 1991) with a goal of controlling the stability of the synaptic matrix. Here we have focused on what would correspond to the development of a cortical network. In our simulation, the network develops from an initial state, in which all synapses are weak and activity does not propagate, to the one in which stimuli elicit network-wide activity. This scenario is observed in cortical networks in vitro (Johnson & Buonomano, 2007), where the underlying synaptic matrix may be shaped by learning dynamics to avoid excitation explosion.

When a stimulus is presented, it is the learning dynamics that make the activity
develop in a stable manner and the synaptic matrix converge to a stable state.
The stability condition of learning rules may be more complicated than what is
expected by controlling the largest eigenvalue. We find that the ratio *r*_{1} is close to the theoretical upper bound 1 under SS but is much less
than 1 under PSD. The classical analysis of the Hebbian rule requires the
synaptic matrix to be controlled with ρ(**W**)<1, which is a
sufficient condition. However, this condition fails in our results. The largest
eigenvalues of all matrices under SS and PSD are larger than 1. We speculate
that *r*_{1}≪1 is a necessary and sufficient condition for the stability
of homeostatic synaptic scaling, and we suggest that *r*_{1} is an important diagnostic variable for the stability. *r*_{1} may play the role of order parameter as in a phase transition in
statistical physics. It is likely that calculating *r*_{1} for a number of learning rules and plotting them all together can
generate a phase diagram of stability of learning rules, in which stable rules
have *r*_{1}≪1 and unstable rules have larger *r*_{1}. Then in this phase diagram with *r*_{1} as an order parameter, PSD resides nearly at the boundary point 0,
and SS is close to the boundary point 1. In this way, each learning rule has a
unique *r*_{1} associated with its stability property.

Interestingly, consistent with our results, a recent study shows that chaotic neural networks can generate coherent patterns of activity, even though real parts of many eigenvalues are greater than 1, both before and after training (Sussillo & Abbott, 2009). During training with an unchaotic neural network, there exist eigenvalues with real parts greater than 1 after training. These results suggest that generating stable network dynamics with different learning rules yields different solutions, in which the synaptic matrix, with or without larger-than-1 eigenvalues, can be shaped by learning dynamics in a stable manner.

### 4.2. Generalizing to Other Learning Rules.

In the classical models of Hebbian learning, Hebb's postulate is rephrased as
modifications of synaptic weights driven by correlations in the firing activity
of pre- and postsynaptic neurons, which is often taken as an additive form
without the scaling factor, Δ*w _{ij}*∼

*F*(ν

_{i}×ν

_{j}). Most classic theoretical studies represent the activity of pre- and postsynaptic neurons in terms of firing rates with the different functions of form

*F*(·) (Gerstner & Kistler, 2002). However, the unique feature of homeostatic synaptic scaling is that the change of weights has a multiplication form with Δ

*w*∼

_{ij}*F*(ν

_{i}×ν

_{j})

*w*.

_{ij}In general, synaptic learning rules can be classified into two categories:
multiplicative and additive rules dependent on whether there is a scale factor
in the function *F* or firing rate and spike timing rules
dependent on which type of *F* is used. Combinations of these two
categories give four specific types of learning rules. Synaptic scaling is a
form of a multiplicative firing-rate rule, in which *F* of the SS
rule is dependent only on postsynaptic neural firing rate, whereas *F* of the PSD rule is dependent on both pre- and
postsynaptic neural firing rates. The matrix analysis conducted in this study
can be applied to other learning rules even when spike timing is considered.

As long as presynaptic activity (firing rate or spike timing) is considered in
any particular learning rule, the matrix structure of this rule also uses the
Hadamard product. In recent years, spike-timing-dependent plasticity (STDP) has
been identified experimentally and studied intensively (Bi & Poo, 2001; Morrison, Diesmann, & Gerstner, 2008). Since STDP needs information
from both pre- and postsynaptic spike times, the matrix analysis we have
explored can be applied. Other presynaptic dependent rules have been proposed in
the literature, such as heterosynaptic depression among all input synapses,
which has been shown to generate stable activity sequences within recurrent
networks (Fiete, Senn, Wang, & Hahnloser, 2010). However, there is no systematic way to study the stability of
these rules. Furthermore, it will be interesting to study the case where two or
more learning rules are used together (Liu & Buonomano, 2009; Fiete et al., 2010; Clopath, Bsing, Vasilaki, & Gerstner, 2010); in this case the order parameter *r*_{1} may have different values, and the stability of combined learning
rules may have a different stable phase. Our analysis may give a clue on this
issue, although further studies are needed.

## Appendix: Simulation of Spiking Neural Network

We used the same neural network model as described by Liu and Buonomano (2009), where simulations were performed using
NEURON (Hines & Carnevale, 1997) with a
time step *t*=0.1 ms. The codes programmed with C++ generated
similar results (Liu & She, 2009). The
code can be downloaded from the author's home page.

### A.1. Neural Dynamics.

*V*is described when

*V*<

*V*as where membrane time constants are 30 ms for all excitatory (E) (

_{thr}*g*=0.1 μS/cm

_{L}^{2};

*C*=3 μF/cm

^{2}) and inhibitory (I) neurons (

*g*=0.1 μS/cm

_{L}^{2};

*C*=1 μF/cm

^{2}). Neurons are heterogeneous in the sense that firing thresholds

*V*are set from a normal distribution (σ

_{thr}^{2}=5% of the mean) with the mean for the E(I) cells as −40(−45) mV. When

*V*is reached at the spiking time

_{thr}*t*,

_{spk}*V*is set to

*V*=40 mV for the duration of the spike (τ

_{peak}_{dur}=1 ms). After the spike,

*V*is reset to the repolarizing potential

*V*=−60(−65) mV for the E(I) cells; at the same time, the afterhyperpolarization

_{reset}*g*is turned on and changed as where τ

_{AHP}_{AHP}=10(2) ms for the E(I) cells. The Dirac function δ is used to set a stepwise increment of for the E(I) cells whenever a spike occurs. The refractory period is τ

_{ref}=2 ms for all neurons.

### A.2. Synaptic Dynamics.

*R*(

*u*) is the short-term depression (facilitation) variable with the time constant τ

_{rec}(τ

_{fac}), and subjects to the pulsed decrease

*uR*δ(

*t*−

*t*) (increase

_{n}*U*(1−

*u*)δ(

*t*−

*t*)) due to the spike at

_{n}*t*. The cumulative synaptic efficacy at any time is the product

_{spk}*Ru*that is incorporated into the single synaptic dynamics below. Specifically,

*E*→

*E*synapses exhibit depression:

*U*=0.5, τ

_{rec}=500 ms, τ

_{fac}=10 ms;

*E*→

*I*synapses exhibit facilitation

*U*=0.2, τ

_{rec}=125 ms, τ

_{fac}=500 ms. All inhibitory synapses exhibit depression as basket cell synapses (Gupta, Wang, & Markram, 2000):

*U*=0.25, τ

_{rec}=700 ms, τ

_{fac}=20 ms.

_{d}∈[0, 2]. The receptor activation

*r*(

*t*) for fast AMPA and GABA

_{A}dynamics follows two-state kinetic models (Destexhe, Mainen, & Sejnowski, 1994): where α=1.5 ms

^{−1}nM

^{−1}and β=0.75 ms

^{−1}for AMPA; α=0.5 ms

^{−1}nM

^{−1}and β=0.25 ms

^{−1}for GABA

_{A}.

*T*=1 nM is the presynaptic transmitter concentration. NMDA is modeled as (Golomb, Wang, & Rinzel, 1994; Buonomano, 2000): where for NMDA, α=0.06 ms

^{−1}, β=0.01 ms

^{−1}, τ

_{s}=50 ms, γ=0.5, θ=0.3, σ=0.5. In all synapses,

*Ru*is included for the short-term plasticity. The ratio of NMDA to AMPA synaptic weights is fixed as

*g*

_{NMDA}=0.6

*g*

_{AMPA}for all E-cells.

### A.3. Network Topology and Parameters.

All simulations are performed using a network with 400 excitatory (E) and 100
inhibitory (I) neurons connected with a probability 0.12 for E→E, and 0.2
for both E→I and I→E, which results in each postsynaptic E-neuron
receiving 48 inputs from other E-neurons and 20 inputs from I-neurons; each
postsynaptic I-neuron receives 80 inputs from E-neurons. Initial synaptic
weights are chosen from a normal distribution with mean
W_{EE}=2/48 nS, W_{EI}=1/80 nS, and
W_{IE}=2/20 nS, respectively, and SD
σ_{EE}=2W_{EE},
σ_{EI}=8W_{EI}, and
σ_{IE}=8W_{IE}. If the initial weights are
nonpositive, they are reset to a uniform distribution from 0 to twice the mean.
To avoid the induction of unphysiological states in which a single presynaptic
neuron fires a postsynaptic neuron, the maximal E→E AMPA synaptic weights
are W_{EE}^{max}=1.5 nS. The maximal E→I AMPA synaptic weights are
set to W_{EI}^{max}=0.4 nS. All inhibitory synaptic weights are fixed. For
learning rules, α_{w}=0.01 and . ν_{goal} is the target activity set to 1(2)Hz for E(I)-cells. A stimulus is
composed by randomly selected 24 E- and 12 I-cells that fire at 1 Hz. The input
spiking timings are assigned to 10±1 ms (mean ± SD) following a
normal distribution relative to the onset of each period of 1 s, thus, one
subset of cells fires at the beginning of each period. Selected input cells are
activated by a 1 Hz excitatory postsynaptic current.

## Acknowledgments

We thank Dean Buonomano, Tiago Carvacrol, and Tyler Lee for helpful discussions, and Nicolas Brunel, Claudia Clopath, Omri Harish, and David Higgins for comments and careful reading of the manuscript. This work was partially supported by the ANR-BBSRC Grant VESTICODE.