Attractor models are simplified models used to describe the dynamics of firing rate profiles of a pool of neurons. The firing rate profile, or the neuronal activity, is thought to carry information. Continuous attractor neural networks (CANNs) describe the neural processing of continuous information such as object position, object orientation, and direction of object motion. Recently it was found that in one-dimensional CANNs, short-term synaptic depression can destabilize bump-shaped neuronal attractor activity profiles. In this article, we study two-dimensional CANNs with short-term synaptic depression and spike frequency adaptation. We found that the dynamics of CANNs with short-term synaptic depression and CANNs with spike frequency adaptation are qualitatively similar. We also found that in both kinds of CANNs, the perturbative approach can be used to predict phase diagrams, dynamical variables, and speed of spontaneous motion.
Neurons communicate with each other through the neurotransmitter diffusion initiated by action potentials, or spikes, and the activity of one neuron can excite or inhibit the activity of another neurons. The firing rates of spike trains are thought to carry information, and correlations between the firing rates of neurons depend on the strength of the couplings between those neurons. With some specific settings of couplings, such as Mexican hat couplings (Amari, 1977; Ben-Yishai, Bar-Or, & Sompolinsky, 1995), networks of neurons can support a continuous family of local neuronal activity profiles on a field, which can be used to represent continuous information, such as object position, object orientation, and direction of object motion.
Local neuronal activities associated with continuous information are observed in various brain regions. Typical examples of cells showing such activity are head direction cells (Taube, Muller, & Ranck, 1990; Blair & Sharp, 1995; Zhang, 1996; Taube & Muller, 1998), place cells (O’Keefe & Dostrovsky, 1971; O’Keefe, 1976; O’Keefe & Burgess, 1996; Samsonovich & McNaughton, 1997) and moving-direction cells (Maunsell & van Essen, 1983; Treue, Hol, & Rauber, 2000). These cells have gaussian-like tuning curves as functions of stimulus. Among the numerous models proposed to describe behaviors of these systems are continuous attractor neural networks (CANNs), and a recent study of persistent activity in monkey prefrontal cortices has provided evidence of continuous attractors in the central nervous system (Wimmer, Nykamp, Constantinidis, & Compte, 2014). Persistent neuronal activity in monkeys’ prefrontal cortices was discovered during delayed-response tasks (Funahashi, Bruce, & Goldman-Rakic, 1993). The study by Wimmer et al. (2014) confirmed that pairwise neuronal correlation predicted by theories can be observed in the brain region they investigated (Ben-Yishai et al., 1995; Pouget, Zhang, Deneve, & Latham, 1998; Wu, Hamaguchi, & Amari, 2008).
A one-dimensional (1D) CANN can support a family of gaussian-like neuronal activity profiles. They are attractors of the network. As shown in Figure 1a, attractors have the same shapes and are centered at positions corresponding to different preferred stimuli. These gaussian-like profiles can shift smoothly along the space of attractors. Families of attractor profiles in two-dimensional (2D) CANNs are similar: each attractor profile is a gaussian-like profile and centered at a particular position in the 2D field (see Figure 1b). The dynamics of a network state profile to track a stimulus is widely studied in the literature (Ben-Yishai et al., 1995; Samsonovich & McNaughton, 1997; Wu et al., 2008; Fung, Wong, & Wu, 2010). If couplings between neurons are static over time (quenched), the steady state of neuronal activity profiles will be static because of the homogeneity and translational invariance of CANNs. If they depend on the firing histories of presynaptic neurons, however, the dynamics of CANNs can be different.
Tsodyks and Markram (1997) proposed a model in which the synaptic efficacies between neurons depend on the amount of available neurotransmitters in the presynaptic neurons, and this amount depends on the firing history of the presynaptic neuron (Tsodyks and Markram, 1997; Tsodyks, Pawelzik, & Markram, 1998). This kind of reduction in synaptic efficacies, due to past presynaptic neuronal activity, is called short-term synaptic depression (STD). There are reports that short-term synaptic depression can exhibit rich dynamics in CANNs (York & van Rossum, 2009; Fung, Wong, Wang, & Wu, 2012). Fung, Wong, Wang et al. (2012) reported that in 1D CANNs, short-term synaptic depression can destabilize attractor profiles. Within a broad range of strengths of divisive global inhibition, if we increase the degree of STD to a moderate range, static activity profiles will be translationally destabilized. After some translational perturbations, a bump-shaped profile can move spontaneously along the attractor space. In these scenarios, both bump-shaped static states and moving states can coexist. If we further increase the degree of STD, no static profile can be found. If the degree of STD is too large, the steady state of the system can only be a trivial solution. Similar behavior is reported by York and van Rossum (2009) with a different model. We can see that the instability induced by STD can reshape the intrinsic dynamics of the system, even when no stimulus is presented.
Network response in a CANN with static couplings is always lagging behind a continuously moving stimulus. However, with short-term synaptic depression, due to the translational instability, the underlying dynamics of the network can make the response overtake the actual stimulus. Fung, Wong, and Wu (2012) suggested that this behavior can be used to implement a delay compensation mechanism. Short-term synaptic depression can also induce global (Loebel & Tsodyks, 2002) and local (Fung, Wang, Lam, Wong, & Wu, 2013; Wang, Lam, Fung, Wong, & Wu, 2014) periodic excitements of neuronal activity profiles. These periodic excitements can enhance information processing in the brain. Fung et al. (2013) recently proposed that periodic excitement driven by the intrinsic dynamics can improve the resolution of CANNs. Kilpatrick (2013) also proposed that the neuronal activity pattern may shift between one stimulus and another. These theories suggested that STD may enhance the capability of CANNs.
Studies on CANNs with short-term synaptic depression are mainly on 1D networks. In this article, we discuss the intrinsic dynamics of bump-shaped solutions of two-dimensional CANNs with short-term synaptic depression. Other 2D models possess rich dynamical behaviors such as spiral waves (Kilpatrick & Bressloff, 2010a), breathing pulses (Kilpatrick & Bressloff, 2010a), and collisions of two bump-shaped profiles (Lu, Sato, & Amari, 2011). Here, we focus on the spontaneous motion of a single bump-shaped profile and analyze its stability. We have also studied the influence of STD on the sizes of bump-shaped profiles and their changes in shape.
We study not only CANNs with STD but also CANNs with spike frequency adaptation. Spike frequency adaptation (SFA) is a dynamical feature commonly observed in neurons. Neurons are suppressed after prolonged firing. SFA can be generated by a number of mechanisms (Brown & Adams, 1980; Madison & Nicoll, 1984; Fleidervish, Friedman, & Gutnick, 1996; Benda & Herz, 2003). It can also destabilize the amplitudes and positions of static bumps. SFA-induced destabilization of bump-shaped states in CANNs is reported in the literature (Kilpatrick & Bressloff, 2010b). What we find in our study on CANNs with SFA is similar to what we found in the case with STD. SFA first destabilizes the translational mode and then the amplitudal mode. There is also a parameter region such that both spontaneous-moving-bump solutions and static-bump solutions can coexist.
In this article, in each case, CANNs with STD and CANNs with SFA, we first introduce the model we used to study the problem and then analyze each scenario using the perturbative method proposed by Fung et al. (2010). These sections are followed by a section discussing the comparison between theoretical and simulation results and discussing the limitations of the perturbative method.
2 The Model
In this study, we analyze CANNs with STD and CANNs with SFA separately. For the case of CANNs with STD, we set . For CANNs with SFA, we set .
3 CANNs with STD
3.1 Stationary Solution
As in the study by Fung, Wong, Wang et al. (2012), can be replaced by rescaled variables because has a dimension . . And k and can be rescaled by and .
3.1.1 Translational Instability
We have simplified equations 2.2, 2.4, and 2.5 by introducing the approximation given by equations 3.5 and 3.6. This simplification, however, is useful for studying only the amplitudal stability of a bump-shaped solution. For the translational stability, we need to consider the stability of the static solution against asymmetric distortions. Displacing originally aligned and profiles of a static solution is a reasonable test, as the dip of is generated by activities of neurons. If the solution is moving, the dip of is always lagging behind. As a result, the asymmetric component of with respect to the center of mass of becomes nonzero. In the calculation, we may drop asymmetric components of for the moment, as we can always choose a frame such that the major asymmetric mode is zero.
3.2 Moving Solution
Since B is proportional to , an increase of implies a decrease in . In Figure 5a, the slope of is discontinuous at about . For , the slope is significantly larger than that in the region of . This implies that the motion of a bump helps the bump to maintain its magnitude. It also agrees with the tendency of the average membrane potential (or neuronal activity) profile in 1D CANN that the bump tends to move to a region with a higher concentration of neurotransmitters (York & van Rossum, 2009; Fung, Wong, Wang et al., 2012).
Figure 5c suggests that an anisotropic mode happens only when the bump is not static, while Figure 5b shows that the average width of profile increases with the strength of STD. The behavior of the average change in width, , is similar to that in height. The anisotropic mode happens only when the bump is moving. This suggests that the bump get widened unevenly due to the asymmetric profile.
3.3 Phase Diagram
In section 3.2, we showed that the perturbative method can successfully predict different modes of distortions of and the intrinsic speed of spontaneous motion. We also predicted the phase boundary separating translationally stable static bumps and translationally unstable static bumps. To predict different phases in the parameter space, however, we need to study the stability of fixed-point solutions given by equations 3.21 to 3.23. A detailed discussion of the stability issue can be found in appendix C.
By studying the stability of fixed-point moving solutions, we can obtain a phase boundary separating silent and moving phases, as plotted as a dashed line in Figure 4. In Figure 4, there is a rather complete phase diagram for the model of 2D CANNs with STD we are studying here. For small enough s (below the solid curve in Figure 4), since static bumps are stable in amplitude and translation, this parameter region supports only static bumps. This region is called the static phase.
For parameters between the dotted line and the solid line, static bumps are stable in amplitude but unstable against translational distortions. So in this case, in a rather short time window, both static bumps and moving bumps are able to exist, because once a translationally unstable static bumps get perturbed, it becomes a moving bump. Since both static bumps and moving bumps are observed in this parameter region, we call it the bistable phase. If we further increase , however, even static bumps cannot be observed. And since only moving bumps can be observed in the parameter region, we call it the moving phase. When both and are large, no nontrivial solution can be found in our analysis or simulations.
In this study, we did not find the homogeneous firing patterns reported by York and van Rossum (2009). In the 1D case, however, homogeneous firing can be found in the regime (Wang et al., 2014). Since we are focusing on moderate magnitudes of , the possibility of uniform firing behavior in 2D CANNs with STD near is reserved for future studies.
4 CANNs with SFA
4.1 Stationary Solution
4.2 Translational Stability
4.3 Moving Solution
Once is satisfied, spontaneous motion of a bump-shaped solution in 2D CANN with SFA becomes possible. As in the STD case, however, lower-order expansions of and are not sufficient to describe the moving solutions. We have to consider higher-order expansions if we want to obtain good predictions on the behavior of moving solutions. Surprisingly, we found that a limited order of expansion of and can give some fairly good predictions on the moving solutions.
The behavior of moving solutions of CANNs with SFA is similar to that of moving solutions of CANNs with STD. For , its trend is basically the same as in the case with STD shown in Figure 5. The transition happens whenever . Also, anisotropic modes will be available only when the bump is moving, while the width of the profile increases as the strength of SFA increases. The behavior of the average change in width is similar to the behavior of the change in height, . The dependence of anisotropic modes on the strength of SFA, , implies that the widening effect is not uniform when the bump is moving, which is similar to what is seen in a 2D CANN with STD.
4.4 Phase Diagram
A phase diagram similar to Figure 4 for SFA can be obtained in a similar manner used for CANNs with STD. In section 4.3, we showed that perturbative analysis is able to predict dynamical variables of the model. By solving for the moving solutions numerically and testing their stability, we can predict the phase boundary separating parameter regions for moving solutions and trivial solution (silent phase). Using it together with the stability conditions for static bumps given in equations 4.6 and 4.13, we can predict the phase diagram for SFA. We found that the predicted phase diagram matches the phase diagram obtained from computer simulations (see Figure 10). As in Figure 4, there are four phases in the phase diagram: a static phase, a bistable phase, a moving phase, and a silent phase. Their meanings are the same as those of their counterparts in Figure 4. Remarkably, expansions up to can also predict the phase diagram well, which is similar to the prediction of intrinsic speed of moving bumps in CANNs with SFA.
5.1 Intrinsic Phases and Phase Diagrams
In the parameter spaces of the two models we studied in this article, there are static, bistable, moving, and silent phases. When is large enough, moving bumps can be found in the bistable and moving phases. This behavior can also be seen in the case of SFA. In both the STD and SFA cases, whenever or is not too large (within the bistable phase), static bumps can still exist for a not insignificant period of time. If or is too large, only trivial solutions are stable.
Behaviors of CANNs with STD are very similar to those of CANNs with SFA. This suggests that multiplicative dynamical suppression mechanisms (represented by STD) and subtractive dynamical suppression mechanisms (represented by SFA) can generate similar intrinsic dynamics, especially with regard to spontaneous motion. It also implies that other smooth models having homogeneous couplings and dynamical suppression mechanisms should have a similar phase diagram consisting of static, moving, bistable, and silent phases.
In some studies on CANNs with STD or SFA it is reported that with some parameters, a uniform firing pattern can be found in the network (York & van Rossum, 2009). In this study, uniform firing is not included because the interactive range of the global inhibition in this model is infinite, so uniform firing may be found only in a tight parameter region near . For the 1D STD case, uniform firing can be found in a region and (Wang et al., 2014). Uniform firing should also be found in 2D CANNs with STD or SFA, as should spiral waves and breathing wave fronts. There is a report on spiral waves and breathing wavefronts in 2D CANNs with STD (Kilpatrick & Bressloff, 2010a), but those phenomena are missing from the phase diagram we predicted because spiral waves and breathing wave fronts are not local patterns. Therefore, in our model, they may exist if the magnitude of divisive global inhibition is very small. Richer dynamics of the model for are reserved for future investigations.
5.2 Effectiveness of Perturbative Approach
In this article, we have shown that in our particular model, the perturbative expansion method is applicable to the study of CANNs with short-term synaptic depression (STD) or spike frequency adaptation (SFA). We found in this study that both STD and SFA can drive similar intrinsic dynamics of the local neural activity profile. In Figures 4 and 10, there are four phases: silent, static, bistable, and moving. Using perturbative expansions on the dynamical variables, we can successfully predict the phase diagrams in both cases with STD and SFA.
As expected, with low-order expansions (i.e., when is small), predictions on s of the static solutions work well in cases with STD and SFA. For moving solutions, higher-order expansions are needed to obtain more accurate solutions. In Figures 5a to 5c and Figure 9, it is shown that low-order perturbative expansions, for STD and for SFA, are able to show the general trend of dynamical variables. Especially, the second-order transitions in all the dynamical variables (i.e., discontinuities of slopes) are observed at this level of perturbative expansion. This suggests that low-order perturbative expansions are good enough for studying general phenomena in different phases.
To predict the behavior of the dynamical variables more accurately, we need to use higher-order perturbative expansions. We have shown in Figures 5a to 5c and Figure 9 that higher-order perturbative expansions, for STD and for SFA, can fit measurements from simulations accurately. Higher-order terms do not, however, significantly improve prediction of the speed of spontaneous motion. This suggests that lower-order perturbative modes are most important to the motion of the local neural activity profile.
5.3 Asymmetric Modes and Moving Solutions
Higher-order perturbative modes are essential for predicting the behavior of dynamical variables of moving solutions accurately because the dynamical variables will become asymmetric about the center of mass of if the neural activity profile is moving. In Figure 11, there are two examples of CANNs with different levels of STD. Figures 11a and 11c are with , which corresponds to the static phase. In this case, both the average membrane potential profile and the fraction of available neurotransmitters are rotationally symmetric about the center of mass of . Therefore, in the static phase, low-order perturbation is good enough for making predictions.
Snapshots of and in the moving phase are shown in Figures 11b and 11d. Here the level of STD is . In this case, seems to be rotationally symmetric about its center of mass. Actually, the shape of the gaussian-like average membrane potential profile is slightly squeezed along the moving direction. For , however, the deformation of the shape is more significant. In Figure 11d, the profile of is strongly biased against the moving direction. This highly skewed profile of makes high-order perturbative mode more important, especially for cases involving stability and predictions of dynamical variables.
5.4 Limitation of Perturbative Approach
In this article, we have shown the perturbative approach is able to successfully predict the phase diagrams of CANNs with STD or SFA, the speed of spontaneous motion, and the dynamical variables (e.g., ). Phenomena richer than spontaneous motion of local neural activity, for example, breathing wave fronts on neural networks, have been considered theoretically (Kilpatrick & Bressloff, 2010a). The perturbative approach is not applicable in that case, however, because the basis functions we used here are local, while the traveling wave fronts are globally spreading. For the same reason, this method cannot be used to analyze spiral waves on a 2D field.
In the case of collisions of two moving bump-shaped profiles of neuronal activity, one may find the dynamics hard to analyze by the perturbative method. This is because there are two centers of mass, one for each bump, and this makes the choice of the origin of basis functions confusing. For example, if one chose one of the centers of mass to be the origin of basis functions, the chosen family of basis functions will not be optimal for the other bump and will make the expansion of that bump less efficient.
In this article, we studied models with short-term synaptic depression and spike frequency adaptation based on a two-dimensional CANN model with divisive global inhibition. We found that their intrinsic dynamics were similar. First, there are four phases in each scenario: static, moving, bistable, and silent. In the moving phase, the bump-shaped profile moves spontaneously, while in the static phase, the bump-shaped profile cannot move spontaneously. Interestingly, in the bistable phase, the CANN can support both moving profiles and static profiles.
Second, there are clear phase transitions between static solutions and moving solutions. In Figures 5, 8, and 9 there is a clear discontinuity in slope between the two states. This suggests that spontaneous motion can affect the shapes of the bump-shaped profiles.
The stability of steady states, the shapes of the dynamical variables, and the speed of spontaneous motion can be predicted by perturbative analysis, but the perturbative method cannot be used in some scenarios, including spiral waves, breathing wavefronts and collisions of multiple bumps in 2D fields.
Appendix A: Amplitudal Stability of Bumps on 2D CANNs with STD
Appendix C: Stability of Moving Bumps on 2D CANNs with STD
Appendix D: Amplitudal Stability of Bumps on 2D CANNs with SFA
Appendix E: Recurrence Relation of for CANNs with SFA
Appendix F: Intrinsic Speed of Moving Bumps on 2D CANNs with SFA
Appendix G: Stability of Moving Bumps on 2D CANNs with SFA
This study is partially supported by the Research Grants Council of Hong Kong (grants 604512, 605813 and N_HKUST 606/12).