Nonequilibrium Statistical Mechanics of Continuous Attractors

Dynamical attractors have found much use in neuroscience as models for carrying out computation and signal processing (Poucet & Save, 2005). While point-like neural attractors and analogies to spin glasses have been widely explored (Hopfield, 1982; Amit, Gutfreund, & Sompolinsky, 1985b), an important class of experiments is explained by continuous attractors, where the collective dynamics of strongly interacting neurons stabilizes a low-dimensional family of activity patterns. Such continuous attractors have been invoked to explain experiments on motor control based on path integration (Seung, 1996; Seung, Lee, Reis, & Tank, 2000), head direction (Kim, Rouault, Druckmann, & Jayaraman, 2017) control, spatial representation in grid or place cells (Yoon et al., 2013; O'Keefe & Dostrovsky, 1971; Colgin et al., 2010; Wills, Lever, Cacucci, Burgess, & O'Keefe, 2005; Wimmer, Nykamp, Constantinidis, & Compte, 2014; Pfeiffer & Foster, 2013), among other information processing tasks (Hopfield, 2015; Roudi & Latham, 2007; Latham, Deneve, & Pouget, 2003; Burak & Fiete, 2012).

These continuous attractor models are at the fascinating intersection of dynamical systems and neural information processing. The neural activity in these models of strongly interacting neurons is described by an emergent collective coordinate (Yoon et al., 2013; Wu, Hamaguchi, & Amari, 2008; Amari, 1977). This collective coordinate stores an internal representation (Sontag, 2003; Erdem & Hasselmo, 2012) of the organism's state in its external environment, such as position in space (Pfeiffer & Foster, 2013; McNaughton et al., 2006) or head direction (Seelig & Jayaraman, 2015).

However, such internal representations are useful only if they can be driven and updated by external signals that provide crucial motor and sensory input (Hopfield, 2015; Pfeiffer & Foster, 2013; Erdem & Hasselmo, 2012; Hardcastle, Ganguli, & Giocomo, 2015; Ocko, Hardcastle, Giocomo, & Ganguli, 2018). Driving and updating the collective coordinate using external sensory signals opens up a variety of capabilities, such as path planning (Ponulak & Hopfield, 2013; Pfeiffer & Foster, 2013), correcting errors in the internal representation or in sensory signals (Erdem & Hasselmo, 2012; Ocko et al., 2018), and the ability to resolve ambiguities in the external sensory and motor input (Hardcastle et al., 2015; Evans, Bicanski, Bush, & Burgess, 2016; Fyhn, Hafting, Treves, Moser, & Moser, 2007).

In all of these examples, the functional use of attractors requires interaction between external signals and the internal recurrent network dynamics. However, with a few significant exceptions (Fung, Wong, Mao, & Wu, 2015; Mi, Fung, Wong, & Wu, 2014; Wu et al., 2008; Wu & Amari, 2005; Monasson & Rosay, 2014; Burak & Fiete, 2012), most theoretical work has either been in the limit of no external forces and strong internal recurrent dynamics, or in the limit of strong external forces where the internal recurrent dynamics can be ignored (Moser, Moser, & McNaughton, 2017; Tsodyks, 1999).

Here, we study continuous attractors in neural networks subject to external driving forces that are neither small relative to internal dynamics nor adiabatic. We show that the physics of the emergent collective coordinate sets limits on the maximum speed at which internal representations can be updated by external signals.

Our approach begins by deriving simple classical and statistical laws satisfied by the collective coordinate of many neurons with strong, structured interactions that are subject to time-varying external signals, Langevin noise, and quenched disorder. Exploiting these equations, we demonstrate two simple principles: (1) an equivalence principle that predicts how much the internal representation lags a rapidly moving external signal, and (2) under externally driven conditions, quenched disorder in network connectivity that can be modeled as a state-dependent effective temperature. Finally, we apply these results to place cell networks and derive a nonequilibrium driving-dependent memory capacity, complementing numerous earlier works on memory capacity in the absence of external driving.

As shown in Figure 1, $Jij0$ encodes the continuous attractor. We will focus on 1D networks with $p$-nearest neighbor excitatory interactions to keep bookkeeping to a minimum: $Jij0=J(1-ε)$ if neurons $|i-j|≤p$⁠, and $Jij0=-Jε$ otherwise. The latter term, $-Jε$⁠, with $0≤ε≤1$⁠, represents long-range, nonspecific inhibitory connections as frequently assumed in models of place cells (Monasson & Rosay, 2014; Hopfield, 2010), head direction cells (Chaudhuri & Fiete, 2016), and other continuous attractors (Seung et al., 2000; Burak & Fiete, 2012).

The disorder matrix $Jijd$ represents random long-range connections, a form of quenched disorder (Seung, 1998; Kilpatrick, Ermentrout, & Doiron, 2013). Finally, $Inext(t)$ represents external driving currents from, for example, sensory and motor input possibly routed through other regions of the brain. The Langevin noise $ηint(t)$ represents private noise internal to each neuron (Lim & Goldman, 2012; Burak & Fiete, 2012).

A neural network like equation 2.1 qualitatively resembles a similarly connected network of Ising spins at fixed magnetization (Monasson & Rosay, 2014). At low noise, the activity in such a system will condense (Monasson & Rosay, 2014; Hopfield, 2010) to a localized droplet, since interfaces between firing and nonfiring neurons are penalized by $J(1-ε)$⁠. The center of mass of such a droplet, $x¯≡∑nnf(in)∑nf(in)$ is an emergent collective coordinate that approximately describes the stable low-dimensional neural activity patterns of these $N$ neurons. Fluctuations about this coordinate have been extensively studied (Wu et al., 2008; Burak & Fiete, 2012; Hopfield, 2015; Monasson & Rosay, 2014).

We focus on how space and time-varying external signals, modeled here as external currents $Inext(t)$⁠, can drive and reposition the droplet along the attractor. We will be primarily interested in a cup-shaped current profile that moves at a constant velocity $v$⁠, $Inext(t)=Icup(n-vt)$⁠, where $Icup(n)=d(w-|n|),n∈[-w,w]$⁠, $Icup(n)=0$ otherwise. Such a localized time-dependent drive could represent landmark-related sensory signals (Hardcastle et al., 2015).

The strength of the external signal is set by the depth $d$ of the cup $Icup(n)$⁠. Previous studies have explored the $d=0$ case—undriven diffusive dynamics of the droplet (Burak & Fiete, 2012; Monasson & Rosay, 2013, 2014, 2015) or the large $d$ limit (Hopfield, 2015) when the internal dynamics can be ignored. Here we focus on an intermediate regime, $d<dmax$⁠, where internal representations are updated continuously by the external currents, without any jumps (Ponulak & Hopfield, 2013; Pfeiffer & Foster, 2013; Erdem & Hasselmo, 2012).

In fact, as shown in the section C.2 we find a threshold signal strength $dmax$ beyond which the external signal destabilizes the droplet, instantly “teleporting” the droplet from any distant location to the cup without continuity along the attractor, erasing any prior positional information held in the internal representation.

We focus here on $d<dmax$⁠, a regime with continuity of internal representations. Such continuity is critical for many applications, such as path planning (Ponulak & Hopfield, 2013; Pfeiffer & Foster, 2013; Erdem & Hasselmo, 2012) and resolving local ambiguities in position within the global context (Hardcastle et al., 2015; Evans et al., 2016; Fyhn et al., 2007). In this regime, the external signal updates the internal representation with finite gain (Fyhn et al., 2007) and can thus fruitfully combine information in both the internal representation and the external signal. Other applications that simply require short-term memory storage of a strongly fluctuating variable may not require this continuity restriction.

We first consider driving the droplet in a network at constant velocity $v$ using an external current $Inext=Icup(n-vt)$⁠. We allow for Langevin noise but no disorder in the couplings $Jd=0$ in this section. For very slow driving (⁠$v→0$⁠), the droplet will settle into and track the bottom of the cup. When driven at a finite velocity $v$⁠, the droplet cannot stay at the bottom since there is no net force exerted by the currents $Inext$ at that point.

Our results here are consistent with the fluctuation-dissipation result obtained in Monasson and Rosay (2014) for driven droplets. In summary, in the co-moving frame of the driving signal, the droplet's position $Δxv$ fluctuates as if it were in thermal equilibrium in the modified potential $Veff=Vcup-FvmotionΔxv$⁠.

The simple equivalence principle implies a striking bound on the update speed of internal representations. A driving signal cannot drive the droplet at velocities greater than some $vcrit$ if the predicted lag for $v>vcrit$ is larger than the cup. In the appendix, we find $vcrit=2d(w+R)/3γ$⁠, where $2R$ is the droplet size.

We now consider the effect of long-range quenched disorder $Jijd$ in the synaptic matrix (Seung, 1998; Kilpatrick et al., 2013), which breaks the exact degeneracy of the continuous attractor, creating an effectively rugged landscape, $Vd(x¯)$⁠, as shown schematically in Figure 3 and computed in sections E.1 and E.2. When driven by a time-varying external signal, $Iiext(t)$⁠, the droplet now experiences a net potential $Vext(x¯,t)+Vd(x¯)$⁠. The first term causes motion with velocity $v$ and a lag predicted by the equivalence principle, and for sufficiently large velocities $v$⁠, the effect of the second term can be modeled as effective Langevin white noise. To see this, note that $Vd(x¯)$ is uncorrelated on length scales larger than the droplet size; hence, for large enough droplet velocity $v$⁠, the forces $Fd(t)≡-∂x¯Vd|x¯=x¯(t)$ due to disorder are effectively random and uncorrelated in time. More precisely, let $σ2=Var(Vd(x¯))$⁠. In section E.3, we compute $Fd(t)$ and show that $Fd(t)$ has an autocorrelation time, $τcor=2R/v$⁠, due to the finite size of the droplet.

Thus, on longer timescales, $Fd(t)$ is uncorrelated and can be viewed as Langevin noise for the droplet center of mass $x¯$⁠, associated with a disordered-induced temperature $Td$⁠. Through repeated simulations with different amounts of disorder $σ2$⁠, we inferred the distribution $p(Δxv)$ of the droplet position in the presence of such disorder-induced fluctuations (see Figure 3). The data collapse in Figure 3b confirms that the effect of disorder (of size $σ2$⁠) on a rapidly moving droplet can indeed be modeled by an effective disorder-induced temperature $Td∼στcor$⁠. (For simplicity, we assume that internal noise $ηint$ in equation 2.1 is absent here. Note that in general, $ηint$ will also contribute to $Td$⁠. Here we focus on the contribution of disorder to an effective temperature $Td$ since internal noise $ηint$ has been considered in prior work (Fung et al., 2015).)

The capacity of a neural network to encode multiple memories has been studied in numerous contexts since Hopfield's original work (Hopfield, 1982). While specifics differ (Amit, Gutfreund, & Sompolinsky, 1985a; Battaglia & Treves, 1998; Monasson & Rosay, 2014; Hopfield, 2010), the capacity is generally set by the failure to retrieve a specific memory because of the effective disorder in neural connectivity due to other stored memories.

However, these works on capacity do not account for nonadiabatic external driving. Here, we use our results to determine the capacity of a place cell network (O'Keefe & Dostrovsky, 1971; Battaglia & Treves, 1998; Monasson & Rosay, 2014) to both encode and manipulate memories of multiple spatial environments at a finite velocity. Place cell networks (Tsodyks, 1999; Monasson & Rosay, 2013, 2014, 2015) encode memories of multiple spatial environments as multiple continuous attractors in one network. Such networks have been used to describe recent experiments on place cells and grid cells in the hippocampus (Yoon et al., 2013; Hardcastle et al., 2015; Moser, Moser, & Roudi, 2014).

In experiments that expose a rodent to different spatial environments $μ=1,…M$ (Alme et al., 2014; Moser, Moser, & McNaughton, 2017; Moser, Moser, & Roudi, 2014; Kubie & Muller, 1991), the same place cells $i=1,…,N$ are seen having “place fields” in different spatial arrangements $πμ(i)$ as seen in Figure 4a, where $πμ$ is a permutation specific to environment $μ$⁠. Consequently, Hebbian plasticity suggests that each environment $μ$ would induce a set of synaptic connections $Jijμ$ that corresponds to the place field arrangement in that environment: $Jijμ=J(1-ε)$ if $|πμ(i)-πμ(j)|<p$⁠. That is, each environment corresponds to a 1D network when the neurons are laid out in a specific permutation $πμ$⁠. The actual network has the sum of all these connections $Jij=∑μ=1MJijμ$ over the $M$ environments the rodent is exposed to.

While $Jij$ above is obtained by summing over $M$ structured environments, from the perspective of, say, $Jij1$⁠, the remaining $Jijμ$ look like long-range disordered connections. We will assume that the permutations $πμ(i)$ corresponding to different environments are random and uncorrelated, a common modeling choice with experimental support (Hopfield, 2010; Monasson & Rosay, 2014, 2015; Alme et al., 2014; Moser et al., 2017). Without loss of generality, we assume that $π0(i)=i$ (blue environment in Figure 4.) Thus, $Jij=Jij0+Jijd,Jijd=∑μ=1M-1Jijμ$⁠. The disordered matrix $Jijd$ then has an effective variance $σ2∼(M-1)/N$⁠. Hence, we can apply our previous results to this system. Now consider driving the droplet with velocity $v$ in environment 1 using external currents. The probability of successfully updating the internal representation over a distance $L$ is given by $Pretrieval=e-rL/v$⁠, where $r$ is given by equation 5.1.

We have considered continuous attractors in neural networks driven by localized time-dependent currents $Icup(n-vt)$⁠. In recent experiments, such currents can represent landmark-related sensory signals (Hardcastle et al., 2015) when a rodent is traversing a spatial environment at velocity $v$ or signals that update the internal representation of head direction (Seelig & Jayaraman, 2015). Several recent experiments have controlled the effective speed of visual stimuli in virtual reality environments (Meshulam, Gauthier, Brody, Tank, & Bialek, 2017; Aronov, Nevers, & Tank, 2017; Kim et al., 2017; Turner-Evans et al., 2017). Other experiments have probed cross-talk between memories of multiple spatial environments (Alme et al., 2014). Our results predict an error rate that rises with speed and with the number of environments.

While our analysis used specific functional forms for, among others, the current profile $Icup(n-vt)$⁠, our bound simply reflects the finite response time in moving emergent objects, much like moving a magnetic domain in a ferromagnet using space and time-varying fields. Thus, we expect our bound to hold qualitatively for other related forms (Hopfield, 2015).

In addition to positional information considered here, continuous attractors are known to also receive velocity information (Major, Baker, Aksay, Seung, & Tank, 2004; McNaughton et al., 2006; Seelig & Jayaraman, 2015; Ocko et al., 2018). We do not consider such input in the main text but extend our analysis to velocity integration in appendix D.

In summary, we found that the nonequilibrium statistical mechanics of a strongly interacting neural network can be captured by a simple equivalence principle and a disorder-induced temperature for the network's collective coordinate. Consequently, we were able to derive a velocity-dependent bound on the number of simultaneous memories that can be stored and retrieved from a network. We discussed how these results, based on general theoretical principles on driven neural networks, allow us to connect robustly to recent time-resolved experiments in neuroscience (Kim et al., 2017; Turner-Evans et al., 2017; Hardcastle et al., 2015; Hardcastle, Maheswaranathan, Ganguli, & Giocomo, 2017; Campbell et al., 2018) on the response of neural networks to dynamic perturbations.

The description of neural activity in terms of such a collective coordinate $x¯$ greatly simplifies the problem, reducing the configuration space from the $2N$ states for the $N$ neurons to $N$-state, and consists of the center of mass of the droplet along the continuous attractor (Wu et al., 2008). The computational abilities of these place cell networks, such as spatial memory storage, path planning, and pattern recognition, are limited to parameter regimes in which such a collective coordinate approximation holds (e.g., noise levels less than a critical value $T<Tc$⁠) .

The droplet can be driven by external signals such as sensory or motor input or input from other parts of the brain. We model such external input by the currents $Inext$ in equation A.1—for example, sensory landmark-based input (Hardcastle et al., 2015). When an animal is physically in a region covered by place fields of neurons $i,i+1,…,i+z$⁠, currents $Iiext$ through $Ii+zext$ can be expected to be high compared to all other currents $Ijext$⁠. Other models of driving in the literature include adding an antisymmetric component $Aij$ to synaptic connectivities $Jij$ (Ponulak & Hopfield, 2013); we consider such a model in appendix D.

For a quasi 1D network with $p$-nearest neighbor interactions and no disorder, $VJ(x¯)$ is constant, giving a smooth, continuous attractor. However, as discussed later, in the presence of disorder, $VJ(x¯)$ has bumps (i.e., quenched disorder) and is no longer a smooth, continuous attractor.

We next numerically verify that the droplet obeys a fluctuation-dissipation-like relation by driving the droplet using external currents $Iext$ and comparing the response to diffusion of the droplet in the absence of external currents.

We use a finite ramp as the external driving, $Inext=n$ with $n<nmax$⁠, and $Inext=0$ otherwise (see Figure 5a). We choose $nmax$ to be such that it takes considerable time for the droplet to relax to its steady-state position at the end of the ramp. We notice that for different slopes of the $Inext$⁠, the droplet has different velocities, and it is natural to define a mobility of the droplet, $μ$⁠, by $v=μf$⁠, where $f$ is the slope of $Inext$⁠. Next, we notice that on a single continuous attractor, the droplet can diffuse because of internal noise in the neural network. Therefore, we can infer the diffusion coefficient $D$ of the droplet from $〈x2〉=2Dt$ for a collection of diffusive trajectories (see Figure 5b), where we have used $x$ to denote the center of mass $x¯$ for the droplet to avoid confusion.

Note that equation A.9 has also been derived for the case of binary neurons with a hard constraint on the number of firing population (Monasson & Rosay, 2014).

We consider the model of sensory input used in the main text: $Icup(n)=d(w-|n|),n∈[-w,w]$⁠, $Icup(n)=0$ otherwise. We focus on time-dependent currents $Inext(t)=Icup(n-vt)$⁠. Such a drive was previously considered in Wu and Amari (2005), albeit without time dependence. Throughout this article, we refer to $w$ as the linear size of the drive, $d$ as the depth of the drive, and set the drive moving at a constant velocity $v$⁠. From now on, we will go to the continuum limit and denote $Inext(t)=Iext(n,t)≡Iext(x,t)$⁠.

We plot $Vext$ given by equation B.1 versus the c.o.m. position of droplet in Figure 6a.

The equivalence principle we introduced in the main text allows us to compute the steady-state position and the effective new potential seen in the co-moving frame. Crucially, the fluctuations of the collective coordinate are described by the potential obtained through the equivalence principle. The principle correctly predicts both the mean (see equation 3.2) and the fluctuation (see equation 3.3) of the lag $Δxv$⁠. Therefore, it is actually a statement about the equivalence of effective dynamics in the rest frame and in the co-moving frame. Specializing to the drive $Icup(x,t)$⁠, the equivalence principle predicts that the effective potential felt by the droplet (moving at constant velocity $v$⁠) in the co-moving frame equals the effective potential in the stationary frame shifted by a linear potential, $Vlin=-FvmotΔxv$⁠, that accounts for the fictitious forces due to the change of coordinates (see Figure 6c).

Since we used equation B.1 for the cup shape and the lag $Δxv$ depends linearly on $v$⁠, we expect that the slope of the linear potential $Vlin$ also depends linearly on $v$⁠. Here the sign convention is chosen such that $Vlin<0$ corresponds to the droplet moving to the right.

In the following, we work in the co-moving frame with velocity $v$ at which the driving signal is moving. We denote the steady-state c.o.m. position in this frame to be $Δxv*$ and a generic position to be $Δxv$⁠.

We are now in position to derive $vcrit$ presented in the main text. We observe that as the driving velocity $v$ increases, $Δxv*$ and $Δxvesc$ will get closer to each other, and there will be a critical velocity such that the two coincide.

Compared to the equation of motion (e.o.m.), equaiton A.1, we see that the first term corresponds to the decay of neurons in the absence of interaction from neighbors (decay from the on state to the off state), and the second term corresponds to the interaction $Jnk$ term in the e.o.m, and the third term corresponds to the $Inext$ in the e.o.m. Since we are interested in the steady-state droplet size, and thus only in the neurons that are on, the effect of the first term can be neglected (also note that $1/τ≪Jij$⁠; when using the Lyapunov function to compute steady-state properties, the first term can be ignored).

To obtain general results, we also account for long-range disordered connections $Jijd$ here. We assume $Jijd$ consists of random connections among all the neurons. We can approximate these random connections as random permutations of $Jij0$⁠, and the full $Jij$ is the sum over $M-1$ such permutations plus $Jij0$⁠.

For the cup-shaped driving and its corresponding effective potential, equation C.1, we are interested in the steady-state droplet size under this driving, so we first evaluate $Veff$ at the steady-state position $Δxv*$ in equation C.2. To make the $R$-dependence explicit in the Lyapunov function, we evaluate $L(x¯)$ under the rigid bump approximation used in Hopfield (2015), assuming $f(ikx¯)=1$ for $|k-x¯|≤R$⁠, and $=0$ otherwise.

Here we present the calculation for maximal driving strength $Iext$ beyond which the activity droplet will “teleport”—that is, disappear at the original location and recondense at the location of the drive, even if these two locations are widely separated. We now refer to this maximal signal strength as the teleportation limit. We can determine this limit by finding out the critical point where the energy barrier of breaking up the droplet at the original location is zero.

For simplicity, we assume that initially, the cup-shaped driving signal is some distance $x0$ from the droplet and not moving (the moving case can be solved in exactly the same way by using the equivalence principle and going to the co-moving frame of the droplet). We consider three scenarios during the teleportation process. (1) In the initial configuration, the droplet has not yet teleported and stays at the original location with radius $R(0,0,m)$⁠. (2) In the intermediate configuration, the activity is no longer contiguous, giving a droplet with radius $δ(d,w,m)$ at the center of the cup, and another droplet with radius $R(d,w,m)-δ(d,w,m)$ at the original location (when teleportation happens, the total firing neurons changes from $R(0,0,m)$ to $R(d,w,m)$⁠). (3) In the final configuration, the droplet has successfully teleported to the center of the cup, with radius $R(d,w,m)$⁠. The three scenarios are depicted schematically in Figure 7.

The global minimum of the Lyapunov function corresponds to scenario 3. However, there is an energy barrier between configuration 1 and configuration 3, corresponding to the $Veff$ difference between configuration 1 and 2. We would like to find the critical split size $δc(d,w,m)$ that maximizes the difference in $Veff$⁠, which corresponds to the largest energy barrier the network has to overcome in order to teleport from configuration 1 to 3. For the purpose of derivation, in the following we rename $L[f(ikm)]$ in equation C.5 as $E0(d,w,m)∥R(d,w,m)$ to emphasize its dependence on the external driving parameters and disordered interactions. The subscript 0 stands for the default one-droplet configuration, and it is understood that $E0(d,w,m)$ is evaluated at the network configuration of a single droplet at location $m$ with radius $R(d,w,m)$⁠.

Now we have obtained the maximum energy barrier during a teleportation process, $ΔE∥δc$⁠. A spontaneous teleportation will occur if $ΔE∥δc≤0$⁠, and this in turn gives an upper bound on the external driving signal strength $d≤dmax$ one can have without any teleportation spontaneous occurring.

We plot the numerical solution of $dmax$ obtained from the solving $ΔE(dc,w,m)∥δc=0$⁠, compared with results obtained from the simulation in Figure 8, and find perfect agreement.

We also obtain an approximate solution by observing that the only relevant scale for the critical split size $δc$ is the radius of the droplet, $R$⁠. We set $δc=cR$ for some constant $0≤c≤1$⁠. In general, $c$ can depend on dimensionless parameters like $p$ and $ε$⁠. Empirically we found the constant to be about 0.29 in our simulation.

Note that the denominator is positive because $w>R$ and $0≤c≤1$⁠. The simulation result also confirms that the critical split size $δc$ stays approximately constant. We have checked that the dependence on parameters $J,w,m$ in equation C.10 agrees with the numerical solution obtained from solving $Ebar(dc,w,m)∥δc=0$⁠, up to the undetermined constant $c$⁠.

Place cell networks (Ocko et al., 2018) and head direction networks (Kim et al., 2017) are known to receive information about both velocity and landmark information. Velocity input can be modeled by adding an antisymmetric part $Aij$ to the connectivity matrix $Jij$⁠, which effectively tilts the continuous attractor.

The antisymmetric part $Aij0$ will provide a velocity $v$ that is proportional to the size $A$ of $Aij0$ for the droplet (see Figure 9). In the presence of disorder, we can simply go to the co-moving frame of velocity $v$⁠, and the droplet experiences an extra disorder-induced noise $ηA$ in addition to the disorder-induced temperature $Td$⁠.

We found that $〈ηA(t)ηA(0)〉∝σ˜δ(t)$ (see Figure 10), where $σ˜2$ is the average number of disordered connection per neuron in units of $2p$⁠.

Therefore, all our results in the main text apply to the case when both the external drive $Iext(x,t)$ and the antisymmetric part $Aij0$ exist. Specifically, we can just replace the velocity $v$ used in the main text as the sum of the two velocities corresponding to $Iext(x,t)$ and $Aij0$⁠.

From now on, we include disorder connections $Jijd$ in addition to ordered connections $Jij0$ that correspond to the nearest $p$-neighbor interactions. We assume $Jijd$ consists of random connections among all the neurons. These random connections can be approximated as random permutations of $Jij0$⁠, such that the full $Jij$ is the sum over $M-1$ such permutations plus $Jij0$⁠.