Continuous attractors have been used to understand recent neuroscience experiments where persistent activity patterns encode internal representations of external attributes like head direction or spatial location. However, the conditions under which the emergent bump of neural activity in such networks can be manipulated by space and time-dependent external sensory or motor signals are not understood. Here, we find fundamental limits on how rapidly internal representations encoded along continuous attractors can be updated by an external signal. We apply these results to place cell networks to derive a velocity-dependent nonequilibrium memory capacity in neural networks.

Dynamical attractors have found much use in neuroscience as models for carrying out computation and signal processing (Poucet & Save, 2005). While point-like neural attractors and analogies to spin glasses have been widely explored (Hopfield, 1982; Amit, Gutfreund, & Sompolinsky, 1985b), an important class of experiments is explained by continuous attractors, where the collective dynamics of strongly interacting neurons stabilizes a low-dimensional family of activity patterns. Such continuous attractors have been invoked to explain experiments on motor control based on path integration (Seung, 1996; Seung, Lee, Reis, & Tank, 2000), head direction (Kim, Rouault, Druckmann, & Jayaraman, 2017) control, spatial representation in grid or place cells (Yoon et al., 2013; O'Keefe & Dostrovsky, 1971; Colgin et al., 2010; Wills, Lever, Cacucci, Burgess, & O'Keefe, 2005; Wimmer, Nykamp, Constantinidis, & Compte, 2014; Pfeiffer & Foster, 2013), among other information processing tasks (Hopfield, 2015; Roudi & Latham, 2007; Latham, Deneve, & Pouget, 2003; Burak & Fiete, 2012).

These continuous attractor models are at the fascinating intersection of dynamical systems and neural information processing. The neural activity in these models of strongly interacting neurons is described by an emergent collective coordinate (Yoon et al., 2013; Wu, Hamaguchi, & Amari, 2008; Amari, 1977). This collective coordinate stores an internal representation (Sontag, 2003; Erdem & Hasselmo, 2012) of the organism's state in its external environment, such as position in space (Pfeiffer & Foster, 2013; McNaughton et al., 2006) or head direction (Seelig & Jayaraman, 2015).

However, such internal representations are useful only if they can be driven and updated by external signals that provide crucial motor and sensory input (Hopfield, 2015; Pfeiffer & Foster, 2013; Erdem & Hasselmo, 2012; Hardcastle, Ganguli, & Giocomo, 2015; Ocko, Hardcastle, Giocomo, & Ganguli, 2018). Driving and updating the collective coordinate using external sensory signals opens up a variety of capabilities, such as path planning (Ponulak & Hopfield, 2013; Pfeiffer & Foster, 2013), correcting errors in the internal representation or in sensory signals (Erdem & Hasselmo, 2012; Ocko et al., 2018), and the ability to resolve ambiguities in the external sensory and motor input (Hardcastle et al., 2015; Evans, Bicanski, Bush, & Burgess, 2016; Fyhn, Hafting, Treves, Moser, & Moser, 2007).

In all of these examples, the functional use of attractors requires interaction between external signals and the internal recurrent network dynamics. However, with a few significant exceptions (Fung, Wong, Mao, & Wu, 2015; Mi, Fung, Wong, & Wu, 2014; Wu et al., 2008; Wu & Amari, 2005; Monasson & Rosay, 2014; Burak & Fiete, 2012), most theoretical work has either been in the limit of no external forces and strong internal recurrent dynamics, or in the limit of strong external forces where the internal recurrent dynamics can be ignored (Moser, Moser, & McNaughton, 2017; Tsodyks, 1999).

Here, we study continuous attractors in neural networks subject to external driving forces that are neither small relative to internal dynamics nor adiabatic. We show that the physics of the emergent collective coordinate sets limits on the maximum speed at which internal representations can be updated by external signals.

Our approach begins by deriving simple classical and statistical laws satisfied by the collective coordinate of many neurons with strong, structured interactions that are subject to time-varying external signals, Langevin noise, and quenched disorder. Exploiting these equations, we demonstrate two simple principles: (1) an equivalence principle that predicts how much the internal representation lags a rapidly moving external signal, and (2) under externally driven conditions, quenched disorder in network connectivity that can be modeled as a state-dependent effective temperature. Finally, we apply these results to place cell networks and derive a nonequilibrium driving-dependent memory capacity, complementing numerous earlier works on memory capacity in the absence of external driving.

We study $N$ interacting neurons following the formalism presented in Hopfield (2015),
$dindt=-inτ+∑k=1NJnkf(ik)+Inext(t)+ηint(t),$
(2.1)
where $f(ik)=(1+e-ik/i0)-1$ is the neural activation function that represents the firing rate of neuron $k$, and $in$ is an internal excitation level of neuron $n$ akin to the membrane potential. We consider synaptic connectivity matrices with two distinct components:
$Jij=Jij0+Jijd.$
(2.2)

As shown in Figure 1, $Jij0$ encodes the continuous attractor. We will focus on 1D networks with $p$-nearest neighbor excitatory interactions to keep bookkeeping to a minimum: $Jij0=J(1-ε)$ if neurons $|i-j|≤p$, and $Jij0=-Jε$ otherwise. The latter term, $-Jε$, with $0≤ε≤1$, represents long-range, nonspecific inhibitory connections as frequently assumed in models of place cells (Monasson & Rosay, 2014; Hopfield, 2010), head direction cells (Chaudhuri & Fiete, 2016), and other continuous attractors (Seung et al., 2000; Burak & Fiete, 2012).

Figure 1:

The effective dynamics of neural networks implicated in head direction and spatial memory is described by a continuous attractor. Consider $N$ neurons connected in a 1D topology, with local excitatory connections between $p$ nearest neighbors (blue), global inhibitory connections (not shown), and random long-range disorder (orange). Any activity pattern quickly condenses into a droplet of contiguous firing neurons (red) of characteristic size; the droplet center of mass $x¯$ is a collective coordinate parameterizing a continuous attractor. The droplet can be driven by space and time-varying external currents $Inext(t)$ (green).

Figure 1:

The effective dynamics of neural networks implicated in head direction and spatial memory is described by a continuous attractor. Consider $N$ neurons connected in a 1D topology, with local excitatory connections between $p$ nearest neighbors (blue), global inhibitory connections (not shown), and random long-range disorder (orange). Any activity pattern quickly condenses into a droplet of contiguous firing neurons (red) of characteristic size; the droplet center of mass $x¯$ is a collective coordinate parameterizing a continuous attractor. The droplet can be driven by space and time-varying external currents $Inext(t)$ (green).

The disorder matrix $Jijd$ represents random long-range connections, a form of quenched disorder (Seung, 1998; Kilpatrick, Ermentrout, & Doiron, 2013). Finally, $Inext(t)$ represents external driving currents from, for example, sensory and motor input possibly routed through other regions of the brain. The Langevin noise $ηint(t)$ represents private noise internal to each neuron (Lim & Goldman, 2012; Burak & Fiete, 2012).

A neural network like equation 2.1 qualitatively resembles a similarly connected network of Ising spins at fixed magnetization (Monasson & Rosay, 2014). At low noise, the activity in such a system will condense (Monasson & Rosay, 2014; Hopfield, 2010) to a localized droplet, since interfaces between firing and nonfiring neurons are penalized by $J(1-ε)$. The center of mass of such a droplet, $x¯≡∑nnf(in)∑nf(in)$ is an emergent collective coordinate that approximately describes the stable low-dimensional neural activity patterns of these $N$ neurons. Fluctuations about this coordinate have been extensively studied (Wu et al., 2008; Burak & Fiete, 2012; Hopfield, 2015; Monasson & Rosay, 2014).

We focus on how space and time-varying external signals, modeled here as external currents $Inext(t)$, can drive and reposition the droplet along the attractor. We will be primarily interested in a cup-shaped current profile that moves at a constant velocity $v$, $Inext(t)=Icup(n-vt)$, where $Icup(n)=d(w-|n|),n∈[-w,w]$, $Icup(n)=0$ otherwise. Such a localized time-dependent drive could represent landmark-related sensory signals (Hardcastle et al., 2015).

The effective dynamics of the collective coordinate $x¯$ in the presence of currents $Inext(t)$ can be obtained by computing the effective force on the droplet of finite size. We find that (see appendix A)
$γx¯˙=-∂x¯Vext(x¯,t),$
(3.1)
where $Vext(x¯,t)$ is a piecewise quadratic potential $Vcup(x¯-vt)$ for currents $Inext(t)=Icup(n-vt)$, and $γ$ is the effective drag coefficient of the droplet. (Here, we neglect rapid transients of timescale $τ$ (Wu et al., 2008).)

The strength of the external signal is set by the depth $d$ of the cup $Icup(n)$. Previous studies have explored the $d=0$ case—undriven diffusive dynamics of the droplet (Burak & Fiete, 2012; Monasson & Rosay, 2013, 2014, 2015) or the large $d$ limit (Hopfield, 2015) when the internal dynamics can be ignored. Here we focus on an intermediate regime, $d, where internal representations are updated continuously by the external currents, without any jumps (Ponulak & Hopfield, 2013; Pfeiffer & Foster, 2013; Erdem & Hasselmo, 2012).

In fact, as shown in the section C.2 we find a threshold signal strength $dmax$ beyond which the external signal destabilizes the droplet, instantly “teleporting” the droplet from any distant location to the cup without continuity along the attractor, erasing any prior positional information held in the internal representation.

We focus here on $d, a regime with continuity of internal representations. Such continuity is critical for many applications, such as path planning (Ponulak & Hopfield, 2013; Pfeiffer & Foster, 2013; Erdem & Hasselmo, 2012) and resolving local ambiguities in position within the global context (Hardcastle et al., 2015; Evans et al., 2016; Fyhn et al., 2007). In this regime, the external signal updates the internal representation with finite gain (Fyhn et al., 2007) and can thus fruitfully combine information in both the internal representation and the external signal. Other applications that simply require short-term memory storage of a strongly fluctuating variable may not require this continuity restriction.

### 3.1  Equivalence Principle

We first consider driving the droplet in a network at constant velocity $v$ using an external current $Inext=Icup(n-vt)$. We allow for Langevin noise but no disorder in the couplings $Jd=0$ in this section. For very slow driving ($v→0$), the droplet will settle into and track the bottom of the cup. When driven at a finite velocity $v$, the droplet cannot stay at the bottom since there is no net force exerted by the currents $Inext$ at that point.

Instead, the droplet must lag the bottom of the moving external drive by an amount $Δxv=x¯-vt$ such that the slope of the potential $Vcup$ provides an effective force $Fvmotion≡γv$ needed to keep the droplet in motion at velocity $v$, that is,
$-∂x¯Vcup(〈Δxv〉)=Fvmotion≡γv.$
(3.2)
This equation, which we call an equivalence principle in analogy with inertial particles in an accelerated frame, is verified by simulations in Figure 2b. Similar results on a lag between driving forces and the response were obtained in earlier works (Fung et al., 2015; Mi et al., 2014).
Figure 2:

(a) The mean position and fluctuations of the droplet driven by currents $Inext=Icup(n-vt)$ are described by an “equivalence” principle; in a frame co-moving with $Incup(t)$ with velocity $v$, we simply add an effective force $Fvmotion=γv$ where $γ$ is a drag coefficient. (b) This prescription correctly predicts that the droplet lags the external driving force by an amount linearly proportional to velocity $v$, as seen in simulations. (c) Fluctuations of the driven droplet's position, due to internal noise in neurons, are also captured by the equivalence principle. If $p(Δxv)$ is the probability of finding the droplet at a lag $Δxv$, we find that $kBTlogp(Δxv)-FvmotionΔxv$ is independent of velocity and can be collapsed onto each other (with fitting parameter $T$). (Inset: $logp(Δxv)$ before subtracting $FvmotionΔxv$.)

Figure 2:

(a) The mean position and fluctuations of the droplet driven by currents $Inext=Icup(n-vt)$ are described by an “equivalence” principle; in a frame co-moving with $Incup(t)$ with velocity $v$, we simply add an effective force $Fvmotion=γv$ where $γ$ is a drag coefficient. (b) This prescription correctly predicts that the droplet lags the external driving force by an amount linearly proportional to velocity $v$, as seen in simulations. (c) Fluctuations of the driven droplet's position, due to internal noise in neurons, are also captured by the equivalence principle. If $p(Δxv)$ is the probability of finding the droplet at a lag $Δxv$, we find that $kBTlogp(Δxv)-FvmotionΔxv$ is independent of velocity and can be collapsed onto each other (with fitting parameter $T$). (Inset: $logp(Δxv)$ before subtracting $FvmotionΔxv$.)

In fact, we find that the the above equivalence principle predicts the entire distribution $p(Δxv)$ of fluctuations of the lag $Δxv$ due to Langevin noise (see Figure 2c). By binning the lag $Δxv(t)$ for trajectories of the droplet obtained from repeated numerical simulations, we determined $p(Δxv)$, the occupancy of the droplet in the moving frame of the drive. As detailed in appendix C, data for different velocities collapse using an effective temperature scale $T$, verifying that
$kBTlogp(Δxv)=-(Vcup(Δxv)-FvmotionΔxv),$
(3.3)

Our results here are consistent with the fluctuation-dissipation result obtained in Monasson and Rosay (2014) for driven droplets. In summary, in the co-moving frame of the driving signal, the droplet's position $Δxv$ fluctuates as if it were in thermal equilibrium in the modified potential $Veff=Vcup-FvmotionΔxv$.

The simple equivalence principle implies a striking bound on the update speed of internal representations. A driving signal cannot drive the droplet at velocities greater than some $vcrit$ if the predicted lag for $v>vcrit$ is larger than the cup. In the appendix, we find $vcrit=2d(w+R)/3γ$, where $2R$ is the droplet size.

Larger driving strength $d$ increases $vcrit$, but as was previously discussed, we require $d in order to retain continuity and stability of the internal representation. Hence, we find an absolute upper bound on the fastest external signal that can be tracked by the internal representation,
$v*=κpJγ-1,$
(4.1)
where $p$ is the range of interactions, $J$ is the synaptic strength, $γ-1$ is the mobility or inverse drag coefficient of the droplet, and $κ$ is a dimensionless $O(1)$ number.

We now consider the effect of long-range quenched disorder $Jijd$ in the synaptic matrix (Seung, 1998; Kilpatrick et al., 2013), which breaks the exact degeneracy of the continuous attractor, creating an effectively rugged landscape, $Vd(x¯)$, as shown schematically in Figure 3 and computed in sections E.1 and E.2. When driven by a time-varying external signal, $Iiext(t)$, the droplet now experiences a net potential $Vext(x¯,t)+Vd(x¯)$. The first term causes motion with velocity $v$ and a lag predicted by the equivalence principle, and for sufficiently large velocities $v$, the effect of the second term can be modeled as effective Langevin white noise. To see this, note that $Vd(x¯)$ is uncorrelated on length scales larger than the droplet size; hence, for large enough droplet velocity $v$, the forces $Fd(t)≡-∂x¯Vd|x¯=x¯(t)$ due to disorder are effectively random and uncorrelated in time. More precisely, let $σ2=Var(Vd(x¯))$. In section E.3, we compute $Fd(t)$ and show that $Fd(t)$ has an autocorrelation time, $τcor=2R/v$, due to the finite size of the droplet.

Figure 3:

Disorder in neural connectivity is well approximated by an effective temperature $Td$ for a moving droplet. (a) Long-range disorder breaks the degeneracy of the continuous attractor, creating a rough landscape. A droplet moving at velocity $v$ in this rough landscape experiences random forces. (b) The fluctuations of a moving droplet's position, relative to the cup's bottom, can be described by an effective temperature $Td$. We define a potential $V(Δxv)=-kBTdlogp(Δxv)$ where $p(Δxv)$ is the probability of the droplet's position fluctuating to a distance $Δxv$ from the peak external current. We find that $V(Δxv)$ corresponding to different amounts of disorder $σ˜2$ (where $σ˜2$ is the average number of long-range disordered connections per neuron in units of $2p$), can be collapsed by the one fitting parameter $Td$. Inset: $Td$ is linearly proportional to the strength of disorder $σ˜$.

Figure 3:

Disorder in neural connectivity is well approximated by an effective temperature $Td$ for a moving droplet. (a) Long-range disorder breaks the degeneracy of the continuous attractor, creating a rough landscape. A droplet moving at velocity $v$ in this rough landscape experiences random forces. (b) The fluctuations of a moving droplet's position, relative to the cup's bottom, can be described by an effective temperature $Td$. We define a potential $V(Δxv)=-kBTdlogp(Δxv)$ where $p(Δxv)$ is the probability of the droplet's position fluctuating to a distance $Δxv$ from the peak external current. We find that $V(Δxv)$ corresponding to different amounts of disorder $σ˜2$ (where $σ˜2$ is the average number of long-range disordered connections per neuron in units of $2p$), can be collapsed by the one fitting parameter $Td$. Inset: $Td$ is linearly proportional to the strength of disorder $σ˜$.

Thus, on longer timescales, $Fd(t)$ is uncorrelated and can be viewed as Langevin noise for the droplet center of mass $x¯$, associated with a disordered-induced temperature $Td$. Through repeated simulations with different amounts of disorder $σ2$, we inferred the distribution $p(Δxv)$ of the droplet position in the presence of such disorder-induced fluctuations (see Figure 3). The data collapse in Figure 3b confirms that the effect of disorder (of size $σ2$) on a rapidly moving droplet can indeed be modeled by an effective disorder-induced temperature $Td∼στcor$. (For simplicity, we assume that internal noise $ηint$ in equation 2.1 is absent here. Note that in general, $ηint$ will also contribute to $Td$. Here we focus on the contribution of disorder to an effective temperature $Td$ since internal noise $ηint$ has been considered in prior work (Fung et al., 2015).)

Thus, the disorder $Jijd$ effectively creates thermal fluctuations about the lag predicted by the equivalence principle; such fluctuations may carry the droplet out of the driving cup $Icup(n-vt)$ and prevent successful update of the internal representation. We found that this effect can be quantified by a simple Arrhenius-like law,
$r∼exp(-ΔE(v,d)/kBTd),$
(5.1)
where $ΔE(v,d)$ is the energy gap between where the droplet sits in the drive and the escape point, predicted by the equivalence principle, and $Td$ is the disorder-induced temperature. Thus, given a network of $N$ neurons, the probability of an external drive moving the droplet successfully across the network is proportional to $exp(-rN)$. (Note that $r$ depends on $N$ in a way such that $exp(-rN)$ becomes a step function as $N→∞$: always successful below a critical amount of disorder (capacity) and always failing beyond this capacity.)

The capacity of a neural network to encode multiple memories has been studied in numerous contexts since Hopfield's original work (Hopfield, 1982). While specifics differ (Amit, Gutfreund, & Sompolinsky, 1985a; Battaglia & Treves, 1998; Monasson & Rosay, 2014; Hopfield, 2010), the capacity is generally set by the failure to retrieve a specific memory because of the effective disorder in neural connectivity due to other stored memories.

However, these works on capacity do not account for nonadiabatic external driving. Here, we use our results to determine the capacity of a place cell network (O'Keefe & Dostrovsky, 1971; Battaglia & Treves, 1998; Monasson & Rosay, 2014) to both encode and manipulate memories of multiple spatial environments at a finite velocity. Place cell networks (Tsodyks, 1999; Monasson & Rosay, 2013, 2014, 2015) encode memories of multiple spatial environments as multiple continuous attractors in one network. Such networks have been used to describe recent experiments on place cells and grid cells in the hippocampus (Yoon et al., 2013; Hardcastle et al., 2015; Moser, Moser, & Roudi, 2014).

In experiments that expose a rodent to different spatial environments $μ=1,…M$ (Alme et al., 2014; Moser, Moser, & McNaughton, 2017; Moser, Moser, & Roudi, 2014; Kubie & Muller, 1991), the same place cells $i=1,…,N$ are seen having “place fields” in different spatial arrangements $πμ(i)$ as seen in Figure 4a, where $πμ$ is a permutation specific to environment $μ$. Consequently, Hebbian plasticity suggests that each environment $μ$ would induce a set of synaptic connections $Jijμ$ that corresponds to the place field arrangement in that environment: $Jijμ=J(1-ε)$ if $|πμ(i)-πμ(j)|. That is, each environment corresponds to a 1D network when the neurons are laid out in a specific permutation $πμ$. The actual network has the sum of all these connections $Jij=∑μ=1MJijμ$ over the $M$ environments the rodent is exposed to.

Figure 4:

Nonequilibrium capacity of place cell networks limits retrieval of spatial memories at finite velocity. (a) Place cell networks model the storage of multiple spatial memories in parts of the hippocampus by coding multiple continuous attractors in the same set of neurons. Neural connections encoding spatial memory 2, 3, … act like long-range disorder for spatial memory 1. Such disorder, through an increased effective temperature, reduces the probability of tracking a finite velocity driving signal. (b) The probability of successful retrieval, $Pretrieval$, decreases with the number of simultaneous memories $M$ and velocity $v$ (with $N=4000,p=10,ε=0.35,τ=1,J=100,d=10,w=30$ held fixed). (c) $Pretrieval$ simulation data collapse when plotted against $M/(N/(logN)2)$ (parameters same as panel b with $v=0.8$ held fixed and $N$ varies). (d) The nonequilibrium capacity $Mc$ as a function of retrieval velocity $v$.

Figure 4:

Nonequilibrium capacity of place cell networks limits retrieval of spatial memories at finite velocity. (a) Place cell networks model the storage of multiple spatial memories in parts of the hippocampus by coding multiple continuous attractors in the same set of neurons. Neural connections encoding spatial memory 2, 3, … act like long-range disorder for spatial memory 1. Such disorder, through an increased effective temperature, reduces the probability of tracking a finite velocity driving signal. (b) The probability of successful retrieval, $Pretrieval$, decreases with the number of simultaneous memories $M$ and velocity $v$ (with $N=4000,p=10,ε=0.35,τ=1,J=100,d=10,w=30$ held fixed). (c) $Pretrieval$ simulation data collapse when plotted against $M/(N/(logN)2)$ (parameters same as panel b with $v=0.8$ held fixed and $N$ varies). (d) The nonequilibrium capacity $Mc$ as a function of retrieval velocity $v$.

While $Jij$ above is obtained by summing over $M$ structured environments, from the perspective of, say, $Jij1$, the remaining $Jijμ$ look like long-range disordered connections. We will assume that the permutations $πμ(i)$ corresponding to different environments are random and uncorrelated, a common modeling choice with experimental support (Hopfield, 2010; Monasson & Rosay, 2014, 2015; Alme et al., 2014; Moser et al., 2017). Without loss of generality, we assume that $π0(i)=i$ (blue environment in Figure 4.) Thus, $Jij=Jij0+Jijd,Jijd=∑μ=1M-1Jijμ$. The disordered matrix $Jijd$ then has an effective variance $σ2∼(M-1)/N$. Hence, we can apply our previous results to this system. Now consider driving the droplet with velocity $v$ in environment 1 using external currents. The probability of successfully updating the internal representation over a distance $L$ is given by $Pretrieval=e-rL/v$, where $r$ is given by equation 5.1.

In the thermodynamic limit $N→∞$, with $w,p,L/N$ held fixed, $Pretrieval$ becomes a Heaviside step function $Θ(Mc-M)$ at some critical value $Mc$ given by
$Mc∼[vΔE(v,d)]2N(logN)2$
(6.1)
for the largest number of memories that can be stored and retrieved at velocity $v$. $ΔE(v,d)=(4dw-3γv-2dR)(-vγ+2dR)/4d$. Figure 4 shows that our numerics agree well with this formula, showing a novel dependence of the capacity of a neural network on the speed of retrieval and the strength of the external drive. Note that the fact that equation 6.1 scales sublinearly in $N$ reflects our choice of “perfect” retrieval in the definition of successful events. As in earlier works, Hopfield (1982), Hertz, Krogh, Palmer, and Horner (1991), and Amit et al. (1985a, 1985b), the precise definition of capacity can change capacity by $log$ factors.

We have considered continuous attractors in neural networks driven by localized time-dependent currents $Icup(n-vt)$. In recent experiments, such currents can represent landmark-related sensory signals (Hardcastle et al., 2015) when a rodent is traversing a spatial environment at velocity $v$ or signals that update the internal representation of head direction (Seelig & Jayaraman, 2015). Several recent experiments have controlled the effective speed of visual stimuli in virtual reality environments (Meshulam, Gauthier, Brody, Tank, & Bialek, 2017; Aronov, Nevers, & Tank, 2017; Kim et al., 2017; Turner-Evans et al., 2017). Other experiments have probed cross-talk between memories of multiple spatial environments (Alme et al., 2014). Our results predict an error rate that rises with speed and with the number of environments.

While our analysis used specific functional forms for, among others, the current profile $Icup(n-vt)$, our bound simply reflects the finite response time in moving emergent objects, much like moving a magnetic domain in a ferromagnet using space and time-varying fields. Thus, we expect our bound to hold qualitatively for other related forms (Hopfield, 2015).

In addition to positional information considered here, continuous attractors are known to also receive velocity information (Major, Baker, Aksay, Seung, & Tank, 2004; McNaughton et al., 2006; Seelig & Jayaraman, 2015; Ocko et al., 2018). We do not consider such input in the main text but extend our analysis to velocity integration in appendix D.

In summary, we found that the nonequilibrium statistical mechanics of a strongly interacting neural network can be captured by a simple equivalence principle and a disorder-induced temperature for the network's collective coordinate. Consequently, we were able to derive a velocity-dependent bound on the number of simultaneous memories that can be stored and retrieved from a network. We discussed how these results, based on general theoretical principles on driven neural networks, allow us to connect robustly to recent time-resolved experiments in neuroscience (Kim et al., 2017; Turner-Evans et al., 2017; Hardcastle et al., 2015; Hardcastle, Maheswaranathan, Ganguli, & Giocomo, 2017; Campbell et al., 2018) on the response of neural networks to dynamic perturbations.

As in the main text, we model $N$ interacting neurons as
$dindt=-inτ+∑k=1NJnkf(ik)+Inext(t)+ηnint(t),wheref(i)=11+e-i/i0.$
(A.1)
The synaptic connection between two different neurons $i,j$ is $Jij=J(1-ε)$ if neurons $i$ and $j$ are separated by a distance of at most $p$ neurons and $Jij=-Jε$ otherwise; note that we set the self-interaction to zero. The internal noise is a white noise, $〈ηnint(t)ηnint(0)〉=Cintδ(t)$ with an amplitude $Cint$. $Inext(t)$ are external driving currents, discussed below.
Such a quasi-1D network with $p$-nearest neighbor interactions resembles a similarly connected network of Ising spins at fixed magnetization in its behavior; the strength of inhibitory connections $ε$ constrains the total number of neurons $2R$ firing at any given time to $2R∼pε-1$. It was shown (Hopfield, 2010; Monasson & Rosay, 2013, 2014) that below a critical temperature $T$, the $w$ firing neurons condense into a contiguous droplet of neural activity, minimizing the total interface between firing and nonfiring neurons. Such a droplet was shown to behave like an emergent quasi-particle that can diffuse or be driven around the continuous attractor. We define the center of mass of the droplet as
$x¯≡∑nnf(in).$
(A.2)

The description of neural activity in terms of such a collective coordinate $x¯$ greatly simplifies the problem, reducing the configuration space from the $2N$ states for the $N$ neurons to $N$-state, and consists of the center of mass of the droplet along the continuous attractor (Wu et al., 2008). The computational abilities of these place cell networks, such as spatial memory storage, path planning, and pattern recognition, are limited to parameter regimes in which such a collective coordinate approximation holds (e.g., noise levels less than a critical value $T) .

The droplet can be driven by external signals such as sensory or motor input or input from other parts of the brain. We model such external input by the currents $Inext$ in equation A.1—for example, sensory landmark-based input (Hardcastle et al., 2015). When an animal is physically in a region covered by place fields of neurons $i,i+1,…,i+z$, currents $Iiext$ through $Ii+zext$ can be expected to be high compared to all other currents $Ijext$. Other models of driving in the literature include adding an antisymmetric component $Aij$ to synaptic connectivities $Jij$ (Ponulak & Hopfield, 2013); we consider such a model in appendix D.

Let ${ikx¯}$ denote the current configuration such that the droplet is centered at location $x¯$. The Lyapunov function of the neural network is given by Hopfield (2015):
$L[x¯]≡L[f(ikx¯)]=1τ∑k∫0f(ikx¯)f-1(x)dx-12∑n,kJnkf(ikx¯)f(inx¯)-∑kf(ikx¯)Ikext(t).$
(A.3)
In a minor abuse of terminology, we will refer to terms in the Lyapunov function as energies, even though energy is not conserved in this system. For future reference, we denote the second term $VJ(x¯)=-1/2∑nkJnkf(ikx¯)f(inx¯)$, which captures the effect of network synaptic connectivities. Under the rigid bump approximation used in Hopfield (2015), that is, ignoring fluctuations of the droplet, we find
$VJ(x¯)=-12∑n,kf(inx¯)Jnkf(ikx¯)$
(A.4)
$≈-12∑|n-x¯|≤R,|k-x¯|≤Rf(inx¯)Jnkf(ikx¯).$
(A.5)

For a quasi 1D network with $p$-nearest neighbor interactions and no disorder, $VJ(x¯)$ is constant, giving a smooth, continuous attractor. However, as discussed later, in the presence of disorder, $VJ(x¯)$ has bumps (i.e., quenched disorder) and is no longer a smooth, continuous attractor.

To quantify the effect of the external driving, we write the third term in equation A.3:
$Vext(x¯,t)=-∑kIkext(t)f(ikx¯)$
(A.6)
$≈-∑|k-x¯|
(A.7)
Thus, the external driving current $Inext(t)$ acts on the droplet through the Lyapunov function $Vext(x¯,t)$. Hence, we define
$Fext(x¯,t)=-∂x¯Vext(x¯,t)$
(A.8)
to be the external force acting on the droplet center of mass.

### A.1  Fluctuation and Dissipation

We next numerically verify that the droplet obeys a fluctuation-dissipation-like relation by driving the droplet using external currents $Iext$ and comparing the response to diffusion of the droplet in the absence of external currents.

We use a finite ramp as the external driving, $Inext=n$ with $n, and $Inext=0$ otherwise (see Figure 5a). We choose $nmax$ to be such that it takes considerable time for the droplet to relax to its steady-state position at the end of the ramp. We notice that for different slopes of the $Inext$, the droplet has different velocities, and it is natural to define a mobility of the droplet, $μ$, by $v=μf$, where $f$ is the slope of $Inext$. Next, we notice that on a single continuous attractor, the droplet can diffuse because of internal noise in the neural network. Therefore, we can infer the diffusion coefficient $D$ of the droplet from $〈x2〉=2Dt$ for a collection of diffusive trajectories (see Figure 5b), where we have used $x$ to denote the center of mass $x¯$ for the droplet to avoid confusion.

Figure 5:

(a) Schematics of the droplet being driven by a linear potential (ramp), illustrating the idea of mobility. Green lines are inputs, red dots are active neurons, and the more transparent ones represent earlier time. (b) Schematics of the droplet diffusing under an input with no gradient, giving rise to diffusion. The inset is the plot of mean-squared distance versus time, clearly showing diffusive behavior. Note here that we have changed the droplet center of mass (c.o.m.) position $x¯$ as $x$ to avoid confusion with the mean position. (c) Comparison between mobility $μ=γ-1$ and diffusion coefficient $D$. Both $μ$ and $D$ depend on blob size and $τ$ in the same way, and thus $D$ is proportional to $μ$.

Figure 5:

(a) Schematics of the droplet being driven by a linear potential (ramp), illustrating the idea of mobility. Green lines are inputs, red dots are active neurons, and the more transparent ones represent earlier time. (b) Schematics of the droplet diffusing under an input with no gradient, giving rise to diffusion. The inset is the plot of mean-squared distance versus time, clearly showing diffusive behavior. Note here that we have changed the droplet center of mass (c.o.m.) position $x¯$ as $x$ to avoid confusion with the mean position. (c) Comparison between mobility $μ=γ-1$ and diffusion coefficient $D$. Both $μ$ and $D$ depend on blob size and $τ$ in the same way, and thus $D$ is proportional to $μ$.

In Figure 5c, we numerically verify that $μ$ and $D$ depend on parameters $τ$ and $R$ in the same way—that $D$ and $μ$ are both proportional to $1/τ$ and independent of $R$. This suggests that $D∝μ$; if we call the proportionality constant to be $kBT$, then we have a fluctuation-dissipation-like relation:
$D=μkBT.$
(A.9)

Note that equation A.9 has also been derived for the case of binary neurons with a hard constraint on the number of firing population (Monasson & Rosay, 2014).

We consider the model of sensory input used in the main text: $Icup(n)=d(w-|n|),n∈[-w,w]$, $Icup(n)=0$ otherwise. We focus on time-dependent currents $Inext(t)=Icup(n-vt)$. Such a drive was previously considered in Wu and Amari (2005), albeit without time dependence. Throughout this article, we refer to $w$ as the linear size of the drive, $d$ as the depth of the drive, and set the drive moving at a constant velocity $v$. From now on, we will go to the continuum limit and denote $Inext(t)=Iext(n,t)≡Iext(x,t)$.

As an example, for $v=0$ (in this case, $Δxv=x¯$), we can write down the potential $Vext$ for the external driving signal $Icup(x)=d(w-|x|)$ by evaluating it at a stationary current profile $f(ikx¯)=1if|k-x¯|≤R,=0otherwise$,
$Vext(x¯)=V1(x¯),|x¯|≤RV2(x¯),|x¯|>R,$
(B.1)
where
$V1(x¯)=-d(R-x¯)(w-R-x¯2)+(R+x¯)(w-w+x¯2),V2(x¯)=-d2(R+w-x¯)2.$
(B.2)

We plot $Vext$ given by equation B.1 versus the c.o.m. position of droplet in Figure 6a.

Figure 6:

(a) $Vext$ for external driving signal $Icup(x,t)$ with $v=0$, plotted from equation B.1 with $d=20$, $R=15$, $w=30$. (b) Effective potential $Veff$ experienced by the droplet for a moving cup-shaped external driving signal, plotted from equation C.1 with $d=10$, $R=15$, $w=30$, $γv=140$. (c) Schematic illustrating the idea of the equivalence principle (see equation 3.2). The difference between the effective potential, $Veff≡-kBTlogp(Δxv)$, experienced by a moving droplet, and that of a stationary droplet, $Vcup$, is a linear potential, $Vlin=-FvmotionΔxv$. The slope $θ$ of the linear potential $Vlin=-FvmotionΔxv$ is proportional to velocity as $Fvmotion=γv$.

Figure 6:

(a) $Vext$ for external driving signal $Icup(x,t)$ with $v=0$, plotted from equation B.1 with $d=20$, $R=15$, $w=30$. (b) Effective potential $Veff$ experienced by the droplet for a moving cup-shaped external driving signal, plotted from equation C.1 with $d=10$, $R=15$, $w=30$, $γv=140$. (c) Schematic illustrating the idea of the equivalence principle (see equation 3.2). The difference between the effective potential, $Veff≡-kBTlogp(Δxv)$, experienced by a moving droplet, and that of a stationary droplet, $Vcup$, is a linear potential, $Vlin=-FvmotionΔxv$. The slope $θ$ of the linear potential $Vlin=-FvmotionΔxv$ is proportional to velocity as $Fvmotion=γv$.

### B.1  A Thermal Equivalence Principle

The equivalence principle we introduced in the main text allows us to compute the steady-state position and the effective new potential seen in the co-moving frame. Crucially, the fluctuations of the collective coordinate are described by the potential obtained through the equivalence principle. The principle correctly predicts both the mean (see equation 3.2) and the fluctuation (see equation 3.3) of the lag $Δxv$. Therefore, it is actually a statement about the equivalence of effective dynamics in the rest frame and in the co-moving frame. Specializing to the drive $Icup(x,t)$, the equivalence principle predicts that the effective potential felt by the droplet (moving at constant velocity $v$) in the co-moving frame equals the effective potential in the stationary frame shifted by a linear potential, $Vlin=-FvmotΔxv$, that accounts for the fictitious forces due to the change of coordinates (see Figure 6c).

Since we used equation B.1 for the cup shape and the lag $Δxv$ depends linearly on $v$, we expect that the slope of the linear potential $Vlin$ also depends linearly on $v$. Here the sign convention is chosen such that $Vlin<0$ corresponds to the droplet moving to the right.

In the following, we work in the co-moving frame with velocity $v$ at which the driving signal is moving. We denote the steady-state c.o.m. position in this frame to be $Δxv*$ and a generic position to be $Δxv$.

When $v>0$, the droplet will sit at a steady-state position $Δxv*<0$. The equivalence principle says we should subtract a velocity-dependent linear potential $FvmotΔxv=γvΔxv$ from $Vext$ to account for the motion:
$Veff(Δxv)=Vcup(Δxv)-γvΔxv.$
(C.1)
We plot $Veff$ vs $Δxv$ in Figure 6b. Notice that there are two extremal points of the potential, corresponding to the steady-state position, $Δxv*$, and the escape position, $Δxvesc$:
$Δxv*=γv/2d,Δxvesc=(dw-γv+dR)/d.$
(C.2)

We are now in position to derive $vcrit$ presented in the main text. We observe that as the driving velocity $v$ increases, $Δxv*$ and $Δxvesc$ will get closer to each other, and there will be a critical velocity such that the two coincide.

By simply equating the expression for $xesc$ and $x*$ and solving for $v$, we found that
$vcrit=2d(w+R)3γ.$
(C.3)

### C.1  Steady-State Droplet Size

Recall that the Lyapunov function of the neural network is given by equatin A.3:
$L[x¯]=1τ∑k∫0f(ikx¯)f-1(x)dx+VJ(x¯)+Vext(x¯,t).$
(C.4)

Compared to the equation of motion (e.o.m.), equaiton A.1, we see that the first term corresponds to the decay of neurons in the absence of interaction from neighbors (decay from the on state to the off state), and the second term corresponds to the interaction $Jnk$ term in the e.o.m, and the third term corresponds to the $Inext$ in the e.o.m. Since we are interested in the steady-state droplet size, and thus only in the neurons that are on, the effect of the first term can be neglected (also note that $1/τ≪Jij$; when using the Lyapunov function to compute steady-state properties, the first term can be ignored).

To obtain general results, we also account for long-range disordered connections $Jijd$ here. We assume $Jijd$ consists of random connections among all the neurons. We can approximate these random connections as random permutations of $Jij0$, and the full $Jij$ is the sum over $M-1$ such permutations plus $Jij0$.

For the cup-shaped driving and its corresponding effective potential, equation C.1, we are interested in the steady-state droplet size under this driving, so we first evaluate $Veff$ at the steady-state position $Δxv*$ in equation C.2. To make the $R$-dependence explicit in the Lyapunov function, we evaluate $L(x¯)$ under the rigid bump approximation used in Hopfield (2015), assuming $f(ikx¯)=1$ for $|k-x¯|≤R$, and $=0$ otherwise.

We find that for $M-1$ sets of disorder interactions, the Lyapunov function is
$L[f(ikx¯)]=J[(εR2-(ε+2p)R+p(p+1)2-pm(2R-p)2]+(γv)24d+Rd(R-2w),$
(C.5)
where we have defined the reduced disorder parameter $m=(M-1)/N$ and have used the equivalence principle in equation 3.2 to add an effective linear potential to take into account the motion of the droplet.
Next, we note that the steady-state droplet size corresponds to a local extremum of the Lyapunov function. Extremizing equation C.5 with respect to droplet radius $R$, we obtain the steady-state droplet radius as a function of the external driving parameters $d,w$, and the reduced disorder parameter $m$,
$R(d,w,m)=2p-4p2m+2wd/J+ε2d/J-8pm+4ε,$
(C.6)
where we observe that in the formula, the only dimensionful parameters $d$ and $J$ appear together to ensure the overall result is dimensionless. Our result for $R$ reduces to $R0=p2ε+14$ by setting $M=1$ and $d=w=0$.

### C.2  Upper Limit on External Signal Strength

Here we present the calculation for maximal driving strength $Iext$ beyond which the activity droplet will “teleport”—that is, disappear at the original location and recondense at the location of the drive, even if these two locations are widely separated. We now refer to this maximal signal strength as the teleportation limit. We can determine this limit by finding out the critical point where the energy barrier of breaking up the droplet at the original location is zero.

For simplicity, we assume that initially, the cup-shaped driving signal is some distance $x0$ from the droplet and not moving (the moving case can be solved in exactly the same way by using the equivalence principle and going to the co-moving frame of the droplet). We consider three scenarios during the teleportation process. (1) In the initial configuration, the droplet has not yet teleported and stays at the original location with radius $R(0,0,m)$. (2) In the intermediate configuration, the activity is no longer contiguous, giving a droplet with radius $δ(d,w,m)$ at the center of the cup, and another droplet with radius $R(d,w,m)-δ(d,w,m)$ at the original location (when teleportation happens, the total firing neurons changes from $R(0,0,m)$ to $R(d,w,m)$). (3) In the final configuration, the droplet has successfully teleported to the center of the cup, with radius $R(d,w,m)$. The three scenarios are depicted schematically in Figure 7.

Figure 7:

Schematics of three scenarios during a teleportation process. In the initial configuration, the droplet is outside the cup. An energetically unfavorable intermediate configuration is penalized by $ΔE$: the droplet breaks apart into two droplets—one outside the cup and one inside it. In the final configuration, with the lowest energy, the droplet inside the cup grows to a full droplet while the droplet outside shrinks to zero size. Above each droplet is its corresponding radius $R$.

Figure 7:

Schematics of three scenarios during a teleportation process. In the initial configuration, the droplet is outside the cup. An energetically unfavorable intermediate configuration is penalized by $ΔE$: the droplet breaks apart into two droplets—one outside the cup and one inside it. In the final configuration, with the lowest energy, the droplet inside the cup grows to a full droplet while the droplet outside shrinks to zero size. Above each droplet is its corresponding radius $R$.

The global minimum of the Lyapunov function corresponds to scenario 3. However, there is an energy barrier between configuration 1 and configuration 3, corresponding to the $Veff$ difference between configuration 1 and 2. We would like to find the critical split size $δc(d,w,m)$ that maximizes the difference in $Veff$, which corresponds to the largest energy barrier the network has to overcome in order to teleport from configuration 1 to 3. For the purpose of derivation, in the following we rename $L[f(ikm)]$ in equation C.5 as $E0(d,w,m)∥R(d,w,m)$ to emphasize its dependence on the external driving parameters and disordered interactions. The subscript 0 stands for the default one-droplet configuration, and it is understood that $E0(d,w,m)$ is evaluated at the network configuration of a single droplet at location $m$ with radius $R(d,w,m)$.

The energy for configuration 1 is simply $E0(0,0,m)$, and the energy for configuration 3 is $E0(d,w,m)$. However, the energy for configuration 2 is not just the sum of $E0$ from the two droplets. Due to global inhibitions presented in the network, when there are two droplets, there will be an extra interaction term when we evaluate the Lyapunov function with respect to this configuration. The interaction energy between two droplets in Figure 7 is
$Eint(m)|R,δ=4JRδ(ε-2pm).$
(C.7)
Therefore, the energy barrier for split size $δ$ is
$ΔE(d,w,m)∥δ=E0(0,0,m)∥R(d,w,m)-δ+E0(d,w,m)∥δ+Eint(m)∥R(d,w,m),δ-E0(0,0,m)∥R(0,0,m).$
(C.8)
Therefore, maximizing $ΔE$ with respect to $δ$, we find
$δc=dwd-8Jpm+4Jε.$
(C.9)

Now we have obtained the maximum energy barrier during a teleportation process, $ΔE∥δc$. A spontaneous teleportation will occur if $ΔE∥δc≤0$, and this in turn gives an upper bound on the external driving signal strength $d≤dmax$ one can have without any teleportation spontaneous occurring.

We plot the numerical solution of $dmax$ obtained from the solving $ΔE(dc,w,m)∥δc=0$, compared with results obtained from the simulation in Figure 8, and find perfect agreement.

Figure 8:

Teleportation depth $dmax$ plotted against disorder parameter $m$. The dots are data obtained from simulations for different $N$ but with $p=10$, $ε=0.35$, $τ=1$, $J=100$, and $w=30$ held fixed. The dotted line is the theoretical curve plotted from solving $ΔE(dc,w,m)∥δc=0$ for $dc$ numerically.

Figure 8:

Teleportation depth $dmax$ plotted against disorder parameter $m$. The dots are data obtained from simulations for different $N$ but with $p=10$, $ε=0.35$, $τ=1$, $J=100$, and $w=30$ held fixed. The dotted line is the theoretical curve plotted from solving $ΔE(dc,w,m)∥δc=0$ for $dc$ numerically.

We also obtain an approximate solution by observing that the only relevant scale for the critical split size $δc$ is the radius of the droplet, $R$. We set $δc=cR$ for some constant $0≤c≤1$. In general, $c$ can depend on dimensionless parameters like $p$ and $ε$. Empirically we found the constant to be about 0.29 in our simulation.

The droplet radius $R$ is a function of $d,w,m$ as we see in equation C.6, but to first-order approximation, we can set $R$ = $R*$ for some steady-state radius $R*$. Then we can solve
$dmax(M)=4J(ε-2pm)w/cR*-1.$
(C.10)

Note that the denominator is positive because $w>R$ and $0≤c≤1$. The simulation result also confirms that the critical split size $δc$ stays approximately constant. We have checked that the dependence on parameters $J,w,m$ in equation C.10 agrees with the numerical solution obtained from solving $Ebar(dc,w,m)∥δc=0$, up to the undetermined constant $c$.

### C.3  Speed Limit on External Driving

Recall that given a certain signal strength $d$, there is an upper bound on how fast the driving can be (see equation C.3). Then in particular, for $dmax$, we obtain an upper bound on how fast an external signal can drive the network:
$vmax=8J(w+R*)(ε-2pm)3γ(w/cR*-1).$
(C.11)
For $w≫R*$, we can approximate
$vmax≈16JcR*(ε/2-pm)3γ.$
(C.12)
In the absence of disorder, $m=0$, the maximum velocity is bounded by
$vmax≤8c3εJR*γ≤8c3εJRmaxγ.$
(C.13)
Recall that in equation C.10, we have
$R(d,w≫R,0)≤R(dmax,w≫R,0)=p2ε+14+2cR*+ORw⪅p2ε+2cRmax,$
(C.14)
where in the second line we have used equation C.6 for $d=dmax$, $m=0$, and $w≫R$. Upon rearranging, we have
$Rmax⪅11-2cp2ε.$
(C.15)
Plugging in equation C.13, we have
$vmax≤8c3εJRmaxγ⪅83(c-1-2)Jpγ.$
(C.16)
Therefore, we have obtained a fundamental limit on how fast the droplet can move under the influence of external signal, namely,
$vfund=κJpγ-1,$
(C.17)
where $κ=8/3(c-1-2)$ is a dimensionless $O(1)$ number.

Place cell networks (Ocko et al., 2018) and head direction networks (Kim et al., 2017) are known to receive information about both velocity and landmark information. Velocity input can be modeled by adding an antisymmetric part $Aij$ to the connectivity matrix $Jij$, which effectively tilts the continuous attractor.

Consider now
$Jij=Jij0+Jijd+Aij0,$
(D.1)
where $Aij0=A$, if $0; $-A$, if $0; and 0 otherwise.

The antisymmetric part $Aij0$ will provide a velocity $v$ that is proportional to the size $A$ of $Aij0$ for the droplet (see Figure 9). In the presence of disorder, we can simply go to the co-moving frame of velocity $v$, and the droplet experiences an extra disorder-induced noise $ηA$ in addition to the disorder-induced temperature $Td$.

Figure 9:

Velocity of droplet $v$ plotted against the size $A$ of the antisymmetric matrix. We hold all other parameters fixed with the same value as in Figure 8.

Figure 9:

Velocity of droplet $v$ plotted against the size $A$ of the antisymmetric matrix. We hold all other parameters fixed with the same value as in Figure 8.

We found that $〈ηA(t)ηA(0)〉∝σ˜δ(t)$ (see Figure 10), where $σ˜2$ is the average number of disordered connection per neuron in units of $2p$.

Figure 10:

Left: At fixed $A=5$, a collection of 500 diffusive trajectories in the co-moving frame at velocity $v$, where $v$ is taken to be the average velocity of all the trajectories. We can infer the diffusion coefficient $D$ from the variance of these trajectories as Var$(x)=2Dt$. Right: log$D$ plotted against log$σ˜2$. The straight line has slope $1/2$, corresponding to $D∝σ˜$.

Figure 10:

Left: At fixed $A=5$, a collection of 500 diffusive trajectories in the co-moving frame at velocity $v$, where $v$ is taken to be the average velocity of all the trajectories. We can infer the diffusion coefficient $D$ from the variance of these trajectories as Var$(x)=2Dt$. Right: log$D$ plotted against log$σ˜2$. The straight line has slope $1/2$, corresponding to $D∝σ˜$.

Therefore, all our results in the main text apply to the case when both the external drive $Iext(x,t)$ and the antisymmetric part $Aij0$ exist. Specifically, we can just replace the velocity $v$ used in the main text as the sum of the two velocities corresponding to $Iext(x,t)$ and $Aij0$.

### E.1  Disordered Connections and Disordered Forces

From now on, we include disorder connections $Jijd$ in addition to ordered connections $Jij0$ that correspond to the nearest $p$-neighbor interactions. We assume $Jijd$ consists of random connections among all the neurons. These random connections can be approximated as random permutations of $Jij0$, such that the full $Jij$ is the sum over $M-1$ such permutations plus $Jij0$.

We clip the $Jij$ matrix according to the following rule for each entry when summing over $Jij0$ and $Jijd$:
$J(1-ε)+J(1-ε)→J(1-ε),J(1-ε)+J(-ε)→J(1-ε),J(-ε)+J(-ε)→J(-ε).$
(E.1)
Therefore, adding more disorder connections to $Jij$ amounts to changing the inhibitory $-Jε$ entries to the exitory $J(1-ε)$.
We would like to characterize the effect of disorder on the system. Under the decomposition $Jij=Jij0+Jijd$, we can define a (quenched) disorder potential that captures all the disorder effects on the network:
$Vd(x¯)≡Vd[f(ikx¯)]=-12∑nkJnkdf(ikx¯)f(inx¯).$
(E.2)
Its corresponding disorder-induced force is then given by
$Fd(x¯)=-∂x¯Vd(x¯).$
(E.3)

### E.2  Variance of Disorder Forces

We compute the distribution of $Vd(x¯)$ using a combinatorial argument as follows.

Under the rigid droplet approximation, calculating $Vd(x¯)$ amounts to summing all the entries within an $R$-by-$R$ diagonal block submatrix $Jij(x¯)$ within the full synaptic matrix $Jij$ (recall that $Vd(x¯)∝∑nkf(in(x¯))Jnkf(ik(x¯))$). Each set of disorder connections is a random permutation of $Jij0$, and thus has the same number of excitatory entries as $Jij0$, namely, $2pN$. Since the inhibitory connections do not play a role in the summation by virtue of equation E.1, it suffices to consider only the effect of adding excitatory connections in $Jijd$ to $Jij0$.

There are $M-1$ sets of disordered connections in $Jijd$, and each has $2pN$ excitatory connections. Suppose we add these $2pN(M-1)$ excitatory connections one by one to $Jij0$. Each time an excitatory entry is added to an entry $y$ in the $R$-by-$R$ block $Jij(x¯)$, there are two possible situations depending on the value of $y$ before addition: if $y=J(1-ε)$ (excitatory), the addition of an excitatory connection does not change the value of $y$ because of the clipping rule in equation E.1; if $y=-Jε$ (inhibitory), the addition of an excitatory connection to $y$ changes $y$ to $J(1-ε)$. In the latter case, the value of $Vd(x¯)$ is changed because the summation of entries within $Jij(x¯)$ has changed, while in the former case, $Vd(x¯)$ stays the same. (Note that if the excitatory connection is added outside $Jij(x¯)$, it does not change $Vd(x¯)$ and thus can be neglected.)

We have in total $2pN(M-1)$ excitatory connections to be added, and in total $(2R-p)2$ potential inhibitory connections in the $R$-by-$R$ block $Jij(x¯)$ to be flipped to an excitatory connection. We are interested in, after adding all the $2pN(M-1)$ excitatory connections, how many inhibitory connections are changed to excitatory connections and the corresponding change in $Vd(x¯)$.

We can get an approximate solution if we assume that the probability of flipping an inhibitory connection does not change after the subsequent addition of excitatory connections and stays constant throughout the addition of all the $2pN(M-1)$ excitatory connections. This requires $2pN(M-1)≪N2$, that is, $M≪N$, which is a reasonable assumption since the capacity cannot be $O(N)$.

For a single addition of exitatory connection, the probability of successfully flipping an inhibitory connection within $Jij(x¯)$ is proportional to the fraction of the inhibitory connections within $Jij(x¯)$ over the total number of entires in $Jij0$:
$q(flip)=(2R-p)2N2.$
(E.4)
So the probability of getting $n$ inhibitory connections flipped is
$P(n)=2pN(M-1)nqn(1-q)2pN(M-1)-n.$
(E.5)
In other words, the distribution of flipping $n$ inhibitory connections to excitatory connections after adding $Jijd$ to $Jij0$ obeys $n∼B(2pN(M-1),q)$. The mean is then
$〈n〉=2pN(M-1)q=2p(2R-p)2M-1N=(2R-p)22pm,$
(E.6)
where we have defined the reduced disorder parameter $m≡(M-1)/N$. The variance is
$〈n2〉=2pN(M-1)q(1-q)=2pN(M-1)(2R-p)2N21-(2R-p)2N2≈(2R-p)22pm,$
(E.7)
where in the last line we have used $N≫2R-p$.
Since changing $n$ inhibitory connections to $n$ excitory connections amounts to changing $Vd(x¯)$ by $-1/2(J(1-ε)-J(-ε))=-J/2$, we have
$Var(Vd(x¯))≡σ2=J2(R-p/2)2pm.$
(E.8)

### E.3  Disorder Temperature from Disorder-Induced Force

We focus here on the case where $Inext$ gives rise to a constant velocity $v$ for the droplet (as in the main text). In the co-moving frame, the disorder-induced force $Fd(x¯)$ acts on the c.o.m. like random kicks with correlation within the droplet size. For fast enough velocity, those random kicks are sufficiently decorrelated and become white noise at temperature $Td$.

To extract this disorder-induced temperature $Td$, we consider the autocorrelation of $Fd[x¯(t)]$ between two different c.o.m. locations $x¯(t)$ and $x¯'(t')$ (and thus different times $t$ and $t'$),
$C(t,t')≡〈Fd[x¯(t)]Fd[x¯(t')]〉,$
(E.9)
where the expectation value is averaging over different realizations of the quenched disorder.
Using equation E.3, we have
$C(t,t')=〈∂x¯Vd(x¯)∂x¯'Vd(x¯')〉$
(E.10)
$=∂x¯∂x¯'〈Vd(x¯)Vd(x¯')〉.$
(E.11)
Within time $t-t'$, if the droplet moves a distance less than its size $2R$, then $Vd$ computed at $t$ and $t'$ will be correlated because $f(ikx¯)$ and $f(ikx'¯)$ have nonzero overlap. Therefore, we expect that the autocorrelation function $〈Vd(x¯)Vd(x¯')〉$ behaves like the 1D Ising model with finite correlation length $ξ=2R$ (up to a prefactor to be fixed later):
$〈Vd(x¯)Vd(x¯')〉∼exp-|x¯-x¯'|ξ.$
(E.12)
Hence, $C(t,t')∼exp-|x¯-x¯'|ξ$. Going to the co-moving frame, we can write the c.o.m. location as before, $Δxv=x¯-vt$, so the autocorrelation function becomes
$C(t,t')∼exp-|(Δxv+vt)-(Δxv'+vt')|ξ=exp-|v(t-t')+(Δxv-Δxv')|ξ≈exp-v|t-t'|ξ,$
(E.13)
where in the last line, we have used that the droplet moves much faster in the stationary frame than the c.o.m. position fluctuates in the co-moving frame, so $v(t-t')≫Δxv-Δxv'$.
Now let us define the correlation time to be $τcor=ξ/v=2R/v$. Then
$C(t,t')∼exp-|t-t'|τcor.$
(E.14)
For $T≡|t-t'|≫τcor$, we want to consider the limiting behavior of $C(t,t')$ under an integral. Note that
$∫0Tdt∫0Tdt'exp-|t-t'|τcor=τcor[2(T-τcor)+2τcore-T/τcor]≈2τcorT(ifT≫τcor).$
(E.15)
Therefore, we have for $T≫τcor$,
$∫0Tdt∫0Tdt'exp-|t-t'|τcor=2τcor∫0Tdt∫0Tdt'δ(t-t'),$
(E.16)
so we can write
$exp-|t-t'|τcor→2τcorδ(t-t'),$
(E.17)
and it is understood that this holds in the integral sense. Therefore, for $T≫τcor$, we expect $Fd(x)$ to act like uncorrelated white noise, and we can write
$C(t,t')=Tdδ(t-t')∝τcorδ(t-t'),$
(E.18)
where $Td$ is a measure of this disorder-induced white noise.
To deduce the form of disorder temperature $Td$, we present the uncollapsed occupancies $-logp(Δxv)=V(Δxv)/kBTd$ (described in the caption of Figure 3) in Figure 11. In a comparison with Figure 3, we can see that $Td$ successfully captures the effect of disorder on the statistics of the emergent droplet if
$Td=k˜τcorσ,$
(E.19)
where $σ$ is given in equation E.8 and $k˜$ is a fitting constant.
Figure 11:

Uncollapsed data for the occupancies $-logp(Δxv)$ for different amounts of long-range disordered connections. Parameters are the same as in Figure 3 (see section of F.1 for further details).

Figure 11:

Uncollapsed data for the occupancies $-logp(Δxv)$ for different amounts of long-range disordered connections. Parameters are the same as in Figure 3 (see section of F.1 for further details).

In this section, we derive the memory capacity for driven place cell network described equation 6.1.

Our continuous attractor network can be applied to study the place cell network. We assume a 1D physical region of length $L$. We study a network with $N$ place cell neurons and assume each neuron has a place field of size $d=2pL/N$ that covers the region $[0,L]$ as a regular tiling. The $N$ neurons are assumed to interact as in the leaky integrate-and-fire model of neurons. The external driving currents $Iext(x,t)$ can model sensory input when the mouse is physically in a region covered by place fields of neurons $i,i+1,…,i+z$; currents $Iiext$ through $Ii+zext$ can be expected to be high compared to all other currents $Ijext$, which corresponds to the cup-shape drive we used throughout the main text.

It has been shown that the collective coordinate in the continuous attractor survives to multiple environments provided the number of stored memories $m is below the capacity $mc$ of the network. Below the capacity, the neural activity droplet is multistable; that is, neural activity forms a stable contiguous droplet as seen in the place field arrangement corresponding to any one of the $m$ environments. Note that such a contiguous droplet will not appear contiguous in the place field arrangement of any other environment. Capacity was shown to scale as $mc=α(p/N,R)N$, where $α$ is an $O(1)$ number that depends on the size of the droplet $R$ and the range of interactions $p$. However, this capacity is about the intrinsic stability of the droplet and does not consider the effect of rapid driving forces.

When the droplet escapes from the driving signal, it has to overcome a certain energy barrier. This is the difference in $Veff$ between the two extremal points $Δxv*$ and $Δxvesc$. Therefore, we define the barrier energy to be $ΔE=Veff(xvesc)-Veff(Δxv*)$, and we evaluate it using equations C.1 and C.2:
$ΔE(v,d)=(4dw-3γv-2dR)(-γv+2dR)4d.$
(F.1)

Note this is the result we used in equation 6.1.

As in the main text, the escape rate $r$ is given by the Arrhenius law:
$r∼exp-ΔE(v,d)kBTd.$
(F.2)
The total period of time of an external drive moving the droplet across a distance $L$ ($L≤N$, but without loss of generality, we can set $L=N$) is $T=L/v$. We can imagine chopping $T$ into infinitesimal intervals $Δt$ s.t. the probability of successfully moving the droplet across $L$ without escaping is
$Pretrieval=limΔt→0(1-rΔt)TΔt=e-rT=e-rN/v=exp(-Nve-ΔE(v,d)/kBTd).$
(F.3)
$Td$ is given by equation E.19:
$Td=2k˜RJ(R-p/2)pmv≡kmv-1,$
(F.4)
where in the last step, we have absorbed all the constants (assuming $R$ is constant over different $m$'s) into the definition of $k$. Now we want to find the scaling behavior of $m$ s.t. In the thermodynamic limit ($N→∞$), $Pretrieval$ becomes a Heaviside step function $Θ(mc-m)$ at some critical memory $mc$. With the aid of some hindsight, we try
$m=α2(logN)2.$
(F.5)
Then in the thermodynamic limit,
$limN→∞Pretrieval=limN→∞exp(-Nve-logNvΔE(v,d)/αkBk)=limN→∞exp(-NvN-vΔE(v,d)/αkBk)=limN→∞exp(-1vN1-vΔE(v,d)/αkBk)=1,αvΔE(v,d)/kBk.$
(F.6)
Therefore, we have arrived at the expression for capacity $mc$ or, in terms of $M=mcN+1≈mcN(N≫1)$,
$Mc=vΔE(v,d)kBk2N(logN)2$
(F.7)
or
$Mc∼vΔE(v,d)2N(logN)2.$
(F.8)

### F.1  Numerics of the Place Cell Network Simulations

In this section, we explain our simulations in Fig. 4 in detail.

Recall that we only determine the Arrhenius-like escape rate $r$ up to an overall constant. We can absorb it into the definition of $ΔE(v,d)$ (given by equation F.1) as an additive constant $a$,
$r=exp-ΔE(v,d)+akBkv(M-1)/N.$
(F.9)
Then the theoretical curve corresponds to
$Pretrieval=e-Nr/v.$
(F.10)

Therefore, our model, equation F.10, has three parameters to determine: $γ$, $k$, and $a$. In Figure 12 we determine the parameters by collapsing data and see that the best fit is found provided $γ=240.30,k=5255.0kB-1,a=-0.35445$. Henceforth, we fix these three parameters to these values.

Figure 12:

Top: Plotting $-1/Tdv=log{v-1logr/[ΔE(v,d)+a]}$ against $log(M-1)$. Different solid lines correspond to data with different $v$, and the dashed line corresponds to the $(M-1)-1/2$ curve. Bottom: Plotting $v-1logrM-1∝ΔE(v,d)$ against $v$. Different solid lines correspond to data with different $M$, and the dashed line corresponds to the $ΔE(v,d)+a$ curve.

Figure 12:

Top: Plotting $-1/Tdv=log{v-1logr/[ΔE(v,d)+a]}$ against $log(M-1)$. Different solid lines correspond to data with different $v$, and the dashed line corresponds to the $(M-1)-1/2$ curve. Bottom: Plotting $v-1logrM-1∝ΔE(v,d)$ against $v$. Different solid lines correspond to data with different $M$, and the dashed line corresponds to the $ΔE(v,d)+a$ curve.

In the bottom plot of Figure 12, we offset the effect of $M$ by multiplying $v-1logr$ by $M-1$, and we see that curves corresponding to different $M$ collapse to each other, confirming the $M-1$ dependence in $Td$. The collapsed line we are left with is just the $v$-dependence of $ΔE(v,d)$, up to overall constant.

In the top panel of Figure 12, we offset the effect of $v$ in $Td$ by multiplying $v-1$ to $logr/[ΔE(v,d)+a]$. We see that different curves corresponding to different $v$'s collapse to each other, confirming the $v-1$ dependence in $Td$. The curve we are left with is the $M$ dependence in $Td$, which we see fits nicely with the predicted $M-1$.

In Figure 4b, we run our simulation with the following parameters held fixed: $N=4000,p=10,ε=0.35,τ=1,J=100,d=10$, and $w=30$. Along the same curve, we vary $M$ from 6 to 30, and the series of curves corresponds to different $v$ from 0.6 to 1.2.

In Figure 4c, we hold the following parameters fixed: $p=10,ε=0.35,τ=1,J=100,d=10,w=30$, and $v=0.8$. Along the same curve, we vary $M/N(logN)2$ from 0.1 to 0.6, and the series of curves corresponds to different $N$ from 1000 to 8000.

In Figures 4b and 4c, the theoretical model we used is equation F.10 with the same parameters given above.

In Figure 4d, we replotted the theory and data from Figure 4b. For the theoretical curve, we find the location where $Pretrieval=0.5$, and call the corresponding $M$ value “theoretical capacity.” For the simulation curve, we extrapolate to where $Pretrieval=0.5$, and call the corresponding $M$ value the “simulation capacity.”

For all simulation curves above, we drag the droplet from one end of the continuous attractor to the other end of the attractor and run the simulation 300 times. We then measure the fraction of successful events (defined as the droplet survived in the cup throughout the entire trajectory of moving) and failed events (defined as the droplet escape from the cup at some point before reaching the other end of the continuous attractor). We define the simulation $Pretreival$ as the fraction of successful events.

We thank Jeremy England, Ila Fiete, John Hopfield, and Dmitry Krotov for discussions. A.M. and D.S. are grateful for support from the Simons Foundation MMLS investigator program. We acknowledge the University of Chicago Research Computing Center for support of this work.

Alme
,
C. B.
,
Miao
,
C.
,
Jezek
,
K.
,
Treves
,
A.
,
Moser
,
E. I.
, &
Moser
,
M.-B.
(
2014
).
Place cells in the hippocampus: Eleven maps for eleven rooms
.
Proc. Natl. Acad. Sci. U.S.A.
,
111
(
52
),
18428
18435
.
Amari
,
S.
(
1977
).
Dynamics of pattern formation in lateral-inhibition type neural fields
.
Biol. Cybern.
,
27
(
2
),
77
87
.
Amit
,
D.
,
Gutfreund
,
H.
, &
Sompolinsky
,
H.
(
1985a
).
Storing infinite numbers of patterns in a spin-glass model of neural networks
.
Phys. Rev. Lett.
,
55
(
14
),
1530
1533
.
Amit
,
D. J.
,
Gutfreund
,
H.
, &
Sompolinsky
,
H.
(
1985b
).
Spin-glass models of neural networks
.
Phys. Rev. A
,
32
(
2
),
1007
.
Aronov
,
D.
,
Nevers
,
R.
, &
Tank
,
D. W.
(
2017
).
Mapping of a non-spatial dimension by the hippocampal-entorhinal circuit
.
Nature
,
543
(
7647
),
719
722
.
Battaglia
,
F.
, &
Treves
,
A.
(
1998
).
Attractor neural networks storing multiple space representations: A model for hippocampal place fields
.
Physical Review E
,
58
(
6
),
7738
7753
.
Burak
,
Y.
, &
Fiete
,
I. R.
(
2012
).
Fundamental limits on persistent activity in networks of noisy neurons
.
Proc. Natl. Acad. Sci. U.S.A.
,
109
(
43
),
17645
17650
.
Campbell
,
M. G.
,
Ocko
,
S. A.
,
Mallory
,
C. S.
,
Low
,
I. I.
,
Ganguli
,
S.
, &
Giocomo
,
L. M.
(
2018
).
Principles governing the integration of landmark and self-motion cues in entorhinal cortical codes for navigation
.
Nature Neuroscience
,
21
(
8
),
1096
.
Chaudhuri
,
R.
, &
Fiete
,
I.
(
2016
).
Computational principles of memory
.
Nat. Neurosci.
,
19
(
3
),
394
403
.
Colgin
,
L. L.
,
Leutgeb
,
S.
,
Jezek
,
K.
,
Leutgeb
,
J. K.
,
Moser
,
E. I.
,
McNaughton
,
B. L.
, &
Moser
,
M.-B.
(
2010
).
Attractor-map versus autoassociation based attractor dynamics in the hippocampal network
.
J. Neurophysiol.
,
104
(
1
),
35
50
.
Erdem
,
U. M.
, &
Hasselmo
,
M.
(
2012
).
A goal-directed spatial navigation model using forward trajectory planning based on grid cells
.
Eur. J. Neurosci.
,
35
(
6
),
916
931
.
Evans
,
T.
,
Bicanski
,
A.
,
Bush
,
D.
, &
Burgess
,
N.
(
2016
).
How environment and self-motion combine in neural representations of space
.
J. Physiol.
,
594
(
22
),
6535
6546
.
Fung
,
C. A.
,
Wong
,
K. M.
,
Mao
,
H.
, &
Wu
,
S.
(
2015
).
Fluctuation-response relation unifies dynamical behaviors in neural fields
.
Physical Review E
,
92
(
2
),
022801
.
Fyhn
,
M.
,
Hafting
,
T.
,
Treves
,
A.
,
Moser
,
M.-B.
, &
Moser
,
E. I.
(
2007
).
Hippocampal remapping and grid realignment in entorhinal cortex
.
Nature
,
446
(
7132
),
190
194
.
Hardcastle
,
K.
,
Ganguli
,
S.
, &
Giocomo
,
L. M.
(
2015
).
Environmental boundaries as an error correction mechanism for grid cells
.
Neuron
,
86
(
3
),
827
839
.
Hardcastle
,
K.
,
Maheswaranathan
,
N.
,
Ganguli
,
S.
, &
Giocomo
,
L. M.
(
2017
).
A multiplexed, heterogeneous, and adaptive code for navigation in medial entorhinal cortex
.
Neuron
,
94
(
2
),
375
387
.
Hertz
,
J.
,
Krogh
,
A.
,
Palmer
,
R. G.
, &
Horner
,
H.
(
1991
).
Introduction to the theory of neural computation
.
Physics Today
,
44
,
70
.
Hopfield
,
J. J.
(
1982
).
Neural networks and physical systems with emergent collective computational abilities.
In
PNAS
,
79
,
2554
2558
.
Hopfield
,
J. J.
(
2010
).
Neurodynamics of mental exploration
.
Proceedings of the National Academy of Sciences
,
107
(
4
),
1648
1653
.
Hopfield
,
J. J.
(
2015
).
Understanding emergent dynamics: Using a collective activity coordinate of a neural network to recognize time-varying patterns
.
Neural Comput.
,
27
(
10
),
2011
2038
.
Kilpatrick
,
Z. P.
,
Ermentrout
,
B.
, &
Doiron
,
B.
(
2013
).
Optimizing working memory with heterogeneity of recurrent cortical excitation
.
J. Neurosci.
,
33
(
48
),
18999
19011
.
Kim
,
S. S.
,
Rouault
,
H.
,
Druckmann
,
S.
, &
Jayaraman
,
V.
(
2017
).
Ring attractor dynamics in the drosophila central brain
.
Science
,
356
(
6340
),
849
853
.
Kubie
,
J. L.
, &
Muller
,
R. U.
(
1991
).
Multiple representations in the hippocampus
.
Hippocampus
,
1
(
3
),
240
242
.
Latham
,
P. E.
,
Deneve
,
S.
, &
Pouget
,
A.
(
2003
).
Optimal computation with attractor networks
.
J. Physiol. Paris
,
97
(
4–6
),
683
694
.
Lim
,
S.
, &
Goldman
,
M. S.
(
2012
).
Noise tolerance of attractor and feedforward memory models
.
Neural Comput.
,
24
(
2
),
332
390
.
Major
,
G.
,
Baker
,
R.
,
Aksay
,
E.
,
Seung
,
H. S.
, &
Tank
,
D. W.
(
2004
).
Plasticity and tuning of the time course of analog persistent firing in a neural integrator
.
Proc. Natl. Acad. Sci. U.S.A.
,
101
(
20
),
7745
7750
.
McNaughton
,
B. L.
,
Battaglia
,
F. P.
,
Jensen
,
O.
,
Moser
,
E. I.
, &
Moser
,
M.-B.
(
2006
).
Path integration and the neural basis of the “cognitive map
.”
Nat. Rev. Neurosci.
,
7
(
8
),
663
678
.
Meshulam
,
L.
,
Gauthier
,
J. L.
,
Brody
,
C. D.
,
Tank
,
D. W.
, &
Bialek
,
W.
(
2017
).
Collective behavior of place and non-place neurons in the hippocampal network
.
Neuron
,
96
(
5
),
1178
1191
.
Mi
,
Y.
,
Fung
,
C. A.
,
Wong
,
K. M.
, &
Wu
,
S.
(
2014
).
Spike frequency adaptation implements anticipative tracking in continuous attractor neural networks
. In
Z.
Ghahramani
,
M.
Welling
,
C.
Cortes
,
N. D.
Lawrence
, &
K. Q.
Weinberger
(Eds.),
Advances in neural information processing systems
,
27
(pp.
505
513
).
Red Hook, NY
:
Curran
.
Monasson
,
R.
, &
Rosay
,
S.
(
2013
).
Crosstalk and transitions between multiple spatial maps in an attractor neural network model of the hippocampus: Phase diagram
.
Physical Review E
,
87
(
6
),
062813
.
Monasson
,
R.
, &
Rosay
,
S.
(
2014
).
Crosstalk and transitions between multiple spatial maps in an attractor neural network model of the hippocampus: Collective motion of the activity
.
Physical Review E
,
89
(
3
).
Monasson
,
R.
, &
Rosay
,
S.
(
2015
).
Transitions between spatial attractors in place-cell models
.
Phys. Rev. Lett.
,
115
(
9
),
098101
.
Moser
,
E. I.
,
Moser
,
M.-B.
, &
McNaughton
,
B. L.
(
2017
).
Spatial representation in the hippocampal formation: A history
.
Nat. Neurosci.
,
20
(
11
),
1448
1464
.
Moser
,
E. I.
,
Moser
,
M.-B.
, &
Roudi
,
Y.
(
2014
).
Network mechanisms of grid cells
.
Philos. Trans. R. Soc. Lond. B Biol. Sci.
,
369
(
1635
),
20120511
.
Ocko
,
S. A.
,
Hardcastle
,
K.
,
Giocomo
,
L. M.
, &
Ganguli
,
S.
(
2018
).
Emergent elasticity in the neural code for space
.
Proc. Natl. Acad. Sci. U.S.A.
,
115
(
50
),
E11798
E11806
.
O'Keefe
,
J.
, &
Dostrovsky
,
J.
(
1971
).
The hippocampus as a spatial map: Preliminary evidence from unit activity in the freely-moving rat
.
Brain Res.
,
34
(
1
),
171
175
.
Pfeiffer
,
B. E.
, &
Foster
,
D. J.
(
2013
).
Hippocampal place-cell sequences depict future paths to remembered goals
.
Nature
,
497
(
7447
),
74
79
.
Ponulak
,
F.
, &
Hopfield
,
J. J.
(
2013
).
Rapid, parallel path planning by propagating wavefronts of spiking neural activity
.
Front. Comput. Neurosci.
,
7
.
Poucet
,
B.
, &
Save
,
E.
(
2005
).
Neuroscience: Attractors in memory
.
Science
,
308
(
5723
),
799
800
.
Roudi
,
Y.
, &
Latham
,
P. E.
(
2007
).
A balanced memory network
.
PLOS Comput. Biol.
,
3
(
9
),
1679
1700
.
Seelig
,
J. D.
, &
Jayaraman
,
V.
(
2015
).
Neural dynamics for landmark orientation and angular path integration
.
Nature
,
521
(
7551
),
186
191
.
Seung
,
H. S.
(
1996
).
How the brain keeps the eyes still
.
Proceedings of the National Academy of Sciences
,
93
(
23
),
13339
13344
.
Seung
,
H. S.
(
1998
).
Continuous attractors and oculomotor control
.
Neural Netw.
,
11
(
7–8
),
1253
1258
.
Seung
,
H. S.
,
Lee
,
D. D.
,
Reis
,
B. Y.
, &
Tank
,
D. W.
(
2000
).
Stability of the memory of eye position in a recurrent network of conductance-based model neurons
.
Neuron
,
26
(
1
),
259
271
.
Sontag
,
E. D.
(
2003
).
Adaptation and regulation with signal detection implies internal model
.
Syst. Control Lett.
,
50
(
2
),
119
126
.
Tsodyks
,
M.
(
1999
).
Attractor neural network models of spatial maps in hippocampus
.
Hippocampus
,
9
(
4
),
481
489
.
Turner-Evans
,
D.
,
Wegener
,
S.
,
Rouault
,
H.
,
Franconville
,
R.
,
Wolff
,
T.
,
Seelig
,
J. D.
, …
Jayaraman
,
V.
(
2017
).
Angular velocity integration in a fly heading circuit
.
Elife
,
6
.
Wills
,
T. J.
,
Lever
,
C.
,
Cacucci
,
F.
,
Burgess
,
N.
, &
O'Keefe
,
J.
(
2005
).
Attractor dynamics in the hippocampal representation of the local environment
.
Science
,
308
(
5723
),
873
876
.
Wimmer
,
K.
,
Nykamp
,
D. Q.
,
Constantinidis
,
C.
, &
Compte
,
A.
(
2014
).
Bump attractor dynamics in prefrontal cortex explains behavioral precision in spatial working memory
.
Nat. Neurosci.
,
17
(
3
),
431
439
.
Wu
,
S.
, &
Amari
,
S.-I.
(
2005
).
Computing with continuous attractors: Stability and online aspects
.
Neural Comput.
,
17
(
10
),
2215
2239
.
Wu
,
S.
,
Hamaguchi
,
K.
, &
Amari
,
S.-I.
(
2008
).
Dynamics and computation of continuous attractors
.
Neural Comput.
,
20
(
4
),
994
1025
.
Yoon
,
K.
,
Buice
,
M. A.
,
Barry
,
C.
,
Hayman
,
R.
,
Burgess
,
N.
, &
Fiete
,
I. R.
(
2013
).
Specific evidence of low-dimensional continuous attractor dynamics in grid cells
.
Nat. Neurosci.
,
16
(
8
),
1077
1084
.

## Author notes

D.S. and A.M. contributed equally to this work.