Continuous attractors have been used to understand recent neuroscience experiments where persistent activity patterns encode internal representations of external attributes like head direction or spatial location. However, the conditions under which the emergent bump of neural activity in such networks can be manipulated by space and time-dependent external sensory or motor signals are not understood. Here, we find fundamental limits on how rapidly internal representations encoded along continuous attractors can be updated by an external signal. We apply these results to place cell networks to derive a velocity-dependent nonequilibrium memory capacity in neural networks.

Dynamical attractors have found much use in neuroscience as models for carrying out computation and signal processing (Poucet & Save, 2005). While point-like neural attractors and analogies to spin glasses have been widely explored (Hopfield, 1982; Amit, Gutfreund, & Sompolinsky, 1985b), an important class of experiments is explained by continuous attractors, where the collective dynamics of strongly interacting neurons stabilizes a low-dimensional family of activity patterns. Such continuous attractors have been invoked to explain experiments on motor control based on path integration (Seung, 1996; Seung, Lee, Reis, & Tank, 2000), head direction (Kim, Rouault, Druckmann, & Jayaraman, 2017) control, spatial representation in grid or place cells (Yoon et al., 2013; O'Keefe & Dostrovsky, 1971; Colgin et al., 2010; Wills, Lever, Cacucci, Burgess, & O'Keefe, 2005; Wimmer, Nykamp, Constantinidis, & Compte, 2014; Pfeiffer & Foster, 2013), among other information processing tasks (Hopfield, 2015; Roudi & Latham, 2007; Latham, Deneve, & Pouget, 2003; Burak & Fiete, 2012).

These continuous attractor models are at the fascinating intersection of dynamical systems and neural information processing. The neural activity in these models of strongly interacting neurons is described by an emergent collective coordinate (Yoon et al., 2013; Wu, Hamaguchi, & Amari, 2008; Amari, 1977). This collective coordinate stores an internal representation (Sontag, 2003; Erdem & Hasselmo, 2012) of the organism's state in its external environment, such as position in space (Pfeiffer & Foster, 2013; McNaughton et al., 2006) or head direction (Seelig & Jayaraman, 2015).

However, such internal representations are useful only if they can be driven and updated by external signals that provide crucial motor and sensory input (Hopfield, 2015; Pfeiffer & Foster, 2013; Erdem & Hasselmo, 2012; Hardcastle, Ganguli, & Giocomo, 2015; Ocko, Hardcastle, Giocomo, & Ganguli, 2018). Driving and updating the collective coordinate using external sensory signals opens up a variety of capabilities, such as path planning (Ponulak & Hopfield, 2013; Pfeiffer & Foster, 2013), correcting errors in the internal representation or in sensory signals (Erdem & Hasselmo, 2012; Ocko et al., 2018), and the ability to resolve ambiguities in the external sensory and motor input (Hardcastle et al., 2015; Evans, Bicanski, Bush, & Burgess, 2016; Fyhn, Hafting, Treves, Moser, & Moser, 2007).

In all of these examples, the functional use of attractors requires interaction between external signals and the internal recurrent network dynamics. However, with a few significant exceptions (Fung, Wong, Mao, & Wu, 2015; Mi, Fung, Wong, & Wu, 2014; Wu et al., 2008; Wu & Amari, 2005; Monasson & Rosay, 2014; Burak & Fiete, 2012), most theoretical work has either been in the limit of no external forces and strong internal recurrent dynamics, or in the limit of strong external forces where the internal recurrent dynamics can be ignored (Moser, Moser, & McNaughton, 2017; Tsodyks, 1999).

Here, we study continuous attractors in neural networks subject to external driving forces that are neither small relative to internal dynamics nor adiabatic. We show that the physics of the emergent collective coordinate sets limits on the maximum speed at which internal representations can be updated by external signals.

Our approach begins by deriving simple classical and statistical laws satisfied by the collective coordinate of many neurons with strong, structured interactions that are subject to time-varying external signals, Langevin noise, and quenched disorder. Exploiting these equations, we demonstrate two simple principles: (1) an equivalence principle that predicts how much the internal representation lags a rapidly moving external signal, and (2) under externally driven conditions, quenched disorder in network connectivity that can be modeled as a state-dependent effective temperature. Finally, we apply these results to place cell networks and derive a nonequilibrium driving-dependent memory capacity, complementing numerous earlier works on memory capacity in the absence of external driving.

We study N interacting neurons following the formalism presented in Hopfield (2015),
dindt=-inτ+k=1NJnkf(ik)+Inext(t)+ηint(t),
(2.1)
where f(ik)=(1+e-ik/i0)-1 is the neural activation function that represents the firing rate of neuron k, and in is an internal excitation level of neuron n akin to the membrane potential. We consider synaptic connectivity matrices with two distinct components:
Jij=Jij0+Jijd.
(2.2)

As shown in Figure 1, Jij0 encodes the continuous attractor. We will focus on 1D networks with p-nearest neighbor excitatory interactions to keep bookkeeping to a minimum: Jij0=J(1-ε) if neurons |i-j|p, and Jij0=-Jε otherwise. The latter term, -Jε, with 0ε1, represents long-range, nonspecific inhibitory connections as frequently assumed in models of place cells (Monasson & Rosay, 2014; Hopfield, 2010), head direction cells (Chaudhuri & Fiete, 2016), and other continuous attractors (Seung et al., 2000; Burak & Fiete, 2012).

Figure 1:

The effective dynamics of neural networks implicated in head direction and spatial memory is described by a continuous attractor. Consider N neurons connected in a 1D topology, with local excitatory connections between p nearest neighbors (blue), global inhibitory connections (not shown), and random long-range disorder (orange). Any activity pattern quickly condenses into a droplet of contiguous firing neurons (red) of characteristic size; the droplet center of mass x¯ is a collective coordinate parameterizing a continuous attractor. The droplet can be driven by space and time-varying external currents Inext(t) (green).

Figure 1:

The effective dynamics of neural networks implicated in head direction and spatial memory is described by a continuous attractor. Consider N neurons connected in a 1D topology, with local excitatory connections between p nearest neighbors (blue), global inhibitory connections (not shown), and random long-range disorder (orange). Any activity pattern quickly condenses into a droplet of contiguous firing neurons (red) of characteristic size; the droplet center of mass x¯ is a collective coordinate parameterizing a continuous attractor. The droplet can be driven by space and time-varying external currents Inext(t) (green).

Close modal

The disorder matrix Jijd represents random long-range connections, a form of quenched disorder (Seung, 1998; Kilpatrick, Ermentrout, & Doiron, 2013). Finally, Inext(t) represents external driving currents from, for example, sensory and motor input possibly routed through other regions of the brain. The Langevin noise ηint(t) represents private noise internal to each neuron (Lim & Goldman, 2012; Burak & Fiete, 2012).

A neural network like equation 2.1 qualitatively resembles a similarly connected network of Ising spins at fixed magnetization (Monasson & Rosay, 2014). At low noise, the activity in such a system will condense (Monasson & Rosay, 2014; Hopfield, 2010) to a localized droplet, since interfaces between firing and nonfiring neurons are penalized by J(1-ε). The center of mass of such a droplet, x¯nnf(in)nf(in) is an emergent collective coordinate that approximately describes the stable low-dimensional neural activity patterns of these N neurons. Fluctuations about this coordinate have been extensively studied (Wu et al., 2008; Burak & Fiete, 2012; Hopfield, 2015; Monasson & Rosay, 2014).

We focus on how space and time-varying external signals, modeled here as external currents Inext(t), can drive and reposition the droplet along the attractor. We will be primarily interested in a cup-shaped current profile that moves at a constant velocity v, Inext(t)=Icup(n-vt), where Icup(n)=d(w-|n|),n[-w,w], Icup(n)=0 otherwise. Such a localized time-dependent drive could represent landmark-related sensory signals (Hardcastle et al., 2015).

The effective dynamics of the collective coordinate x¯ in the presence of currents Inext(t) can be obtained by computing the effective force on the droplet of finite size. We find that (see appendix A)
γx¯˙=-x¯Vext(x¯,t),
(3.1)
where Vext(x¯,t) is a piecewise quadratic potential Vcup(x¯-vt) for currents Inext(t)=Icup(n-vt), and γ is the effective drag coefficient of the droplet. (Here, we neglect rapid transients of timescale τ (Wu et al., 2008).)

The strength of the external signal is set by the depth d of the cup Icup(n). Previous studies have explored the d=0 case—undriven diffusive dynamics of the droplet (Burak & Fiete, 2012; Monasson & Rosay, 2013, 2014, 2015) or the large d limit (Hopfield, 2015) when the internal dynamics can be ignored. Here we focus on an intermediate regime, d<dmax, where internal representations are updated continuously by the external currents, without any jumps (Ponulak & Hopfield, 2013; Pfeiffer & Foster, 2013; Erdem & Hasselmo, 2012).

In fact, as shown in the section C.2 we find a threshold signal strength dmax beyond which the external signal destabilizes the droplet, instantly “teleporting” the droplet from any distant location to the cup without continuity along the attractor, erasing any prior positional information held in the internal representation.

We focus here on d<dmax, a regime with continuity of internal representations. Such continuity is critical for many applications, such as path planning (Ponulak & Hopfield, 2013; Pfeiffer & Foster, 2013; Erdem & Hasselmo, 2012) and resolving local ambiguities in position within the global context (Hardcastle et al., 2015; Evans et al., 2016; Fyhn et al., 2007). In this regime, the external signal updates the internal representation with finite gain (Fyhn et al., 2007) and can thus fruitfully combine information in both the internal representation and the external signal. Other applications that simply require short-term memory storage of a strongly fluctuating variable may not require this continuity restriction.

3.1  Equivalence Principle

We first consider driving the droplet in a network at constant velocity v using an external current Inext=Icup(n-vt). We allow for Langevin noise but no disorder in the couplings Jd=0 in this section. For very slow driving (v0), the droplet will settle into and track the bottom of the cup. When driven at a finite velocity v, the droplet cannot stay at the bottom since there is no net force exerted by the currents Inext at that point.

Instead, the droplet must lag the bottom of the moving external drive by an amount Δxv=x¯-vt such that the slope of the potential Vcup provides an effective force Fvmotionγv needed to keep the droplet in motion at velocity v, that is,
-x¯Vcup(Δxv)=Fvmotionγv.
(3.2)
This equation, which we call an equivalence principle in analogy with inertial particles in an accelerated frame, is verified by simulations in Figure 2b. Similar results on a lag between driving forces and the response were obtained in earlier works (Fung et al., 2015; Mi et al., 2014).
Figure 2:

(a) The mean position and fluctuations of the droplet driven by currents Inext=Icup(n-vt) are described by an “equivalence” principle; in a frame co-moving with Incup(t) with velocity v, we simply add an effective force Fvmotion=γv where γ is a drag coefficient. (b) This prescription correctly predicts that the droplet lags the external driving force by an amount linearly proportional to velocity v, as seen in simulations. (c) Fluctuations of the driven droplet's position, due to internal noise in neurons, are also captured by the equivalence principle. If p(Δxv) is the probability of finding the droplet at a lag Δxv, we find that kBTlogp(Δxv)-FvmotionΔxv is independent of velocity and can be collapsed onto each other (with fitting parameter T). (Inset: logp(Δxv) before subtracting FvmotionΔxv.)

Figure 2:

(a) The mean position and fluctuations of the droplet driven by currents Inext=Icup(n-vt) are described by an “equivalence” principle; in a frame co-moving with Incup(t) with velocity v, we simply add an effective force Fvmotion=γv where γ is a drag coefficient. (b) This prescription correctly predicts that the droplet lags the external driving force by an amount linearly proportional to velocity v, as seen in simulations. (c) Fluctuations of the driven droplet's position, due to internal noise in neurons, are also captured by the equivalence principle. If p(Δxv) is the probability of finding the droplet at a lag Δxv, we find that kBTlogp(Δxv)-FvmotionΔxv is independent of velocity and can be collapsed onto each other (with fitting parameter T). (Inset: logp(Δxv) before subtracting FvmotionΔxv.)

Close modal
In fact, we find that the the above equivalence principle predicts the entire distribution p(Δxv) of fluctuations of the lag Δxv due to Langevin noise (see Figure 2c). By binning the lag Δxv(t) for trajectories of the droplet obtained from repeated numerical simulations, we determined p(Δxv), the occupancy of the droplet in the moving frame of the drive. As detailed in appendix C, data for different velocities collapse using an effective temperature scale T, verifying that
kBTlogp(Δxv)=-(Vcup(Δxv)-FvmotionΔxv),
(3.3)

Our results here are consistent with the fluctuation-dissipation result obtained in Monasson and Rosay (2014) for driven droplets. In summary, in the co-moving frame of the driving signal, the droplet's position Δxv fluctuates as if it were in thermal equilibrium in the modified potential Veff=Vcup-FvmotionΔxv.

The simple equivalence principle implies a striking bound on the update speed of internal representations. A driving signal cannot drive the droplet at velocities greater than some vcrit if the predicted lag for v>vcrit is larger than the cup. In the appendix, we find vcrit=2d(w+R)/3γ, where 2R is the droplet size.

Larger driving strength d increases vcrit, but as was previously discussed, we require d<dmax in order to retain continuity and stability of the internal representation. Hence, we find an absolute upper bound on the fastest external signal that can be tracked by the internal representation,
v*=κpJγ-1,
(4.1)
where p is the range of interactions, J is the synaptic strength, γ-1 is the mobility or inverse drag coefficient of the droplet, and κ is a dimensionless O(1) number.

We now consider the effect of long-range quenched disorder Jijd in the synaptic matrix (Seung, 1998; Kilpatrick et al., 2013), which breaks the exact degeneracy of the continuous attractor, creating an effectively rugged landscape, Vd(x¯), as shown schematically in Figure 3 and computed in sections E.1 and E.2. When driven by a time-varying external signal, Iiext(t), the droplet now experiences a net potential Vext(x¯,t)+Vd(x¯). The first term causes motion with velocity v and a lag predicted by the equivalence principle, and for sufficiently large velocities v, the effect of the second term can be modeled as effective Langevin white noise. To see this, note that Vd(x¯) is uncorrelated on length scales larger than the droplet size; hence, for large enough droplet velocity v, the forces Fd(t)-x¯Vd|x¯=x¯(t) due to disorder are effectively random and uncorrelated in time. More precisely, let σ2=Var(Vd(x¯)). In section E.3, we compute Fd(t) and show that Fd(t) has an autocorrelation time, τcor=2R/v, due to the finite size of the droplet.

Figure 3:

Disorder in neural connectivity is well approximated by an effective temperature Td for a moving droplet. (a) Long-range disorder breaks the degeneracy of the continuous attractor, creating a rough landscape. A droplet moving at velocity v in this rough landscape experiences random forces. (b) The fluctuations of a moving droplet's position, relative to the cup's bottom, can be described by an effective temperature Td. We define a potential V(Δxv)=-kBTdlogp(Δxv) where p(Δxv) is the probability of the droplet's position fluctuating to a distance Δxv from the peak external current. We find that V(Δxv) corresponding to different amounts of disorder σ˜2 (where σ˜2 is the average number of long-range disordered connections per neuron in units of 2p), can be collapsed by the one fitting parameter Td. Inset: Td is linearly proportional to the strength of disorder σ˜.

Figure 3:

Disorder in neural connectivity is well approximated by an effective temperature Td for a moving droplet. (a) Long-range disorder breaks the degeneracy of the continuous attractor, creating a rough landscape. A droplet moving at velocity v in this rough landscape experiences random forces. (b) The fluctuations of a moving droplet's position, relative to the cup's bottom, can be described by an effective temperature Td. We define a potential V(Δxv)=-kBTdlogp(Δxv) where p(Δxv) is the probability of the droplet's position fluctuating to a distance Δxv from the peak external current. We find that V(Δxv) corresponding to different amounts of disorder σ˜2 (where σ˜2 is the average number of long-range disordered connections per neuron in units of 2p), can be collapsed by the one fitting parameter Td. Inset: Td is linearly proportional to the strength of disorder σ˜.

Close modal

Thus, on longer timescales, Fd(t) is uncorrelated and can be viewed as Langevin noise for the droplet center of mass x¯, associated with a disordered-induced temperature Td. Through repeated simulations with different amounts of disorder σ2, we inferred the distribution p(Δxv) of the droplet position in the presence of such disorder-induced fluctuations (see Figure 3). The data collapse in Figure 3b confirms that the effect of disorder (of size σ2) on a rapidly moving droplet can indeed be modeled by an effective disorder-induced temperature Tdστcor. (For simplicity, we assume that internal noise ηint in equation 2.1 is absent here. Note that in general, ηint will also contribute to Td. Here we focus on the contribution of disorder to an effective temperature Td since internal noise ηint has been considered in prior work (Fung et al., 2015).)

Thus, the disorder Jijd effectively creates thermal fluctuations about the lag predicted by the equivalence principle; such fluctuations may carry the droplet out of the driving cup Icup(n-vt) and prevent successful update of the internal representation. We found that this effect can be quantified by a simple Arrhenius-like law,
rexp(-ΔE(v,d)/kBTd),
(5.1)
where ΔE(v,d) is the energy gap between where the droplet sits in the drive and the escape point, predicted by the equivalence principle, and Td is the disorder-induced temperature. Thus, given a network of N neurons, the probability of an external drive moving the droplet successfully across the network is proportional to exp(-rN). (Note that r depends on N in a way such that exp(-rN) becomes a step function as N: always successful below a critical amount of disorder (capacity) and always failing beyond this capacity.)

The capacity of a neural network to encode multiple memories has been studied in numerous contexts since Hopfield's original work (Hopfield, 1982). While specifics differ (Amit, Gutfreund, & Sompolinsky, 1985a; Battaglia & Treves, 1998; Monasson & Rosay, 2014; Hopfield, 2010), the capacity is generally set by the failure to retrieve a specific memory because of the effective disorder in neural connectivity due to other stored memories.

However, these works on capacity do not account for nonadiabatic external driving. Here, we use our results to determine the capacity of a place cell network (O'Keefe & Dostrovsky, 1971; Battaglia & Treves, 1998; Monasson & Rosay, 2014) to both encode and manipulate memories of multiple spatial environments at a finite velocity. Place cell networks (Tsodyks, 1999; Monasson & Rosay, 2013, 2014, 2015) encode memories of multiple spatial environments as multiple continuous attractors in one network. Such networks have been used to describe recent experiments on place cells and grid cells in the hippocampus (Yoon et al., 2013; Hardcastle et al., 2015; Moser, Moser, & Roudi, 2014).

In experiments that expose a rodent to different spatial environments μ=1,M (Alme et al., 2014; Moser, Moser, & McNaughton, 2017; Moser, Moser, & Roudi, 2014; Kubie & Muller, 1991), the same place cells i=1,,N are seen having “place fields” in different spatial arrangements πμ(i) as seen in Figure 4a, where πμ is a permutation specific to environment μ. Consequently, Hebbian plasticity suggests that each environment μ would induce a set of synaptic connections Jijμ that corresponds to the place field arrangement in that environment: Jijμ=J(1-ε) if |πμ(i)-πμ(j)|<p. That is, each environment corresponds to a 1D network when the neurons are laid out in a specific permutation πμ. The actual network has the sum of all these connections Jij=μ=1MJijμ over the M environments the rodent is exposed to.

Figure 4:

Nonequilibrium capacity of place cell networks limits retrieval of spatial memories at finite velocity. (a) Place cell networks model the storage of multiple spatial memories in parts of the hippocampus by coding multiple continuous attractors in the same set of neurons. Neural connections encoding spatial memory 2, 3, … act like long-range disorder for spatial memory 1. Such disorder, through an increased effective temperature, reduces the probability of tracking a finite velocity driving signal. (b) The probability of successful retrieval, Pretrieval, decreases with the number of simultaneous memories M and velocity v (with N=4000,p=10,ε=0.35,τ=1,J=100,d=10,w=30 held fixed). (c) Pretrieval simulation data collapse when plotted against M/(N/(logN)2) (parameters same as panel b with v=0.8 held fixed and N varies). (d) The nonequilibrium capacity Mc as a function of retrieval velocity v.

Figure 4:

Nonequilibrium capacity of place cell networks limits retrieval of spatial memories at finite velocity. (a) Place cell networks model the storage of multiple spatial memories in parts of the hippocampus by coding multiple continuous attractors in the same set of neurons. Neural connections encoding spatial memory 2, 3, … act like long-range disorder for spatial memory 1. Such disorder, through an increased effective temperature, reduces the probability of tracking a finite velocity driving signal. (b) The probability of successful retrieval, Pretrieval, decreases with the number of simultaneous memories M and velocity v (with N=4000,p=10,ε=0.35,τ=1,J=100,d=10,w=30 held fixed). (c) Pretrieval simulation data collapse when plotted against M/(N/(logN)2) (parameters same as panel b with v=0.8 held fixed and N varies). (d) The nonequilibrium capacity Mc as a function of retrieval velocity v.

Close modal

While Jij above is obtained by summing over M structured environments, from the perspective of, say, Jij1, the remaining Jijμ look like long-range disordered connections. We will assume that the permutations πμ(i) corresponding to different environments are random and uncorrelated, a common modeling choice with experimental support (Hopfield, 2010; Monasson & Rosay, 2014, 2015; Alme et al., 2014; Moser et al., 2017). Without loss of generality, we assume that π0(i)=i (blue environment in Figure 4.) Thus, Jij=Jij0+Jijd,Jijd=μ=1M-1Jijμ. The disordered matrix Jijd then has an effective variance σ2(M-1)/N. Hence, we can apply our previous results to this system. Now consider driving the droplet with velocity v in environment 1 using external currents. The probability of successfully updating the internal representation over a distance L is given by Pretrieval=e-rL/v, where r is given by equation 5.1.

In the thermodynamic limit N, with w,p,L/N held fixed, Pretrieval becomes a Heaviside step function Θ(Mc-M) at some critical value Mc given by
Mc[vΔE(v,d)]2N(logN)2
(6.1)
for the largest number of memories that can be stored and retrieved at velocity v. ΔE(v,d)=(4dw-3γv-2dR)(-vγ+2dR)/4d. Figure 4 shows that our numerics agree well with this formula, showing a novel dependence of the capacity of a neural network on the speed of retrieval and the strength of the external drive. Note that the fact that equation 6.1 scales sublinearly in N reflects our choice of “perfect” retrieval in the definition of successful events. As in earlier works, Hopfield (1982), Hertz, Krogh, Palmer, and Horner (1991), and Amit et al. (1985a, 1985b), the precise definition of capacity can change capacity by log factors.

We have considered continuous attractors in neural networks driven by localized time-dependent currents Icup(n-vt). In recent experiments, such currents can represent landmark-related sensory signals (Hardcastle et al., 2015) when a rodent is traversing a spatial environment at velocity v or signals that update the internal representation of head direction (Seelig & Jayaraman, 2015). Several recent experiments have controlled the effective speed of visual stimuli in virtual reality environments (Meshulam, Gauthier, Brody, Tank, & Bialek, 2017; Aronov, Nevers, & Tank, 2017; Kim et al., 2017; Turner-Evans et al., 2017). Other experiments have probed cross-talk between memories of multiple spatial environments (Alme et al., 2014). Our results predict an error rate that rises with speed and with the number of environments.

While our analysis used specific functional forms for, among others, the current profile Icup(n-vt), our bound simply reflects the finite response time in moving emergent objects, much like moving a magnetic domain in a ferromagnet using space and time-varying fields. Thus, we expect our bound to hold qualitatively for other related forms (Hopfield, 2015).

In addition to positional information considered here, continuous attractors are known to also receive velocity information (Major, Baker, Aksay, Seung, & Tank, 2004; McNaughton et al., 2006; Seelig & Jayaraman, 2015; Ocko et al., 2018). We do not consider such input in the main text but extend our analysis to velocity integration in appendix D.

In summary, we found that the nonequilibrium statistical mechanics of a strongly interacting neural network can be captured by a simple equivalence principle and a disorder-induced temperature for the network's collective coordinate. Consequently, we were able to derive a velocity-dependent bound on the number of simultaneous memories that can be stored and retrieved from a network. We discussed how these results, based on general theoretical principles on driven neural networks, allow us to connect robustly to recent time-resolved experiments in neuroscience (Kim et al., 2017; Turner-Evans et al., 2017; Hardcastle et al., 2015; Hardcastle, Maheswaranathan, Ganguli, & Giocomo, 2017; Campbell et al., 2018) on the response of neural networks to dynamic perturbations.

As in the main text, we model N interacting neurons as
dindt=-inτ+k=1NJnkf(ik)+Inext(t)+ηnint(t),wheref(i)=11+e-i/i0.
(A.1)
The synaptic connection between two different neurons i,j is Jij=J(1-ε) if neurons i and j are separated by a distance of at most p neurons and Jij=-Jε otherwise; note that we set the self-interaction to zero. The internal noise is a white noise, ηnint(t)ηnint(0)=Cintδ(t) with an amplitude Cint. Inext(t) are external driving currents, discussed below.
Such a quasi-1D network with p-nearest neighbor interactions resembles a similarly connected network of Ising spins at fixed magnetization in its behavior; the strength of inhibitory connections ε constrains the total number of neurons 2R firing at any given time to 2Rpε-1. It was shown (Hopfield, 2010; Monasson & Rosay, 2013, 2014) that below a critical temperature T, the w firing neurons condense into a contiguous droplet of neural activity, minimizing the total interface between firing and nonfiring neurons. Such a droplet was shown to behave like an emergent quasi-particle that can diffuse or be driven around the continuous attractor. We define the center of mass of the droplet as
x¯nnf(in).
(A.2)

The description of neural activity in terms of such a collective coordinate x¯ greatly simplifies the problem, reducing the configuration space from the 2N states for the N neurons to N-state, and consists of the center of mass of the droplet along the continuous attractor (Wu et al., 2008). The computational abilities of these place cell networks, such as spatial memory storage, path planning, and pattern recognition, are limited to parameter regimes in which such a collective coordinate approximation holds (e.g., noise levels less than a critical value T<Tc) .

The droplet can be driven by external signals such as sensory or motor input or input from other parts of the brain. We model such external input by the currents Inext in equation A.1—for example, sensory landmark-based input (Hardcastle et al., 2015). When an animal is physically in a region covered by place fields of neurons i,i+1,,i+z, currents Iiext through Ii+zext can be expected to be high compared to all other currents Ijext. Other models of driving in the literature include adding an antisymmetric component Aij to synaptic connectivities Jij (Ponulak & Hopfield, 2013); we consider such a model in appendix D.

Let {ikx¯} denote the current configuration such that the droplet is centered at location x¯. The Lyapunov function of the neural network is given by Hopfield (2015):
L[x¯]L[f(ikx¯)]=1τk0f(ikx¯)f-1(x)dx-12n,kJnkf(ikx¯)f(inx¯)-kf(ikx¯)Ikext(t).
(A.3)
In a minor abuse of terminology, we will refer to terms in the Lyapunov function as energies, even though energy is not conserved in this system. For future reference, we denote the second term VJ(x¯)=-1/2nkJnkf(ikx¯)f(inx¯), which captures the effect of network synaptic connectivities. Under the rigid bump approximation used in Hopfield (2015), that is, ignoring fluctuations of the droplet, we find
VJ(x¯)=-12n,kf(inx¯)Jnkf(ikx¯)
(A.4)
-12|n-x¯|R,|k-x¯|Rf(inx¯)Jnkf(ikx¯).
(A.5)

For a quasi 1D network with p-nearest neighbor interactions and no disorder, VJ(x¯) is constant, giving a smooth, continuous attractor. However, as discussed later, in the presence of disorder, VJ(x¯) has bumps (i.e., quenched disorder) and is no longer a smooth, continuous attractor.

To quantify the effect of the external driving, we write the third term in equation A.3:
Vext(x¯,t)=-kIkext(t)f(ikx¯)
(A.6)
-|k-x¯|<RIkext(t)f(ikx¯).
(A.7)
Thus, the external driving current Inext(t) acts on the droplet through the Lyapunov function Vext(x¯,t). Hence, we define
Fext(x¯,t)=-x¯Vext(x¯,t)
(A.8)
to be the external force acting on the droplet center of mass.

A.1  Fluctuation and Dissipation

We next numerically verify that the droplet obeys a fluctuation-dissipation-like relation by driving the droplet using external currents Iext and comparing the response to diffusion of the droplet in the absence of external currents.

We use a finite ramp as the external driving, Inext=n with n<nmax, and Inext=0 otherwise (see Figure 5a). We choose nmax to be such that it takes considerable time for the droplet to relax to its steady-state position at the end of the ramp. We notice that for different slopes of the Inext, the droplet has different velocities, and it is natural to define a mobility of the droplet, μ, by v=μf, where f is the slope of Inext. Next, we notice that on a single continuous attractor, the droplet can diffuse because of internal noise in the neural network. Therefore, we can infer the diffusion coefficient D of the droplet from x2=2Dt for a collection of diffusive trajectories (see Figure 5b), where we have used x to denote the center of mass x¯ for the droplet to avoid confusion.

Figure 5:

(a) Schematics of the droplet being driven by a linear potential (ramp), illustrating the idea of mobility. Green lines are inputs, red dots are active neurons, and the more transparent ones represent earlier time. (b) Schematics of the droplet diffusing under an input with no gradient, giving rise to diffusion. The inset is the plot of mean-squared distance versus time, clearly showing diffusive behavior. Note here that we have changed the droplet center of mass (c.o.m.) position x¯ as x to avoid confusion with the mean position. (c) Comparison between mobility μ=γ-1 and diffusion coefficient D. Both μ and D depend on blob size and τ in the same way, and thus D is proportional to μ.

Figure 5:

(a) Schematics of the droplet being driven by a linear potential (ramp), illustrating the idea of mobility. Green lines are inputs, red dots are active neurons, and the more transparent ones represent earlier time. (b) Schematics of the droplet diffusing under an input with no gradient, giving rise to diffusion. The inset is the plot of mean-squared distance versus time, clearly showing diffusive behavior. Note here that we have changed the droplet center of mass (c.o.m.) position x¯ as x to avoid confusion with the mean position. (c) Comparison between mobility μ=γ-1 and diffusion coefficient D. Both μ and D depend on blob size and τ in the same way, and thus D is proportional to μ.

Close modal
In Figure 5c, we numerically verify that μ and D depend on parameters τ and R in the same way—that D and μ are both proportional to 1/τ and independent of R. This suggests that Dμ; if we call the proportionality constant to be kBT, then we have a fluctuation-dissipation-like relation:
D=μkBT.
(A.9)

Note that equation A.9 has also been derived for the case of binary neurons with a hard constraint on the number of firing population (Monasson & Rosay, 2014).

We consider the model of sensory input used in the main text: Icup(n)=d(w-|n|),n[-w,w], Icup(n)=0 otherwise. We focus on time-dependent currents Inext(t)=Icup(n-vt). Such a drive was previously considered in Wu and Amari (2005), albeit without time dependence. Throughout this article, we refer to w as the linear size of the drive, d as the depth of the drive, and set the drive moving at a constant velocity v. From now on, we will go to the continuum limit and denote Inext(t)=Iext(n,t)Iext(x,t).

As an example, for v=0 (in this case, Δxv=x¯), we can write down the potential Vext for the external driving signal Icup(x)=d(w-|x|) by evaluating it at a stationary current profile f(ikx¯)=1if|k-x¯|R,=0otherwise,
Vext(x¯)=V1(x¯),|x¯|RV2(x¯),|x¯|>R,
(B.1)
where
V1(x¯)=-d(R-x¯)(w-R-x¯2)+(R+x¯)(w-w+x¯2),V2(x¯)=-d2(R+w-x¯)2.
(B.2)

We plot Vext given by equation B.1 versus the c.o.m. position of droplet in Figure 6a.

Figure 6:

(a) Vext for external driving signal Icup(x,t) with v=0, plotted from equation B.1 with d=20, R=15, w=30. (b) Effective potential Veff experienced by the droplet for a moving cup-shaped external driving signal, plotted from equation C.1 with d=10, R=15, w=30, γv=140. (c) Schematic illustrating the idea of the equivalence principle (see equation 3.2). The difference between the effective potential, Veff-kBTlogp(Δxv), experienced by a moving droplet, and that of a stationary droplet, Vcup, is a linear potential, Vlin=-FvmotionΔxv. The slope θ of the linear potential Vlin=-FvmotionΔxv is proportional to velocity as Fvmotion=γv.

Figure 6:

(a) Vext for external driving signal Icup(x,t) with v=0, plotted from equation B.1 with d=20, R=15, w=30. (b) Effective potential Veff experienced by the droplet for a moving cup-shaped external driving signal, plotted from equation C.1 with d=10, R=15, w=30, γv=140. (c) Schematic illustrating the idea of the equivalence principle (see equation 3.2). The difference between the effective potential, Veff-kBTlogp(Δxv), experienced by a moving droplet, and that of a stationary droplet, Vcup, is a linear potential, Vlin=-FvmotionΔxv. The slope θ of the linear potential Vlin=-FvmotionΔxv is proportional to velocity as Fvmotion=γv.

Close modal

B.1  A Thermal Equivalence Principle

The equivalence principle we introduced in the main text allows us to compute the steady-state position and the effective new potential seen in the co-moving frame. Crucially, the fluctuations of the collective coordinate are described by the potential obtained through the equivalence principle. The principle correctly predicts both the mean (see equation 3.2) and the fluctuation (see equation 3.3) of the lag Δxv. Therefore, it is actually a statement about the equivalence of effective dynamics in the rest frame and in the co-moving frame. Specializing to the drive Icup(x,t), the equivalence principle predicts that the effective potential felt by the droplet (moving at constant velocity v) in the co-moving frame equals the effective potential in the stationary frame shifted by a linear potential, Vlin=-FvmotΔxv, that accounts for the fictitious forces due to the change of coordinates (see Figure 6c).

Since we used equation B.1 for the cup shape and the lag Δxv depends linearly on v, we expect that the slope of the linear potential Vlin also depends linearly on v. Here the sign convention is chosen such that Vlin<0 corresponds to the droplet moving to the right.

In the following, we work in the co-moving frame with velocity v at which the driving signal is moving. We denote the steady-state c.o.m. position in this frame to be Δxv* and a generic position to be Δxv.

When v>0, the droplet will sit at a steady-state position Δxv*<0. The equivalence principle says we should subtract a velocity-dependent linear potential FvmotΔxv=γvΔxv from Vext to account for the motion:
Veff(Δxv)=Vcup(Δxv)-γvΔxv.
(C.1)
We plot Veff vs Δxv in Figure 6b. Notice that there are two extremal points of the potential, corresponding to the steady-state position, Δxv*, and the escape position, Δxvesc:
Δxv*=γv/2d,Δxvesc=(dw-γv+dR)/d.
(C.2)

We are now in position to derive vcrit presented in the main text. We observe that as the driving velocity v increases, Δxv* and Δxvesc will get closer to each other, and there will be a critical velocity such that the two coincide.

By simply equating the expression for xesc and x* and solving for v, we found that
vcrit=2d(w+R)3γ.
(C.3)

C.1  Steady-State Droplet Size

Recall that the Lyapunov function of the neural network is given by equatin A.3:
L[x¯]=1τk0f(ikx¯)f-1(x)dx+VJ(x¯)+Vext(x¯,t).
(C.4)

Compared to the equation of motion (e.o.m.), equaiton A.1, we see that the first term corresponds to the decay of neurons in the absence of interaction from neighbors (decay from the on state to the off state), and the second term corresponds to the interaction Jnk term in the e.o.m, and the third term corresponds to the Inext in the e.o.m. Since we are interested in the steady-state droplet size, and thus only in the neurons that are on, the effect of the first term can be neglected (also note that 1/τJij; when using the Lyapunov function to compute steady-state properties, the first term can be ignored).

To obtain general results, we also account for long-range disordered connections Jijd here. We assume Jijd consists of random connections among all the neurons. We can approximate these random connections as random permutations of Jij0, and the full Jij is the sum over M-1 such permutations plus Jij0.

For the cup-shaped driving and its corresponding effective potential, equation C.1, we are interested in the steady-state droplet size under this driving, so we first evaluate Veff at the steady-state position Δxv* in equation C.2. To make the R-dependence explicit in the Lyapunov function, we evaluate L(x¯) under the rigid bump approximation used in Hopfield (2015), assuming f(ikx¯)=1 for |k-x¯|R, and =0 otherwise.

We find that for M-1 sets of disorder interactions, the Lyapunov function is
L[f(ikx¯)]=J[(εR2-(ε+2p)R+p(p+1)2-pm(2R-p)2]+(γv)24d+Rd(R-2w),
(C.5)
where we have defined the reduced disorder parameter m=(M-1)/N and have used the equivalence principle in equation 3.2 to add an effective linear potential to take into account the motion of the droplet.
Next, we note that the steady-state droplet size corresponds to a local extremum of the Lyapunov function. Extremizing equation C.5 with respect to droplet radius R, we obtain the steady-state droplet radius as a function of the external driving parameters d,w, and the reduced disorder parameter m,
R(d,w,m)=2p-4p2m+2wd/J+ε2d/J-8pm+4ε,
(C.6)
where we observe that in the formula, the only dimensionful parameters d and J appear together to ensure the overall result is dimensionless. Our result for R reduces to R0=p2ε+14 by setting M=1 and d=w=0.

C.2  Upper Limit on External Signal Strength

Here we present the calculation for maximal driving strength Iext beyond which the activity droplet will “teleport”—that is, disappear at the original location and recondense at the location of the drive, even if these two locations are widely separated. We now refer to this maximal signal strength as the teleportation limit. We can determine this limit by finding out the critical point where the energy barrier of breaking up the droplet at the original location is zero.

For simplicity, we assume that initially, the cup-shaped driving signal is some distance x0 from the droplet and not moving (the moving case can be solved in exactly the same way by using the equivalence principle and going to the co-moving frame of the droplet). We consider three scenarios during the teleportation process. (1) In the initial configuration, the droplet has not yet teleported and stays at the original location with radius R(0,0,m). (2) In the intermediate configuration, the activity is no longer contiguous, giving a droplet with radius δ(d,w,m) at the center of the cup, and another droplet with radius R(d,w,m)-δ(d,w,m) at the original location (when teleportation happens, the total firing neurons changes from R(0,0,m) to R(d,w,m)). (3) In the final configuration, the droplet has successfully teleported to the center of the cup, with radius R(d,w,m). The three scenarios are depicted schematically in Figure 7.

Figure 7:

Schematics of three scenarios during a teleportation process. In the initial configuration, the droplet is outside the cup. An energetically unfavorable intermediate configuration is penalized by ΔE: the droplet breaks apart into two droplets—one outside the cup and one inside it. In the final configuration, with the lowest energy, the droplet inside the cup grows to a full droplet while the droplet outside shrinks to zero size. Above each droplet is its corresponding radius R.

Figure 7:

Schematics of three scenarios during a teleportation process. In the initial configuration, the droplet is outside the cup. An energetically unfavorable intermediate configuration is penalized by ΔE: the droplet breaks apart into two droplets—one outside the cup and one inside it. In the final configuration, with the lowest energy, the droplet inside the cup grows to a full droplet while the droplet outside shrinks to zero size. Above each droplet is its corresponding radius R.

Close modal

The global minimum of the Lyapunov function corresponds to scenario 3. However, there is an energy barrier between configuration 1 and configuration 3, corresponding to the Veff difference between configuration 1 and 2. We would like to find the critical split size δc(d,w,m) that maximizes the difference in Veff, which corresponds to the largest energy barrier the network has to overcome in order to teleport from configuration 1 to 3. For the purpose of derivation, in the following we rename L[f(ikm)] in equation C.5 as E0(d,w,m)R(d,w,m) to emphasize its dependence on the external driving parameters and disordered interactions. The subscript 0 stands for the default one-droplet configuration, and it is understood that E0(d,w,m) is evaluated at the network configuration of a single droplet at location m with radius R(d,w,m).

The energy for configuration 1 is simply E0(0,0,m), and the energy for configuration 3 is E0(d,w,m). However, the energy for configuration 2 is not just the sum of E0 from the two droplets. Due to global inhibitions presented in the network, when there are two droplets, there will be an extra interaction term when we evaluate the Lyapunov function with respect to this configuration. The interaction energy between two droplets in Figure 7 is
Eint(m)|R,δ=4JRδ(ε-2pm).
(C.7)
Therefore, the energy barrier for split size δ is
ΔE(d,w,m)δ=E0(0,0,m)R(d,w,m)-δ+E0(d,w,m)δ+Eint(m)R(d,w,m),δ-E0(0,0,m)R(0,0,m).
(C.8)
Therefore, maximizing ΔE with respect to δ, we find
δc=dwd-8Jpm+4Jε.
(C.9)

Now we have obtained the maximum energy barrier during a teleportation process, ΔEδc. A spontaneous teleportation will occur if ΔEδc0, and this in turn gives an upper bound on the external driving signal strength ddmax one can have without any teleportation spontaneous occurring.

We plot the numerical solution of dmax obtained from the solving ΔE(dc,w,m)δc=0, compared with results obtained from the simulation in Figure 8, and find perfect agreement.

Figure 8:

Teleportation depth dmax plotted against disorder parameter m. The dots are data obtained from simulations for different N but with p=10, ε=0.35, τ=1, J=100, and w=30 held fixed. The dotted line is the theoretical curve plotted from solving ΔE(dc,w,m)δc=0 for dc numerically.

Figure 8:

Teleportation depth dmax plotted against disorder parameter m. The dots are data obtained from simulations for different N but with p=10, ε=0.35, τ=1, J=100, and w=30 held fixed. The dotted line is the theoretical curve plotted from solving ΔE(dc,w,m)δc=0 for dc numerically.

Close modal

We also obtain an approximate solution by observing that the only relevant scale for the critical split size δc is the radius of the droplet, R. We set δc=cR for some constant 0c1. In general, c can depend on dimensionless parameters like p and ε. Empirically we found the constant to be about 0.29 in our simulation.

The droplet radius R is a function of d,w,m as we see in equation C.6, but to first-order approximation, we can set R = R* for some steady-state radius R*. Then we can solve
dmax(M)=4J(ε-2pm)w/cR*-1.
(C.10)

Note that the denominator is positive because w>R and 0c1. The simulation result also confirms that the critical split size δc stays approximately constant. We have checked that the dependence on parameters J,w,m in equation C.10 agrees with the numerical solution obtained from solving Ebar(dc,w,m)δc=0, up to the undetermined constant c.

C.3  Speed Limit on External Driving

Recall that given a certain signal strength d, there is an upper bound on how fast the driving can be (see equation C.3). Then in particular, for dmax, we obtain an upper bound on how fast an external signal can drive the network:
vmax=8J(w+R*)(ε-2pm)3γ(w/cR*-1).
(C.11)
For wR*, we can approximate
vmax16JcR*(ε/2-pm)3γ.
(C.12)
In the absence of disorder, m=0, the maximum velocity is bounded by
vmax8c3εJR*γ8c3εJRmaxγ.
(C.13)
Recall that in equation C.10, we have
R(d,wR,0)R(dmax,wR,0)=p2ε+14+2cR*+ORwp2ε+2cRmax,
(C.14)
where in the second line we have used equation C.6 for d=dmax, m=0, and wR. Upon rearranging, we have
Rmax11-2cp2ε.
(C.15)
Plugging in equation C.13, we have
vmax8c3εJRmaxγ83(c-1-2)Jpγ.
(C.16)
Therefore, we have obtained a fundamental limit on how fast the droplet can move under the influence of external signal, namely,
vfund=κJpγ-1,
(C.17)
where κ=8/3(c-1-2) is a dimensionless O(1) number.

Place cell networks (Ocko et al., 2018) and head direction networks (Kim et al., 2017) are known to receive information about both velocity and landmark information. Velocity input can be modeled by adding an antisymmetric part Aij to the connectivity matrix Jij, which effectively tilts the continuous attractor.

Consider now
Jij=Jij0+Jijd+Aij0,
(D.1)
where Aij0=A, if 0<i-jp; -A, if 0<j-ip; and 0 otherwise.

The antisymmetric part Aij0 will provide a velocity v that is proportional to the size A of Aij0 for the droplet (see Figure 9). In the presence of disorder, we can simply go to the co-moving frame of velocity v, and the droplet experiences an extra disorder-induced noise ηA in addition to the disorder-induced temperature Td.

Figure 9:

Velocity of droplet v plotted against the size A of the antisymmetric matrix. We hold all other parameters fixed with the same value as in Figure 8.

Figure 9:

Velocity of droplet v plotted against the size A of the antisymmetric matrix. We hold all other parameters fixed with the same value as in Figure 8.

Close modal

We found that ηA(t)ηA(0)σ˜δ(t) (see Figure 10), where σ˜2 is the average number of disordered connection per neuron in units of 2p.

Figure 10:

Left: At fixed A=5, a collection of 500 diffusive trajectories in the co-moving frame at velocity v, where v is taken to be the average velocity of all the trajectories. We can infer the diffusion coefficient D from the variance of these trajectories as Var(x)=2Dt. Right: logD plotted against logσ˜2. The straight line has slope 1/2, corresponding to Dσ˜.

Figure 10:

Left: At fixed A=5, a collection of 500 diffusive trajectories in the co-moving frame at velocity v, where v is taken to be the average velocity of all the trajectories. We can infer the diffusion coefficient D from the variance of these trajectories as Var(x)=2Dt. Right: logD plotted against logσ˜2. The straight line has slope 1/2, corresponding to Dσ˜.

Close modal

Therefore, all our results in the main text apply to the case when both the external drive Iext(x,t) and the antisymmetric part Aij0 exist. Specifically, we can just replace the velocity v used in the main text as the sum of the two velocities corresponding to Iext(x,t) and Aij0.

E.1  Disordered Connections and Disordered Forces

From now on, we include disorder connections Jijd in addition to ordered connections Jij0 that correspond to the nearest p-neighbor interactions. We assume Jijd consists of random connections among all the neurons. These random connections can be approximated as random permutations of Jij0, such that the full Jij is the sum over M-1 such permutations plus Jij0.

We clip the Jij matrix according to the following rule for each entry when summing over Jij0 and Jijd:
J(1-ε)+J(1-ε)J(1-ε),J(1-ε)+J(-ε)J(1-ε),J(-ε)+J(-ε)J(-ε).
(E.1)
Therefore, adding more disorder connections to Jij amounts to changing the inhibitory -Jε entries to the exitory J(1-ε).
We would like to characterize the effect of disorder on the system. Under the decomposition Jij=Jij0+Jijd, we can define a (quenched) disorder potential that captures all the disorder effects on the network:
Vd(x¯)Vd[f(ikx¯)]=-12nkJnkdf(ikx¯)f(inx¯).
(E.2)
Its corresponding disorder-induced force is then given by
Fd(x¯)=-x¯Vd(x¯).
(E.3)

E.2  Variance of Disorder Forces

We compute the distribution of Vd(x¯) using a combinatorial argument as follows.

Under the rigid droplet approximation, calculating Vd(x¯) amounts to summing all the entries within an R-by-R diagonal block submatrix Jij(x¯) within the full synaptic matrix Jij (recall that Vd(x¯)nkf(in(x¯))Jnkf(ik(x¯))). Each set of disorder connections is a random permutation of Jij0, and thus has the same number of excitatory entries as Jij0, namely, 2pN. Since the inhibitory connections do not play a role in the summation by virtue of equation E.1, it suffices to consider only the effect of adding excitatory connections in Jijd to Jij0.

There are M-1 sets of disordered connections in Jijd, and each has 2pN excitatory connections. Suppose we add these 2pN(M-1) excitatory connections one by one to Jij0. Each time an excitatory entry is added to an entry y in the R-by-R block Jij(x¯), there are two possible situations depending on the value of y before addition: if y=J(1-ε) (excitatory), the addition of an excitatory connection does not change the value of y because of the clipping rule in equation E.1; if y=-Jε (inhibitory), the addition of an excitatory connection to y changes y to J(1-ε). In the latter case, the value of Vd(x¯) is changed because the summation of entries within Jij(x¯) has changed, while in the former case, Vd(x¯) stays the same. (Note that if the excitatory connection is added outside Jij(x¯), it does not change Vd(x¯) and thus can be neglected.)

We have in total 2pN(M-1) excitatory connections to be added, and in total (2R-p)2 potential inhibitory connections in the R-by-R block Jij(x¯) to be flipped to an excitatory connection. We are interested in, after adding all the 2pN(M-1) excitatory connections, how many inhibitory connections are changed to excitatory connections and the corresponding change in Vd(x¯).

We can get an approximate solution if we assume that the probability of flipping an inhibitory connection does not change after the subsequent addition of excitatory connections and stays constant throughout the addition of all the 2pN(M-1) excitatory connections. This requires 2pN(M-1)N2, that is, MN, which is a reasonable assumption since the capacity cannot be O(N).

For a single addition of exitatory connection, the probability of successfully flipping an inhibitory connection within Jij(x¯) is proportional to the fraction of the inhibitory connections within Jij(x¯) over the total number of entires in Jij0:
q(flip)=(2R-p)2N2.
(E.4)
So the probability of getting n inhibitory connections flipped is
P(n)=2pN(M-1)nqn(1-q)2pN(M-1)-n.
(E.5)
In other words, the distribution of flipping n inhibitory connections to excitatory connections after adding Jijd to Jij0 obeys nB(2pN(M-1),q). The mean is then
n=2pN(M-1)q=2p(2R-p)2M-1N=(2R-p)22pm,
(E.6)
where we have defined the reduced disorder parameter m(M-1)/N. The variance is
n2=2pN(M-1)q(1-q)=2pN(M-1)(2R-p)2N21-(2R-p)2N2(2R-p)22pm,
(E.7)
where in the last line we have used N2R-p.
Since changing n inhibitory connections to n excitory connections amounts to changing Vd(x¯) by -1/2(J(1-ε)-J(-ε))=-J/2, we have
Var(Vd(x¯))σ2=J2(R-p/2)2pm.
(E.8)

E.3  Disorder Temperature from Disorder-Induced Force

We focus here on the case where Inext gives rise to a constant velocity v for the droplet (as in the main text). In the co-moving frame, the disorder-induced force Fd(x¯) acts on the c.o.m. like random kicks with correlation within the droplet size. For fast enough velocity, those random kicks are sufficiently decorrelated and become white noise at temperature Td.

To extract this disorder-induced temperature Td, we consider the autocorrelation of Fd[x¯(t)] between two different c.o.m. locations x¯(t) and x¯'(t') (and thus different times t and t'),
C(t,t')Fd[x¯(t)]Fd[x¯(t')],
(E.9)
where the expectation value is averaging over different realizations of the quenched disorder.
Using equation E.3, we have
C(t,t')=x¯Vd(x¯)x¯'Vd(x¯')
(E.10)
=x¯x¯'Vd(x¯)Vd(x¯').
(E.11)
Within time t-t', if the droplet moves a distance less than its size 2R, then Vd computed at t and t' will be correlated because f(ikx¯) and f(ikx'¯) have nonzero overlap. Therefore, we expect that the autocorrelation function Vd(x¯)Vd(x¯') behaves like the 1D Ising model with finite correlation length ξ=2R (up to a prefactor to be fixed later):
Vd(x¯)Vd(x¯')exp-|x¯-x¯'|ξ.
(E.12)
Hence, C(t,t')exp-|x¯-x¯'|ξ. Going to the co-moving frame, we can write the c.o.m. location as before, Δxv=x¯-vt, so the autocorrelation function becomes
C(t,t')exp-|(Δxv+vt)-(Δxv'+vt')|ξ=exp-|v(t-t')+(Δxv-Δxv')|ξexp-v|t-t'|ξ,
(E.13)
where in the last line, we have used that the droplet moves much faster in the stationary frame than the c.o.m. position fluctuates in the co-moving frame, so v(t-t')Δxv-Δxv'.
Now let us define the correlation time to be τcor=ξ/v=2R/v. Then
C(t,t')exp-|t-t'|τcor.
(E.14)
For T|t-t'|τcor, we want to consider the limiting behavior of C(t,t') under an integral. Note that
0Tdt0Tdt'exp-|t-t'|τcor=τcor[2(T-τcor)+2τcore-T/τcor]2τcorT(ifTτcor).
(E.15)
Therefore, we have for Tτcor,
0Tdt0Tdt'exp-|t-t'|τcor=2τcor0Tdt0Tdt'δ(t-t'),
(E.16)
so we can write
exp-|t-t'|τcor2τcorδ(t-t'),
(E.17)
and it is understood that this holds in the integral sense. Therefore, for Tτcor, we expect Fd(x) to act like uncorrelated white noise, and we can write
C(t,t')=Tdδ(t-t')τcorδ(t-t'),
(E.18)
where Td is a measure of this disorder-induced white noise.
To deduce the form of disorder temperature Td, we present the uncollapsed occupancies -logp(Δxv)=V(Δxv)/kBTd (described in the caption of Figure 3) in Figure 11. In a comparison with Figure 3, we can see that Td successfully captures the effect of disorder on the statistics of the emergent droplet if
Td=k˜τcorσ,
(E.19)
where σ is given in equation E.8 and k˜ is a fitting constant.
Figure 11:

Uncollapsed data for the occupancies -logp(Δxv) for different amounts of long-range disordered connections. Parameters are the same as in Figure 3 (see section of F.1 for further details).

Figure 11:

Uncollapsed data for the occupancies -logp(Δxv) for different amounts of long-range disordered connections. Parameters are the same as in Figure 3 (see section of F.1 for further details).

Close modal

In this section, we derive the memory capacity for driven place cell network described equation 6.1.

Our continuous attractor network can be applied to study the place cell network. We assume a 1D physical region of length L. We study a network with N place cell neurons and assume each neuron has a place field of size d=2pL/N that covers the region [0,L] as a regular tiling. The N neurons are assumed to interact as in the leaky integrate-and-fire model of neurons. The external driving currents Iext(x,t) can model sensory input when the mouse is physically in a region covered by place fields of neurons i,i+1,,i+z; currents Iiext through Ii+zext can be expected to be high compared to all other currents Ijext, which corresponds to the cup-shape drive we used throughout the main text.

It has been shown that the collective coordinate in the continuous attractor survives to multiple environments provided the number of stored memories m<mc is below the capacity mc of the network. Below the capacity, the neural activity droplet is multistable; that is, neural activity forms a stable contiguous droplet as seen in the place field arrangement corresponding to any one of the m environments. Note that such a contiguous droplet will not appear contiguous in the place field arrangement of any other environment. Capacity was shown to scale as mc=α(p/N,R)N, where α is an O(1) number that depends on the size of the droplet R and the range of interactions p. However, this capacity is about the intrinsic stability of the droplet and does not consider the effect of rapid driving forces.

When the droplet escapes from the driving signal, it has to overcome a certain energy barrier. This is the difference in Veff between the two extremal points Δxv* and Δxvesc. Therefore, we define the barrier energy to be ΔE=Veff(xvesc)-Veff(Δxv*), and we evaluate it using equations C.1 and C.2:
ΔE(v,d)=(4dw-3γv-2dR)(-γv+2dR)4d.
(F.1)

Note this is the result we used in equation 6.1.

As in the main text, the escape rate r is given by the Arrhenius law:
rexp-ΔE(v,d)kBTd.
(F.2)
The total period of time of an external drive moving the droplet across a distance L (LN, but without loss of generality, we can set L=N) is T=L/v. We can imagine chopping T into infinitesimal intervals Δt s.t. the probability of successfully moving the droplet across L without escaping is
Pretrieval=limΔt0(1-rΔt)TΔt=e-rT=e-rN/v=exp(-Nve-ΔE(v,d)/kBTd).
(F.3)
Td is given by equation E.19:
Td=2k˜RJ(R-p/2)pmvkmv-1,
(F.4)
where in the last step, we have absorbed all the constants (assuming R is constant over different m's) into the definition of k. Now we want to find the scaling behavior of m s.t. In the thermodynamic limit (N), Pretrieval becomes a Heaviside step function Θ(mc-m) at some critical memory mc. With the aid of some hindsight, we try
m=α2(logN)2.
(F.5)
Then in the thermodynamic limit,
limNPretrieval=limNexp(-Nve-logNvΔE(v,d)/αkBk)=limNexp(-NvN-vΔE(v,d)/αkBk)=limNexp(-1vN1-vΔE(v,d)/αkBk)=1,α<vΔE(v,d)/kBk0,α>vΔE(v,d)/kBk.
(F.6)
Therefore, we have arrived at the expression for capacity mc or, in terms of M=mcN+1mcN(N1),
Mc=vΔE(v,d)kBk2N(logN)2
(F.7)
or
McvΔE(v,d)2N(logN)2.
(F.8)

F.1  Numerics of the Place Cell Network Simulations

In this section, we explain our simulations in Fig. 4 in detail.

Recall that we only determine the Arrhenius-like escape rate r up to an overall constant. We can absorb it into the definition of ΔE(v,d) (given by equation F.1) as an additive constant a,
r=exp-ΔE(v,d)+akBkv(M-1)/N.
(F.9)
Then the theoretical curve corresponds to
Pretrieval=e-Nr/v.
(F.10)

Therefore, our model, equation F.10, has three parameters to determine: γ, k, and a. In Figure 12 we determine the parameters by collapsing data and see that the best fit is found provided γ=240.30,k=5255.0kB-1,a=-0.35445. Henceforth, we fix these three parameters to these values.

Figure 12:

Top: Plotting -1/Tdv=log{v-1logr/[ΔE(v,d)+a]} against log(M-1). Different solid lines correspond to data with different v, and the dashed line corresponds to the (M-1)-1/2 curve. Bottom: Plotting v-1logrM-1ΔE(v,d) against v. Different solid lines correspond to data with different M, and the dashed line corresponds to the ΔE(v,d)+a curve.

Figure 12:

Top: Plotting -1/Tdv=log{v-1logr/[ΔE(v,d)+a]} against log(M-1). Different solid lines correspond to data with different v, and the dashed line corresponds to the (M-1)-1/2 curve. Bottom: Plotting v-1logrM-1ΔE(v,d) against v. Different solid lines correspond to data with different M, and the dashed line corresponds to the ΔE(v,d)+a curve.

Close modal

In the bottom plot of Figure 12, we offset the effect of M by multiplying v-1logr by M-1, and we see that curves corresponding to different M collapse to each other, confirming the M-1 dependence in Td. The collapsed line we are left with is just the v-dependence of ΔE(v,d), up to overall constant.

In the top panel of Figure 12, we offset the effect of v in Td by multiplying v-1 to logr/[ΔE(v,d)+a]. We see that different curves corresponding to different v's collapse to each other, confirming the v-1 dependence in Td. The curve we are left with is the M dependence in Td, which we see fits nicely with the predicted M-1.

In Figure 4b, we run our simulation with the following parameters held fixed: N=4000,p=10,ε=0.35,τ=1,J=100,d=10, and w=30. Along the same curve, we vary M from 6 to 30, and the series of curves corresponds to different v from 0.6 to 1.2.

In Figure 4c, we hold the following parameters fixed: p=10,ε=0.35,τ=1,J=100,d=10,w=30, and v=0.8. Along the same curve, we vary M/N(logN)2 from 0.1 to 0.6, and the series of curves corresponds to different N from 1000 to 8000.

In Figures 4b and 4c, the theoretical model we used is equation F.10 with the same parameters given above.

In Figure 4d, we replotted the theory and data from Figure 4b. For the theoretical curve, we find the location where Pretrieval=0.5, and call the corresponding M value “theoretical capacity.” For the simulation curve, we extrapolate to where Pretrieval=0.5, and call the corresponding M value the “simulation capacity.”

For all simulation curves above, we drag the droplet from one end of the continuous attractor to the other end of the attractor and run the simulation 300 times. We then measure the fraction of successful events (defined as the droplet survived in the cup throughout the entire trajectory of moving) and failed events (defined as the droplet escape from the cup at some point before reaching the other end of the continuous attractor). We define the simulation Pretreival as the fraction of successful events.

We thank Jeremy England, Ila Fiete, John Hopfield, and Dmitry Krotov for discussions. A.M. and D.S. are grateful for support from the Simons Foundation MMLS investigator program. We acknowledge the University of Chicago Research Computing Center for support of this work.

Alme
,
C. B.
,
Miao
,
C.
,
Jezek
,
K.
,
Treves
,
A.
,
Moser
,
E. I.
, &
Moser
,
M.-B.
(
2014
).
Place cells in the hippocampus: Eleven maps for eleven rooms
.
Proc. Natl. Acad. Sci. U.S.A.
,
111
(
52
),
18428
18435
.
Amari
,
S.
(
1977
).
Dynamics of pattern formation in lateral-inhibition type neural fields
.
Biol. Cybern.
,
27
(
2
),
77
87
.
Amit
,
D.
,
Gutfreund
,
H.
, &
Sompolinsky
,
H.
(
1985a
).
Storing infinite numbers of patterns in a spin-glass model of neural networks
.
Phys. Rev. Lett.
,
55
(
14
),
1530
1533
.
Amit
,
D. J.
,
Gutfreund
,
H.
, &
Sompolinsky
,
H.
(
1985b
).
Spin-glass models of neural networks
.
Phys. Rev. A
,
32
(
2
),
1007
.
Aronov
,
D.
,
Nevers
,
R.
, &
Tank
,
D. W.
(
2017
).
Mapping of a non-spatial dimension by the hippocampal-entorhinal circuit
.
Nature
,
543
(
7647
),
719
722
.
Battaglia
,
F.
, &
Treves
,
A.
(
1998
).
Attractor neural networks storing multiple space representations: A model for hippocampal place fields
.
Physical Review E
,
58
(
6
),
7738
7753
.
Burak
,
Y.
, &
Fiete
,
I. R.
(
2012
).
Fundamental limits on persistent activity in networks of noisy neurons
.
Proc. Natl. Acad. Sci. U.S.A.
,
109
(
43
),
17645
17650
.
Campbell
,
M. G.
,
Ocko
,
S. A.
,
Mallory
,
C. S.
,
Low
,
I. I.
,
Ganguli
,
S.
, &
Giocomo
,
L. M.
(
2018
).
Principles governing the integration of landmark and self-motion cues in entorhinal cortical codes for navigation
.
Nature Neuroscience
,
21
(
8
),
1096
.
Chaudhuri
,
R.
, &
Fiete
,
I.
(
2016
).
Computational principles of memory
.
Nat. Neurosci.
,
19
(
3
),
394
403
.
Colgin
,
L. L.
,
Leutgeb
,
S.
,
Jezek
,
K.
,
Leutgeb
,
J. K.
,
Moser
,
E. I.
,
McNaughton
,
B. L.
, &
Moser
,
M.-B.
(
2010
).
Attractor-map versus autoassociation based attractor dynamics in the hippocampal network
.
J. Neurophysiol.
,
104
(
1
),
35
50
.
Erdem
,
U. M.
, &
Hasselmo
,
M.
(
2012
).
A goal-directed spatial navigation model using forward trajectory planning based on grid cells
.
Eur. J. Neurosci.
,
35
(
6
),
916
931
.
Evans
,
T.
,
Bicanski
,
A.
,
Bush
,
D.
, &
Burgess
,
N.
(
2016
).
How environment and self-motion combine in neural representations of space
.
J. Physiol.
,
594
(
22
),
6535
6546
.
Fung
,
C. A.
,
Wong
,
K. M.
,
Mao
,
H.
, &
Wu
,
S.
(
2015
).
Fluctuation-response relation unifies dynamical behaviors in neural fields
.
Physical Review E
,
92
(
2
),
022801
.
Fyhn
,
M.
,
Hafting
,
T.
,
Treves
,
A.
,
Moser
,
M.-B.
, &
Moser
,
E. I.
(
2007
).
Hippocampal remapping and grid realignment in entorhinal cortex
.
Nature
,
446
(
7132
),
190
194
.
Hardcastle
,
K.
,
Ganguli
,
S.
, &
Giocomo
,
L. M.
(
2015
).
Environmental boundaries as an error correction mechanism for grid cells
.
Neuron
,
86
(
3
),
827
839
.
Hardcastle
,
K.
,
Maheswaranathan
,
N.
,
Ganguli
,
S.
, &
Giocomo
,
L. M.
(
2017
).
A multiplexed, heterogeneous, and adaptive code for navigation in medial entorhinal cortex
.
Neuron
,
94
(
2
),
375
387
.
Hertz
,
J.
,
Krogh
,
A.
,
Palmer
,
R. G.
, &
Horner
,
H.
(
1991
).
Introduction to the theory of neural computation
.
Physics Today
,
44
,
70
.
Hopfield
,
J. J.
(
1982
).
Neural networks and physical systems with emergent collective computational abilities.
In
PNAS
,
79
,
2554
2558
.
Hopfield
,
J. J.
(
2010
).
Neurodynamics of mental exploration
.
Proceedings of the National Academy of Sciences
,
107
(
4
),
1648
1653
.
Hopfield
,
J. J.
(
2015
).
Understanding emergent dynamics: Using a collective activity coordinate of a neural network to recognize time-varying patterns
.
Neural Comput.
,
27
(
10
),
2011
2038
.
Kilpatrick
,
Z. P.
,
Ermentrout
,
B.
, &
Doiron
,
B.
(
2013
).
Optimizing working memory with heterogeneity of recurrent cortical excitation
.
J. Neurosci.
,
33
(
48
),
18999
19011
.
Kim
,
S. S.
,
Rouault
,
H.
,
Druckmann
,
S.
, &
Jayaraman
,
V.
(
2017
).
Ring attractor dynamics in the drosophila central brain
.
Science
,
356
(
6340
),
849
853
.
Kubie
,
J. L.
, &
Muller
,
R. U.
(
1991
).
Multiple representations in the hippocampus
.
Hippocampus
,
1
(
3
),
240
242
.
Latham
,
P. E.
,
Deneve
,
S.
, &
Pouget
,
A.
(
2003
).
Optimal computation with attractor networks
.
J. Physiol. Paris
,
97
(
4–6
),
683
694
.
Lim
,
S.
, &
Goldman
,
M. S.
(
2012
).
Noise tolerance of attractor and feedforward memory models
.
Neural Comput.
,
24
(
2
),
332
390
.
Major
,
G.
,
Baker
,
R.
,
Aksay
,
E.
,
Seung
,
H. S.
, &
Tank
,
D. W.
(
2004
).
Plasticity and tuning of the time course of analog persistent firing in a neural integrator
.
Proc. Natl. Acad. Sci. U.S.A.
,
101
(
20
),
7745
7750
.
McNaughton
,
B. L.
,
Battaglia
,
F. P.
,
Jensen
,
O.
,
Moser
,
E. I.
, &
Moser
,
M.-B.
(
2006
).
Path integration and the neural basis of the “cognitive map
.”
Nat. Rev. Neurosci.
,
7
(
8
),
663
678
.
Meshulam
,
L.
,
Gauthier
,
J. L.
,
Brody
,
C. D.
,
Tank
,
D. W.
, &
Bialek
,
W.
(
2017
).
Collective behavior of place and non-place neurons in the hippocampal network
.
Neuron
,
96
(
5
),
1178
1191
.
Mi
,
Y.
,
Fung
,
C. A.
,
Wong
,
K. M.
, &
Wu
,
S.
(
2014
).
Spike frequency adaptation implements anticipative tracking in continuous attractor neural networks
. In
Z.
Ghahramani
,
M.
Welling
,
C.
Cortes
,
N. D.
Lawrence
, &
K. Q.
Weinberger
(Eds.),
Advances in neural information processing systems
,
27
(pp.
505
513
).
Red Hook, NY
:
Curran
.
Monasson
,
R.
, &
Rosay
,
S.
(
2013
).
Crosstalk and transitions between multiple spatial maps in an attractor neural network model of the hippocampus: Phase diagram
.
Physical Review E
,
87
(
6
),
062813
.
Monasson
,
R.
, &
Rosay
,
S.
(
2014
).
Crosstalk and transitions between multiple spatial maps in an attractor neural network model of the hippocampus: Collective motion of the activity
.
Physical Review E
,
89
(
3
).
Monasson
,
R.
, &
Rosay
,
S.
(
2015
).
Transitions between spatial attractors in place-cell models
.
Phys. Rev. Lett.
,
115
(
9
),
098101
.
Moser
,
E. I.
,
Moser
,
M.-B.
, &
McNaughton
,
B. L.
(
2017
).
Spatial representation in the hippocampal formation: A history
.
Nat. Neurosci.
,
20
(
11
),
1448
1464
.
Moser
,
E. I.
,
Moser
,
M.-B.
, &
Roudi
,
Y.
(
2014
).
Network mechanisms of grid cells
.
Philos. Trans. R. Soc. Lond. B Biol. Sci.
,
369
(
1635
),
20120511
.
Ocko
,
S. A.
,
Hardcastle
,
K.
,
Giocomo
,
L. M.
, &
Ganguli
,
S.
(
2018
).
Emergent elasticity in the neural code for space
.
Proc. Natl. Acad. Sci. U.S.A.
,
115
(
50
),
E11798
E11806
.
O'Keefe
,
J.
, &
Dostrovsky
,
J.
(
1971
).
The hippocampus as a spatial map: Preliminary evidence from unit activity in the freely-moving rat
.
Brain Res.
,
34
(
1
),
171
175
.
Pfeiffer
,
B. E.
, &
Foster
,
D. J.
(
2013
).
Hippocampal place-cell sequences depict future paths to remembered goals
.
Nature
,
497
(
7447
),
74
79
.
Ponulak
,
F.
, &
Hopfield
,
J. J.
(
2013
).
Rapid, parallel path planning by propagating wavefronts of spiking neural activity
.
Front. Comput. Neurosci.
,
7
.
Poucet
,
B.
, &
Save
,
E.
(
2005
).
Neuroscience: Attractors in memory
.
Science
,
308
(
5723
),
799
800
.
Roudi
,
Y.
, &
Latham
,
P. E.
(
2007
).
A balanced memory network
.
PLOS Comput. Biol.
,
3
(
9
),
1679
1700
.
Seelig
,
J. D.
, &
Jayaraman
,
V.
(
2015
).
Neural dynamics for landmark orientation and angular path integration
.
Nature
,
521
(
7551
),
186
191
.
Seung
,
H. S.
(
1996
).
How the brain keeps the eyes still
.
Proceedings of the National Academy of Sciences
,
93
(
23
),
13339
13344
.
Seung
,
H. S.
(
1998
).
Continuous attractors and oculomotor control
.
Neural Netw.
,
11
(
7–8
),
1253
1258
.
Seung
,
H. S.
,
Lee
,
D. D.
,
Reis
,
B. Y.
, &
Tank
,
D. W.
(
2000
).
Stability of the memory of eye position in a recurrent network of conductance-based model neurons
.
Neuron
,
26
(
1
),
259
271
.
Sontag
,
E. D.
(
2003
).
Adaptation and regulation with signal detection implies internal model
.
Syst. Control Lett.
,
50
(
2
),
119
126
.
Tsodyks
,
M.
(
1999
).
Attractor neural network models of spatial maps in hippocampus
.
Hippocampus
,
9
(
4
),
481
489
.
Turner-Evans
,
D.
,
Wegener
,
S.
,
Rouault
,
H.
,
Franconville
,
R.
,
Wolff
,
T.
,
Seelig
,
J. D.
, …
Jayaraman
,
V.
(
2017
).
Angular velocity integration in a fly heading circuit
.
Elife
,
6
.
Wills
,
T. J.
,
Lever
,
C.
,
Cacucci
,
F.
,
Burgess
,
N.
, &
O'Keefe
,
J.
(
2005
).
Attractor dynamics in the hippocampal representation of the local environment
.
Science
,
308
(
5723
),
873
876
.
Wimmer
,
K.
,
Nykamp
,
D. Q.
,
Constantinidis
,
C.
, &
Compte
,
A.
(
2014
).
Bump attractor dynamics in prefrontal cortex explains behavioral precision in spatial working memory
.
Nat. Neurosci.
,
17
(
3
),
431
439
.
Wu
,
S.
, &
Amari
,
S.-I.
(
2005
).
Computing with continuous attractors: Stability and online aspects
.
Neural Comput.
,
17
(
10
),
2215
2239
.
Wu
,
S.
,
Hamaguchi
,
K.
, &
Amari
,
S.-I.
(
2008
).
Dynamics and computation of continuous attractors
.
Neural Comput.
,
20
(
4
),
994
1025
.
Yoon
,
K.
,
Buice
,
M. A.
,
Barry
,
C.
,
Hayman
,
R.
,
Burgess
,
N.
, &
Fiete
,
I. R.
(
2013
).
Specific evidence of low-dimensional continuous attractor dynamics in grid cells
.
Nat. Neurosci.
,
16
(
8
),
1077
1084
.

Author notes

D.S. and A.M. contributed equally to this work.