The Role of Hierarchical Dynamical Functions in Coding for Episodic Memory and Cognition

The challenge for episodic memory models is understanding how neural circuits could encode and retrieve the complex range of features that a human can recall from episodic memory of individual experiences. A complete model of episodic memory will require understanding of not only the neural mechanisms for encoding and retrieval in regions directly implicated in episodic memory such as the hippocampus and entorhinal cortex but also the interaction of regions coding episodic memory with higher cognitive function throughout distributed regions of the neocortex (Bhandari & Badre, 2018; Hasselmo & Stern, 2018; Lundqvist, Herman, Warden, Brincat, & Miller, 2018).

Episodic memory was first defined by Tulving (1972) to focus on the difference between the encoding and recollection of a specific sensory experience at a specific place and time versus the encoding and retrieval of facts in general knowledge, which was addressed by the term “semantic memory” (Baddeley, 2001; Tulving, 1984, 2001). The term “semantic memory” had been developed by Quillian (Collins & Quillian, 1969) as a description of memory for facts and general knowledge about the world. Episodic memory was later contrasted with other forms of memory such as knowing how to do things, referred to as procedural or implicit memory (Cohen & Eichenbaum, 1993; Cohen & Squire, 1980), and with the active, conscious maintenance of recent information, known as working memory (Baddeley, 2001). Despite being contrasted with semantic memory, a model of episodic memory must ultimately include its interaction with elements of semantic memory. When we remember throwing a ball to a dog, we code many features of the episodic memory in terms of general knowledge of the rules governing the interaction with both objects and agents (Buschman & Miller, 2014; Badre & Frank, 2012). In fact, human episodic memory does not accurately retrieve most details of episodes (Misra, Marconi, Peterson, & Kreiman, 2018), but instead, we reconstruct much of the world from general semantic knowledge.

Here, we address some of the challenges for a complete model of episodic memory within the framework of an earlier model (Hasselmo, 2009, 2012; Hasselmo, Giocomo, Brandon, & Yoshida, 2010). This earlier model addressed how neural mechanisms for coding of grid cells in entorhinal cortex could provide the coding of the spatial location and time of events within an episode, linking individual events and items with neural representations of spatio-temporal trajectories (see Figure 1). This framework describes the neural encoding of all features of episodic memory in terms of multiple spatio-temporal functions corresponding not only to the movement of an agent within a scene but also to the regular features of objects and barriers in the environment as well as the movement of other agents and objects (Hasselmo et al., 2010). A recent extension of this framework proposes that dynamical gating matrices represented and regulated by neural activity (Hasselmo, 2018) can mediate encoding and retrieval of the spatio-temporal functions with the capacity for interpolation or extrapolation of complex functions. This could involve creation of dynamical matrices within structures such as the hippocampus, entorhinal cortex, and retrosplenial cortex.

As an example, Figure 2 shows how sampling of specific points on a trajectory (asterisks) allows computational creation of a dynamical matrix and starting state vector representing properties such as location, velocity, and acceleration. The dynamical matrix and initial state vector can be represented by patterns of neural activity, and Figure 2 shows how a dynamical matrix and state vector can encode and retrieve the spatio-temporal trajectory of an object (Hasselmo, 2018). Retrieval of the full trajectory can be generated by sequential multiplication of the initial state vector by the dynamical matrix (Hasselmo, 2018). More complex trajectories can be encoded by higher level dynamical matrices resetting the component elements of lower level dynamical matrices, to allow transitions between different trajectory segments. This direct influence of higher matrices on the elements of lower level matrices is an important part of the framework. The encoding of spatio-temporal trajectories by dynamical matrices can facilitate the subsequent retrieval of the memories through reactivation of these functions. The coding of multiple spatio-temporal functions moves beyond the coding used in many models in which episodic memories are represented as arrays of feature vectors (Norman & O'Reilly, 2003; Hasselmo & Wyble, 1997; Treves & Rolls, 1994; McNaughton & Morris, 1987).

This framework addresses the problem that a complete theory of episodic memory function must account for the encoding and retrieval of all aspects of mental experience, including not only the overall spatial location and time of an event but also the detailed sensory features of individual objects and agents within that event, including their location and time of appearance, as described previously (Hasselmo et al., 2010), as well as their direction of movement, their speed of movement, the configuration of component features (Hasselmo et al., 2010), and semantic knowledge about these objects and agents. Episodic memory also includes encoding and retrieval of memory for internal thoughts, motivations, and intentions of an individual agent and their conjectures about the thoughts and intentions of other agents. Thus, a complete neural model of episodic memory requires a general neural code for representing all aspects of human cognition that ultimately includes all dimensions of cognition and rule learning that contribute to the experiences that can be retrieved from episodic memory.

Tulving defined a query for episodic memory as “What did you do at time T in place P?” (Tulving, 1984) and also described retrieval as the mental voyage from the current time to the past episode being remembered, but he did not focus on time within the past episode or independent coding of the evolution of different features. Tulving (1984) states that his model “describes a ‘snapshot view’ of episodic memory [that] produces many snapshots whose orderly succession can create the mnemonic illusion of the flow of past time.”

The framework presented here seeks to use neural encoding and retrieval of spatio-temporal trajectories to move beyond the focus on snapshots in Tulving's model. This focus on single snapshots is also characteristic of many neural network models that code the full viewpoint of the world as a static vector at one point in time. In contrast, the framework of episodic memory reviewed here uses models (Hasselmo, 2009, 2012; Hasselmo et al., 2010) that include encoding and retrieval of features such as speed and direction of movement on multiple different time scales. Rather than representing discrete snapshots of time, this framework represents the velocity vectors of individual agents within an episode, so that their position could be computed at different spatial and temporal scales (rather than a single frame rate). The use of velocity vectors allows continuous motion to be interpolated or extrapolated beyond individual snapshots (Figure 2) and can include encoding and retrieval of factors such as head direction and the speed and direction of movements. Consistent with this, other researchers have shown that episodic memories involve encoding and retrieval of a perspective, either consisting of the point of view of the person retrieving the memory (a “field” perspective) or a third-person observer perspective (Conway, 2009; Robinson & Swanson, 1993; Nigro & Neisser, 1983). The description below will show how this property of episodic memory could arise from the functional role of neurons that respond on the basis of current head direction and code the position of barriers or spatial viewpoints.

A complete neural theory of episodic memory must address circuit mechanisms that could encode and retrieve the location and time of events and features on multiple scales (Howard, MacDonald, et al., 2014; Howard, Fotedar, Datey, & Hasselmo, 2004; Howard & Kahana, 2002). Episodic memories can include not only the encoding and retrieval of the place and time of an event (when John gave Mary a book) but also the location and movement of agents and objects within that event (standing by the bookshelf, John gave Mary a book that she set on the table) and the location and movement of features within individual agents and objects (the book was open in John's right hand, and Mary closed it with the index finger of her left hand). This framework contrasts with many existing neural network models of brain function that summarize views of the world as single snapshots that are coded in large-scale vectors and processed in a feedforward manner (LeCun, Bengio, & Hinton, 2015). A complete model will require representing, interpolating, and extrapolating complex relationships between stimulus features in multiple different scales of space and time.

As shown in Figure 1, many episodic memories involve encoding and retrieval of segments of continuous spatio-temporal trajectories. An example spatio-temporal trajectory is shown with lines in the figure. The trajectory includes the agent's sense of relative location but also contains a point of view from a specific direction at each moment, and aspects of the timing of the trajectory, such as the speed of walking, and the relative timing of events at one location. Different positions along this spatio-temporal trajectory are associated with individual events involving interacting with specific people or objects.

As noted in a previous article (Hasselmo et al., 2010), this same framework could be used to encode and retrieve information on different scales for episodic memory. For example, that article notes that the appearance of each individual person or object in the episode would be associated with separate encoding of the locations and spatio-temporal trajectory of the people or objects for possible subsequent retrieval. This encoding can take place on multiple scales for both trajectory encoding and object encoding. Multiple scales of grid cell coding have been shown to be effective for encoding and retrieval of trajectories in episodic memory (Hasselmo, 2009, 2012; Hasselmo et al., 2010) and the planning of future trajectories (Erdem & Hasselmo, 2012, 2014). Encoding can also address different object scales. Instead of encoding and retrieving only the spatio-temporal trajectory of one person, episodic memory could encode and retrieve the position of limbs on different humans or animals and the movement of these limbs to different locations. On an even smaller scale, this framework could be used to encode and retrieve the position and movement of fingers within a hand or the position and movement of eyelids and lips within a face. The accurate encoding of spatio-temporal trajectories could potentially provide a general framework for encoding and retrieval of both episodic and semantic memory on multiple different scales.

The coding of spatio-temporal trajectories was previously described in terms of coding of speed and direction at each position along a trajectory (Hasselmo, 2009, 2012; Hasselmo et al., 2010). However, extension of this work models how trajectories can also be coded in terms of complex functions such as linear dynamical systems that include both velocity and acceleration (Hasselmo, 2018) or representations such as Bezier functions, which would allow a potentially more concise representation of significant segments of a trajectory (Hasselmo, 2018). For example, as shown in Figure 2, the trajectory of a projectile such as a ball thrown through the air can be encoded and retrieved as a small dynamical matrix and initial state vector representing the full trajectory of the dynamical system rather than coding individual snapshots of the position of the ball along the trajectory.

As described previously (Hasselmo, 2018), the dynamical matrix could consist of patterns of activity in another population that interacts in a multiplicative manner on the dendritic tree. This dynamical system provides a highly accurate method for retrieval (gray dots in Figure 2) to interpolate between remembered positions or extrapolate beyond the last remembered position by replicating the application of the dynamical relationship to compute different points along the trajectory. This framework for coding of relations by complex functions could be used to code the complex relations necessary for performance of complex cognitive tasks such as Raven's progressive matrices (Hasselmo, 2018), which represents a test of general intelligence (Barrett, Hill, Santoro, Morcos, & Lillicrap, 2018; Raudies & Hasselmo, 2017; Rasmussen & Eliasmith, 2011; Carpenter, Just, & Shell, 1990).

The same technique for coding the transformation between adjacent points on a trajectory could be used for coding the uniform transformation of many points in space, for example, because of rotation of an object or an agent. If this transformation is coded as a pattern of gating activity in the network (Hasselmo, 2018), then the activity pattern replicating the transformation could then be replicated to generalize the transformation across different features. In this framework, relatively small transformation matrices could be used to compute the viewpoint of the world at any position along a trajectory. This would provide a much more efficient mechanism for encoding and retrieval of episodic memory (Hasselmo, 2018). Rather than storing a snapshot of every spatial view of the world for an agent moving around a room for several hours, instead the system can store the allocentric location and direction of the features of the room, and encode and retrieve the allocentric location and direction of a person's trajectory, and the egocentric angle of different visual features can be extracted and retrieved from the transformation of location and direction anywhere along the trajectory, as shown in Figure 3. This framework is similar to the index theory of hippocampal function presented previously (Teyler & Rudy, 2007; Teyler & DiScenna, 1986) but provides an explicit mathematical representation of how episodic memory can link to a concise semantic memory representation of functions that allows interpolation and extrapolation of more complex details of a memory in neocortical circuits.

Neuropsychological and fMRI studies demonstrate an important role for the hippocampus and associated structures in episodic memory. Bilateral removal of anterior hippocampus, entorhinal cortex, and parahippocampal gyrus in patient H. M. caused striking deficits for episodic memory tests (Corkin, 1984; Scoville & Milner, 1957) but sparing of working memory (Scoville & Milner, 1957). Neurophysiological data described below demonstrate that the patterns of neuronal activity in the hippocampus and entorhinal cortex are associated with encoding and retrieval of episodic memories in human participants (Rutishauser, Schuman, & Mamelak, 2008; Kreiman, 2007; Kreiman, Koch, & Fried, 2000a) and with retrieval or planning of spatio-temporal trajectories during behavioral tasks in rodents (Pfeiffer & Foster, 2013; Johnson & Redish, 2007).

Episodic memory impairments can also be caused by damage to subcortical regions including the anterior thalamus and mammillary bodies (Vann & Aggleton, 2004) and by damage to the medial septum and medial pFC caused by aneurysms of the anterior communicating artery (DeLuca, 1993; DeLuca & Cicerone, 1991). Subcortical structures contain neurons coding aspects of behavior such as head direction in the anterior thalamus (Stackman & Taube, 1998; Taube, 1995) and movement direction in the medial septum (Welday, Shlifer, Bloom, Zhang, & Blair, 2011). The medial septum and anterior thalamus also influence theta rhythm dynamics in cortical circuits that could be essential to a temporal code for episodic memory (Buzsáki, 2002; Bland & Colom, 1993; Stewart & Fox, 1990; Buzsáki, Leung, & Vanderwolf, 1983).

Studies in animals demonstrate the role of the hippocampus and associated cortical regions and connections via the fornix with the medial septum and anterior thalamus. Nonhuman primates show permanent impairments of memory for trial unique objects in delayed nonmatch to sample tasks after hippocampal lesions (Zola-Morgan & Squire, 1986), and combined hippocampal, perirhinal, and parahippocampal lesions (Meunier, Bachevalier, Mishkin, & Murray, 1993; Zola-Morgan, Squire, Clower, & Rempel, 1993) and transient impairments after entorhinal lesions (Leonard, Amaral, Squire, & Zola-Morgan, 1995; Gaffan & Murray, 1992). Lesions of the fornix, which disrupt medial septum input and anterior thalamic output for the hippocampus, impair the construction of a snapshot memory for spatial location of visual features (Gaffan & Harrison, 1989) or associations with responses (Gaffan et al., 1984).

Similarly, studies in rodents showed impairments in delayed alternation caused by hippocampal lesions (Ainge, van der Meer, Langston, & Wood, 2007) or lesions of the fornix or the septum (Ennaceur, Neave, & Aggleton, 1996; Aggleton, Neave, Nagle, & Hunt, 1995; Freeman & Stanton, 1991; Stanton, Thomas, & Brito, 1984). An increase in the number of repeat visits to arms in the eight-arm radial maze occurs with hippocampal lesions (Becker, Walker, & Olton, 1980) and fornix lesions (Hudon, Doré, & Goulet, 2002; Olton, Becker, & Handelmann, 1979) and by lesions (Mitchell, Rawlins, Steward, & Olton, 1982) or inactivation of the medial septum (Chrobak, Stackman, & Walsh, 1989).

Hippocampal lesions impair finding of a single fixed platform location in the Morris water maze (Morris, Anderson, Lynch, & Baudry, 1986) including in a variant relevant to episodic memory requiring the rat to retrieve a new platform location on each day (Steele & Morris, 1999). Impairments in the Morris water maze also appear with lesions of the dorsal entorhinal cortex (Steffenach, Witter, Moser, & Moser, 2005), dorsal presubiculum (postsubiculum; Taube, Kesslak, & Cotman, 1992), and fornix (Eichenbaum, Stewart, & Morris, 1990) and also with pharmacological inactivation (Brioni, Decker, Gamboa, Izquierdo, & McGaugh, 1990) or lesions of the medial septum (Marston, Everitt, & Robbins, 1993) that will reduce theta rhythm. Performance in the Morris water maze is also impaired after lesions of the retrosplenial cortex (Czajkowski et al., 2014) and parietal lesions (Hoh, Kolb, Eppel, Vanderwolf, & Cain, 2003).

The involvement of medial temporal lobe structures in encoding and retrieval of episodic memories is consistent with fMRI data from human participants. Human participants show substantial fMRI activation associated with the encoding of new information into episodic memory (Kirchhoff, Wagner, Maril, & Stern, 2000; Wagner et al., 1998; Stern et al., 1996). Consistent with coding of a spatio-temporal trajectory, fMRI data demonstrated hippocampal and parahippocampal activity at the start of a trajectory when a participant retrieves a previously learned overlapping trajectory (Brown & Stern, 2014; Brown, Ross, Keller, Hasselmo, & Stern, 2010). Subsequent work showed that retrieval of a planned trajectory involves reactivation of intermediate locations on the trajectory (Brown et al., 2016). Studies show coding of navigationally relevant information in fMRI activity, with hippocampus coding path length and entorhinal cortex coding the Euclidean distance to the goal (Chrastil, Sherrill, Hasselmo, & Stern, 2015; Howard, Javadi, et al., 2014). Evidence also indicates that the coding of the trajectory and features along the trajectory involve an interaction of regions coding spatial location and direction (Chrastil, Sherrill, Hasselmo, & Stern, 2016), with regions coding visual cues such as optic flow and feature angle (Chrastil et al., 2015; Sherrill et al., 2015). These fMRI data support the role of large-scale neural activity in these regions for the encoding and retrieval of spatio-temporal trajectories.

As noted above, research has not converged on a model for the neural coding of the broad range of events in an episodic memory. The nature of coding depends on the details of neurophysiological interactions within cortical circuits. Here, we will review evidence for functional coding by neurons in the structures implicated in episodic memory, with a focus on the open questions about coding of episodes. Electrophysiological recordings in the human hippocampus, amygdala, and entorhinal cortex demonstrate firing properties consistent with their role in the encoding and retrieval of episodic memories. Unit recordings in these structures show neural responses to highly specific categories of visual stimuli such as faces or animals (Rutishauser et al., 2011; Kreiman, Koch, & Fried, 2000b) and reveal neurons that fire during encoding of stimuli as well as during retrieval by mental imagery (Kreiman et al., 2000a). Neurons in these structures show highly detailed coding of individual identities of famous actors or famous landmarks (Quiroga, Reddy, Kreiman, Koch, & Fried, 2005) and highly selective firing during retrieval as tested by self-report of mental imagery (Ison, Quian Quiroga, & Fried, 2015) and during recognition or recall of stimuli (Rutishauser et al., 2008). Thus, extensive data show clear responses to nonspatial stimuli in humans. In rodents, neurons show a variety of responses to the spatial and temporal dimensions of behavioral tasks but also show responses to nonspatial features such as objects or odors in the environment (Wood, Dudchenko, & Eichenbaum, 1999). Neurons also show responses suggestive of the retrieval of prior experience (Ferbinteanu & Shapiro, 2003) or the planning of future experience (Johnson & Redish, 2007). The following sections provide more details on these properties.

The effects of fornix lesions and medial septum lesions described above highlight the potential role of theta rhythm in coding of episodic memory. These same lesions of medial septum or fornix reduce theta power in the hippocampus (Rawlins, Feldon, & Gray, 1979) and entorhinal cortex (Mitchell et al., 1982). Theta rhythm is a prominent band of rhythmic activity in the range of 6–10 Hz that appears in structures including the hippocampus, entorhinal cortex, and retrosplenial cortex during movement in a range of different species (Buzsáki, 2002; Buzsáki et al., 1983) and shows important correlations with running speed. Theta rhythm is also associated with gamma frequency oscillations (Colgin et al., 2009; Csicsvari, Hirase, Czurkó, Mamiya, & Buzsáki, 1999; Bragin et al., 1995).

Evidence suggests that the theta rhythm could play an essential role in a temporal code for episodic and semantic memory. There have been general discussions about whether the coding of sensory input involves a code based on mean firing rate versus a temporal code based on the timing of individual spikes. The phenomenon of spiking relative to theta phase provides some of the strongest evidence for a temporal code. The field will benefit from more explicit testing of detailed codes that can address the full complexity of cognitive representations retrieved during episodic memory function.

Lesions of the medial septum cause impairments of memory encoding performance that correlate with the change in theta rhythm (Givens & Olton, 1990, 1994; Winson, 1978), and the successful learning of memory traces is accompanied by increases in theta oscillatory power (Griffin, Asaka, Darling, & Berry, 2004; Seager, Johnson, Chabot, Asaka, & Berry, 2002; Berry & Thompson, 1978). Activation of both GABAergic and cholinergic subpopulations in the medial septum enhances temporal precision of hippocampal spiking activity (Dannenberg et al., 2015), and combined lesions of both GABAergic and cholinergic subpopulations in the medial septum severely impair memory performance (Pang, Nocera, Secor, & Yoder, 2001). Activation of glutamatergic neurons in the medial septum enhances theta rhythmicity in the hippocampus via activation of cholinergic and GABergic neurons in the medial septum (Robinson et al., 2016). Data from EEG studies indicate that theta oscillations recorded in the medial temporal lobe of human epileptic patients have been shown to increase during memory encoding and during navigation through real-world and virtual environments (Aghajan et al., 2017; Bohbot, Copara, Gotman, & Ekstrom, 2017; Ekstrom et al., 2005; Kahana, Sekuler, Caplan, Kirschen, & Madsen, 1999). Some cells in human hippocampus do not show strong theta rhythmicity in their autocorrelograms (Viskontas, Ekstrom, Wilson, & Fried, 2007), but human hippocampal neurons do show spiking that is specific to the phase of theta rhythm oscillations, and this phase specificity during stimuli predicts the subsequent ability to successfully retrieve those stimuli (Rutishauser, Ross, Mamelak, & Schuman, 2010).

Oscillatory dynamics at theta or gamma frequencies could be essential to the neural coding of episodic information. For example, as described further below, individual neurons in the hippocampus code sequences of positions along a trajectory based on the phase of firing relative to theta rhythm oscillations (O'Keefe & Recce, 1993). This phase code for position has also been demonstrated in the entorhinal cortex (Hafting, Fyhn, Bonnevie, Moser, & Moser, 2008) and the lateral septum, a hippocampal output structure where neurons do not exhibit spatial receptive fields but instead code position solely via phase (Tingley & Buzsáki, 2018). In addition, phase coding has also been shown for the duration of time within an interval (Pastalkova, Itskov, Amarasingham, & Buzsáki, 2008) and might also exist for coding of the position of an object in a sequence or other dimensions of experience. Thus, theta rhythm might provide a substrate for a phase code of different dimensions of memory and behavior.

Phase coding might also exist for other frequencies. For example, different populations of neurons show spiking at specific phases relative to gamma frequency oscillations (Senior, Huxter, Allen, O'Neill, & Csicsvari, 2008). The duration of each gamma cycle may be too short for coding of spiking on different phases within a gamma cycle, but a binary code could be maintained by different neurons firing at either the peak or trough of individual gamma cycles. For example, firing on the peak could represent a “1” in a binary code, whereas firing on the trough could represent a “0” in a binary code. The updating of this binary code would require relatively complex patterns of synaptic connectivity to ensure that the binary code for individual neuron phase would be updated based on the update of other neuron binary codes.

Alternately, theta rhythm might play a role for the functional dynamics of cortical circuits, setting differences in dynamics for the encoding of new information on one phase of the oscillation, versus the retrieval of information at a different phase of the oscillation (Hasselmo, Bodelón, & Wyble, 2002). This is supported by empirical data on the timing of synaptic modification at different phases of theta rhythm (Hyman, Wyble, Goyal, Rossi, & Hasselmo, 2003; Huerta & Lisman, 1995), data on the effect of phase-specific manipulations that influence encoding or retrieval (Siegle & Wilson, 2014; Douchamps, Jeewajee, Blundell, Burgess, & Lever, 2013), and data on the coordination of activity in different brain regions relative to hippocampal theta (Colgin et al., 2009; Jones & Wilson, 2005; Hyman, Wyble, Rossi, & Hasselmo, 2002).

Other frequencies could be involved in similar regulation of functional gating. Neocortical regions show bursts of activity at beta and gamma frequencies associated with gating of information into working memory or top–down regulation of the flow of information into working memory (Lundqvist et al., 2016, 2018; Buschman, Denovellis, Diogo, Bullock, & Miller, 2012). Simulations have shown how bursts of gamma frequency activity could be involved in maintaining information in working memory (Lundqvist, Herman, & Lansner, 2011; Lundqvist, Compte, & Lansner, 2010; Fransén & Lansner, 1998) and how the network resonance of cortical regions at gamma and beta frequencies could be used to gate the flow of information between different cortical regions (Sherfey, Ardid, Hass, Hasselmo, & Kopell, 2018). Simulations need to explore the broad range of functional episodic memory mechanisms that could involve network oscillatory dynamics.

By Tulving's initial definition, episodic memory requires a code for the spatial location of a memory. As noted above, this could include the overall location of an agent during an event but could also include the location of individual agents and objects within an event or even the location of parts of objects such as a person's hand (Hasselmo et al., 2010). Encoding and retrieval of the spatio-temporal trajectory of an agent could involve neurons in the hippocampus termed “place cells,” which show responses dependent on the current, spatial location of a rodent (O'Keefe, 1976; O'Keefe & Dostrovsky, 1971). Virtual navigation studies show selective responses to virtual location by hippocampal neurons in humans (Ekstrom et al., 2003).

Neuronal activity of place cells does not only represent the coding of current position but also demonstrates retrieval or planning of segments of spatio-temporal trajectories. Early studies of place cells focused on a rate code for spatial location (O'Keefe, 1976; O'Keefe & Dostrovsky, 1971), but later studies revealed a striking temporal code in which the finer-grained position of an animal relative to the position along a trajectory through a specific firing field is reflected in a shift from late to early phase of firing of place cells relative to hippocampal theta rhythm oscillations (Skaggs, McNaughton, Wilson, & Barnes, 1996; O'Keefe & Recce, 1993). On a population level, it appears that place cells fire in sequential order within a theta rhythm based on their relative coding of position (Feng, Silva, & Foster, 2015; Foster & Wilson, 2007). In addition, when animals are making a decision about future movement based on memory of the task, experimental data show place cell sequences that appear to be associated with the retrieval of prior trajectories or planning of future trajectories by the animal (Ólafsdóttir, Barry, Saleem, Hassabis, & Spiers, 2015; Pfeiffer & Foster, 2013; Johnson & Redish, 2007). Sequential firing of place cells within a theta cycle has been found to reflect future trajectories to a goal location providing one possible neural mechanism for prospective planning (Wikenheiser & Redish, 2015).

These studies provide the evidence of specific retrieval activity that could be linked to episodic memory. Related to this, neurons have also been shown to fire selectively for specific left or right trajectories in continuous spatial alternation tasks (Frank, Brown, & Wilson, 2000; Wood, Dudchenko, Robitsek, & Eichenbaum, 2000) and in a delayed nonmatch to position task (Griffin, Eichenbaum, & Hasselmo, 2007), suggesting that they may depend on the retrieval of prior trajectories or planning of future trajectories (Hasselmo & Eichenbaum, 2005). Recordings on a plus-shaped maze with different future or past arms show that some neurons selectively respond based on the prior segment of the trajectory, whereas others show selectivity for the future segment of a trajectory (Ferbinteanu & Shapiro, 2003). Surprisingly, neurons did not show trajectory specificity in the delayed version of the spatial alternation task possibly because of difficulty in determining the time of retrieval (Ainge et al., 2007).

Thus, evidence supports the encoding and retrieval of trajectories through of a series of spatial locations by the firing rate and firing phase of place cells. However, the mechanism by which the coding of place arises is still uncertain. Neurons show changes in response to the running speed and direction (Terrazas et al., 2005; Gothard, Skaggs, Moore, & McNaughton, 1996; Muller & Kubie, 1987; McNaughton, Barnes, & O'Keefe, 1983), and models have proposed that place cells could arise from path integration in other regions that drive the hippocampal place representation (McNaughton, Battaglia, Jensen, Moser, & Moser, 2006; Redish & Touretzky, 1998; Redish & Touretzky, 1997; Samsonovich & McNaughton, 1997; Touretzky & Redish, 1996). However, path integration suffers from accumulation of error. As an alternative, other models propose that place cells could be driven by the learned response to visual features in particular configurations (Bicanski & Burgess, 2018; Byrne, Becker, & Burgess, 2007; Hetherington & Shapiro, 1993), based on a transformation of egocentric feature angle to allocentric position based on the current heading direction of the animal (Byrne et al., 2007).

The coding of spatial location along a spatio-temporal trajectory has been proposed to involve an interaction of both place cells and grid cells (Hasselmo, 2009, 2012; Hasselmo et al., 2010). Place cells in the hippocampus primarily code individual locations in the environment, whereas grid cells in the entorhinal cortex have been shown to fire when a foraging animal visits an array of spatial locations in the environment (Hafting, Fyhn, Molden, Moser, & Moser, 2005). The firing fields occur in a regular array of locations that can be described as falling on the vertices of tessellated equilateral triangles or in a hexagonal pattern. Different grid cells show different spatial scales (Stensola et al., 2012; Barry, Hayman, Burgess, & Jeffery, 2007; Sargolini et al., 2006; Hafting et al., 2005), allowing a population of grid cells to code individual spatial locations (Mathis, Stemmler, & Herz, 2015; Sreenivasan & Fiete, 2011; Fiete, Burak, & Brookings, 2008; Gorchetchnikov & Grossberg, 2007). Models also indicate how grid cell firing can be used to explore and find possible relational trajectories between locations (Erdem & Hasselmo, 2012, 2014) or to directly compute the relationship between different spatial locations (Bush, Barry, Manson, & Burgess, 2015). Many grid cells conjunctively code both location and current head direction of the animal (Sargolini et al., 2006).

Models of grid cell location implicitly focus on different coding principles. For example, a firing rate code for grid cells is the emphasis of models using attractor dynamics to generate grid cell firing patterns within a population of neurons with feedback inhibition (Widloski & Fiete, 2014; Yoon et al., 2013; Giocomo, Moser, & Moser, 2011; Burak & Fiete, 2009; Fuhs & Touretzky, 2006; McNaughton et al., 2006; Conklin & Eliasmith, 2005). These models are supported by the distinct grid cell modules that share properties of spacing and orientation (Stensola et al., 2012; Barry et al., 2007) and the change in correlation with different anatomical distances between grid cells (Heys, Rangarajan, & Dombeck, 2014). These models use path integration that could accumulate errors but could be reset by sensory cues (Hardcastle, Ganguli, & Giocomo, 2015; Bush & Burgess, 2014; Touretzky & Redish, 1996).

Alternately, a temporal code model of grid cells codes location based on firing updated by shifts in frequency of input theta rhythm oscillations induced by velocity (Bush & Burgess, 2014; Hasselmo, 2008, 2014; Giocomo et al., 2011; Burgess, 2008; Burgess, Barry, & O'Keefe, 2007). These models directly replicate data on the theta phase precession firing of grid cells relative to theta rhythm oscillations (Hafting et al., 2008). Consistent with this model based on theta rhythm, experiments show a loss of grid cell spatial selectivity during loss of theta rhythm caused by inactivation of medial septum (Brandon et al., 2011; Koenig, Linder, Leutgeb, & Leutgeb, 2011), which spares the coding of head direction (Brandon et al., 2011) and place cell firing in the hippocampus (Brandon, Koenig, Leutgeb, & Leutgeb, 2014). The loss of grid cell selectivity appears to specifically depend on inactivation of GABAergic neurons in the medial septum (Robinson & Brandon, 2018). Inactivation of cholinergic neurons does not appear to alter theta rhythm or speed coding in a manner that would alter grid cells (Dannenberg, Kelley, Hoyland, Monaghan, & Hasselmo, 2019).

The role of oscillatory dynamics is further supported by changes in intrinsic rhythmicity of entorhinal neurons with spatial scale (Jeewajee, Barry, O'Keefe, & Burgess, 2008) and shifts in running speed (Hinman, Brandon, Climer, Chapman, & Hasselmo, 2016; Jeewajee et al., 2008). The influence of internal noise can be reduced by redundant coding in a population of neurons (Zilli & Hasselmo, 2010) or combining interference with attractor dynamics (Bush & Burgess, 2014; Hasselmo & Shay, 2014). The influence of external noise can be reduced by resetting location based on the angle of sensory cues (Bush & Burgess, 2014), consistent with more accurate coding near environmental boundaries than at a distance from boundaries (Hardcastle et al., 2015). The role of sensory input angle is supported by the loss of spatial coding during inactivation of regions providing head direction input (Winter, Clark, & Taube, 2015).

Another model of grid cells involves the self-organization of input from place cells (Si, Kropff, & Treves, 2012; Kropff & Treves, 2008) with spike-frequency accommodation of entorhinal neurons resulting in a time-varying response to input from place cells that self-organize into the pattern of grid cell responses. These models (Dordek, Soudry, Meir, & Derdikman, 2016; Krupic, Bauza, Burton, Lever, & O'Keefe, 2014) are supported by evidence showing that development of place cells appears earlier than grid cells (Wills, Barry, & Cacucci, 2012; Wills, Cacucci, Burgess, & O'Keefe, 2010) and that place cell inactivation reduces grid cell firing (Bonnevie et al., 2013), whereas grid cell inactivation does not prevent place cell firing (Brandon et al., 2014), although changes in grid cell scale do influence place cell scale (Mallory, Hardcastle, Bant, & Giocomo, 2018).

As noted above, the coding of episodic memory also requires the coding of the specific time of events on multiple different scales, ranging from the time of one episode versus another episode, to the timing of different events within an episode, to the timing of specific movements during a particular event. The neural coding of time could involve the patterns of activity shown by populations of neurons referred to as time cells. This refers to neurons that respond at specific time intervals during behavioral tasks. These cells have been described as episode cells (Pastalkova et al., 2008) or time cells (MacDonald, Lepage, Eden, & Eichenbaum, 2011). They have been shown to fire during the delay period between the encounter of a cue and a delayed response (MacDonald, Carrow, Place, & Eichenbaum, 2013; MacDonald et al., 2011). These types of responses have also been shown in spatial alternation tasks, in which neurons will fire at specific intervals during running on a running wheel (Pastalkova et al., 2008) or during running on a treadmill (Kraus et al., 2015; Kraus, Robinson, White, Eichenbaum, & Hasselmo, 2013).

The coding of time in episodic memory has been proposed to occur on multiple different temporal scales (Howard, MacDonald, et al., 2014; Howard & Kahana, 2002). Consistent with this, calcium imaging data on multiple time cells in the hippocampus show that these neurons not only respond at specific time intervals within individual trials (Mau et al., 2018) but that these neurons also show changes in their response across trials within a day, and the neurons also change their population response across days (Mau et al., 2018). This provides strong empirical evidence for spatio-temporal coding across multiple scales. It is important to note that the neurons that code specific time intervals also often code specific spatial locations as place cells or grid cells (Kraus et al., 2013, 2015). Thus, the same populations of neurons can simultaneously code space and time along a spatio-temporal trajectory. The models of grid cell firing in the entorhinal cortex can account for the fact that many grid cells and place cells will also respond at specific time intervals during running (Hasselmo, 2008; Burgess et al., 2007). Consistent with the fact described above that grid cells are dependent on oscillatory input, the inactivation of medial septum also prevents the temporal specificity of firing by time cells (Wang, Romani, Lustig, Leonardo, & Pastalkova, 2015). Alternately, both abstract models (Howard, MacDonald, et al., 2014; Howard & Kahana, 2002) and biological detailed models (Liu, Tiganj, Hasselmo, & Howard, 2019; Tiganj, Hasselmo, & Howard, 2015) show how the coding of time can arise from exponential decay of spiking rate, consistent with data from slice preparations (Tahvildari, Fransén, Alonso, & Hasselmo, 2007) and in vivo recording (Tsao et al., 2018).

The memory of a spatio-temporal trajectory of an agent not only requires its location over time in the environment but also includes encoding and retrieval of its heading within the environment. In fact, the spatio-temporal trajectory includes not only the direction of overall body movement but also the moment-to-moment direction of the animal's head (Raudies, Brandon, Chapman, & Hasselmo, 2015). Consistent with this, many neurons coding the head direction of rodents were discovered, initially in the dorsal presubiculum by Ranck and Taube (Taube, Muller, & Ranck, 1990) and later in a range of structures including the anterior thalamus (Taube, 1995), the lateral mammillary nucleus (Stackman & Taube, 1998), the entorhinal cortex (Giocomo et al., 2014; Brandon, Bogaard, Schultheiss, & Hasselmo, 2013; Sargolini et al., 2006), and the retrosplenial cortex (Cho & Sharp, 2001; Chen, Lin, Green, Barnes, & McNaughton, 1994). Head direction cells respond selectively based on the current allocentric direction of the animal's head, regardless of the location of the animal in the environment and independent of the relative position of individual landmarks or features. During sleep, head direction cells appear to replay the sequence of positions encountered during behavior (Peyrache, Lacroix, Petersen, & Buzsáki, 2015). An angular velocity signal based on self-motion appears to be integrated to drive head direction neurons in structures such as the anterior thalamus, lateral mammillary nucleus, and dorsal tegmental nucleus (Clark & Taube, 2012; Taube & Bassett, 2003). As noted above, episodic memory includes specific viewpoints of an event, and head direction cells could provide an important means of remembering specific viewpoints. In addition, evidence indicates that the allocentric spatial code could arise from a transformation from egocentric viewpoints, which could use an interaction of head direction cells with egocentric viewpoint cells to create an allocentric coordinate frame (Bicanski & Burgess, 2018; Byrne et al., 2007).

The coding of direction is standard in robotics that use coding of the “pose” of the agent, which includes both location and heading direction (Milford, Wiles, & Wyeth, 2010; Milford & Wyeth, 2010). Vision-based robotics includes coding of barriers and objects and the transformation from the coordinates relative to the heading of the agent (egocentric coordinates) into coordinates of the absolute position of the agent relative to the overall environment (allocentric coordinates; Milford et al., 2010; Milford & Wyeth, 2010; Byrne et al., 2007).

The head direction signal is not equivalent to a movement direction signal, and analysis of periods where head direction does not match movement direction shows that these cells primarily code head direction (Raudies et al., 2015), although they can code angular head motion (Clark & Taube, 2011). This supports the notion that place codes are driven more strongly by the angle of visual features dependent on head direction, rather than path integration.

As noted above and in a previous article (Hasselmo et al., 2010), the coding of episodic memory not only requires coding of the spatio-temporal trajectories of a single person but also would require the coding of the relative spatio-temporal trajectory of objects and barriers within the environment. This could include tracking of the position of environment boundaries relative to the agent. For example, your memory of walking through an apartment would include spatial viewpoints of the position of different walls of the rooms and hallways as you follow a trajectory through the environment, as shown in Figure 3.

A range of data has demonstrated the influence of environmental boundaries on coding by populations of neurons. Early data showed that place cell firing depends on the relative position of the walls of the environment (O'Keefe & Burgess, 1996), leading to the novel theoretical proposal that a specific class of neurons might code the position of an animal relative to boundaries (Hartley, Lever, Burgess, & O'Keefe, 2014; Savelli, Yoganarasimha, & Knierim, 2008; Burgess, Jackson, Hartley, & O'Keefe, 2000). This modeling prediction was later supported by data showing the existence of boundary cells that respond at a specific distance and angle from boundaries in allocentric coordinates (Lever, Burton, Jeewajee, O'Keefe, & Burgess, 2009; Savelli et al., 2008; Solstad, Boccara, Kropff, Moser, & Moser, 2008). Modeling proposed that egocentric visual input about boundaries combined with head direction cells could be transformed to code allocentric spatial location (Bicanski & Burgess, 2018; Byrne et al., 2007). The egocentric coding of boundary position has been demonstrated in recent data, which shows neurons with selectivity for the distance and angle of boundaries in egocentric coordinates (Hinman, Chapman, & Hasselmo, 2017, 2019; Carstensen, Alexander, Hinman, & Hasselmo, 2018), suggesting that coding of boundaries in egocentric coordinate frames could contribute to the coding of these boundaries in allocentric coordinate frames.

As mentioned above, the coding of an episodic memory can involve nonspatial features, such as the identity of specific objects (Kreiman et al., 2000a). A number of studies have focused on the potential representation of objects in the hippocampus and entorhinal cortex. Recordings in the hippocampus and related cortical structures in humans show neurons with many types of nonspatial object selectivity (Rutishauser et al., 2011; Kreiman, 2007; Quiroga et al., 2005; Kreiman et al., 2000b). Similarly, many neurons in rodent hippocampus also show selective responses to nonspatial features such as specific odors or rewards (Wood et al., 1999; Otto & Eichenbaum, 1992; Wiener, Paul, & Eichenbaum, 1989). Recordings in the rodent lateral entorhinal cortex show that some neurons respond to specific objects (Keene et al., 2016; Deshmukh & Knierim, 2011) or odors (Young, Otto, Fox, & Eichenbaum, 1997), although these neurons appear less frequently than spatially coding neurons in the medial entorhinal cortex. Recordings also show object responses in the rodent perirhinal cortex (Deshmukh, Johnson, & Knierim, 2012). Responses to specific objects have been shown in the perirhinal cortex and entorhinal cortex of monkeys (Suzuki, Miller, & Desimone, 1997; Riches, Wilson, & Brown, 1991; Wilson, Riches, & Brown, 1990), along with evidence that these responses decrease or increase with repeated presentations of objects (Suzuki et al., 1997; Riches et al., 1991), consistent with models of familiarity recognition in which the responses to familiar objects increase or decrease (Bogacz & Brown, 2003; Sohal & Hasselmo, 2000).

A full episodic memory also requires encoding and retrieval of the location and movement of objects in addition to the interaction of agents in an episode. Neurons in the hippocampus respond to a particular orientation and distance from objects (Deshmukh & Knierim, 2013), and entorhinal neurons respond to landmark position as well as path integration (Campbell et al., 2018). Coding of objects also occurs in egocentric coordinates (Wang et al., 2018), as well as coding of the previous location of an object as trace cells in the hippocampus (O'Keefe & Nadel, 1978) and entorhinal cortex (Tsao, Moser, & Moser, 2013). Of particular interest with regard to shared mechanisms for coding agents and objects, there is new evidence that neurons that respond to the current location of a rat also respond to the location of another rat in the same environment (Danjo, Toyoizumi, & Fujisawa, 2018), which includes showing theta phase coding of the position of the other rat within a firing field. These data provide important evidence of the relational coding in hippocampus of object location in egocentric coordinates relative to the agent. These mechanisms for relational coding could relate to the general coding of objects and features in a broad range of neocortical structures. There might be a general mechanism for the coding of cognitive representations within cortical regions that could be applied across sensory modalities, as supported by evidence that cortical regions can take over the function of other damaged cortical regions (Abel, Weiller, Huber, Willmes, & Specht, 2015; Roe, Pallas, Kwon, & Sur, 1992; Roe, Pallas, Hahm, & Sur, 1990).

In contrast to a snapshot view of episodic memory, the spatio-temporal trajectory view of episodic memory would code functions to allow a smooth continuous reconstruction of the elements of a trajectory. This requires coding of not only position at different time points but also the velocity and possibly even the acceleration along the trajectory. This would allow more accurate interpolation or extrapolation of the trajectory on any desired scale (Hasselmo, 2018). Consistent with this framework, neurons respond not only to the position and time during events but also to dynamical variables such as running speed. The coding of running speed has been shown in a number of different functional classes of neurons in the entorhinal cortex (Hinman, Dannenberg, Alexander, & Hasselmo, 2018; Hinman et al., 2016; Kropff, Carmichael, Moser, & Moser, 2015; Wills et al., 2012) and hippocampus (McNaughton et al., 1983).

Cells throughout the hippocampal formation spike theta rhythmically, and in addition to the firing rate speed signal in MEC, the coding of speed by the frequency of rhythmic firing has been shown experimentally in single cells in medial entorhinal cortex (MEC) (Hinman et al., 2016; Jeewajee et al., 2008). The rhythmic frequency of single cell firing usually increases as a rat runs faster, but there are also cases where the rhythmic frequency decreases as the rat runs faster (Dannenberg et al., 2019). Given that both speed signals have been identified in MEC, we investigated whether the same cells express both speed signals similarly and found that the two signals are actually independently expressed in single cells (Hinman et al., 2016). This suggests that both rate coding and temporal coding might occur for running speed. The change in coding of running speed over different time scales also suggests that neurons might be coding acceleration (Dannenberg et al., 2019). Acceleration has been used to enhance the encoding and retrieval of trajectories using dynamical matrices (Hasselmo, 2018). Another potential implication is that temporal coding of running speed not only may involve rhythmic frequency but might actually involve a phase code for running speed in terms of the timing of spikes relative to theta phase or even gamma phase for different running speeds.

The coding of spatio-temporal trajectories requires that different trajectories overlapping in the same location in the environment can be disambiguated. This is consistent with neurophysiological data showing that the coding of trajectories is highly context dependent. Early data showed that place cells in one environment will often turn off in a different environment and be replaced by other place cells that were inactive in the first environment (Lever, Wills, Cacucci, Burgess, & O'Keefe, 2002; Muller & Kubie, 1987). Even within an environment, neurons will show strongly context-dependent activity. For example, when a rat runs on the stem of spatial alternation task, individual neurons will fire selectively based on the past or future turning response in the hippocampus (Ferbinteanu & Shapiro, 2003; Wood et al., 2000) and entorhinal cortex (Frank et al., 2000). Neurons also code other features of the environment such as the identity of odors or the presence of reward (Wood et al., 1999; Eichenbaum, Kuperstein, Fagan, & Nagode, 1987) and can show highly selective firing based on the influence of a spatial context on the reward valence of individual objects (McKenzie et al., 2016; Komorowski, Manns, & Eichenbaum, 2009). In some cases, the change in coding may reflect a rotation of firing fields relative to the environment (Kinsky, Sullivan, Mau, Hasselmo, & Eichenbaum, 2018). In addition, recent work has revealed that spatially specific but nongrid cells of the MEC globally remapped in response to contextual changes to an environment (Diehl, Hon, Leutgeb, & Leutgeb, 2017). Grid cells, on the other hand, have been found to exhibit either nodal rate alterations or translational shifts of the full map in response to contextual alterations to the environment (Diehl et al., 2017; Marozzi, Ginzberg, Alenda, & Jeffery, 2015). There is some evidence that contextually sensitive or nonsensitive MEC neurons actually form molecularly and anatomically distinct subpopulations within the region (Kitamura et al., 2014, 2015; Ray et al., 2014). These data are consistent with episodic coding of spatio-temporal trajectories, rather than coding only in a static spatial map.

As described here, episodic coding of an event within an episode and the more general coding of objects for cognitive processing can and should involve multiple different spatio-temporal trajectories in different coordinate systems. This is consistent with evidence from the retrosplenial cortex, which provides an important interface between sensory systems such as the visual and somatosensory system with memory-related structures such as the entorhinal cortex and hippocampus. In particular, electrophysiological data from the retrosplenial cortex demonstrate the coding of multiple different dimensions that include allocentric coding of location in space, egocentric coding of turning directions on a maze, and pathway-centric coding of position relative to the start and end of a particular path (Alexander & Nitz, 2015, 2017). This coding of multiple different coordinate systems is important for the robust and flexible coding of the environment both for coordinate transformations necessary when planning different behaviors and for the robust coding of episodic memory of a trajectory.

To account for the multiple complex representations in episodic memory and the transformations between different coordinate systems, we need more sophisticated neural representations of these types of competing production systems that can neurally represent higher order relations within sensory input as complex functions based on dynamical matrices, which could also act as transformation matrices between different coordinate frames. The coding of complex relations can be useful both for modeling of cognitive processing (thought) and for coding the full richness of episodic memory and cognition.

Although we have reviewed much exciting data on the functional coding relevant to episodic memory in the hippocampus, entorhinal cortex, and related cortical structures, future progress in this field requires more extensive exploration of the full space of possible neural models. The space of possible neural models of cognition is enormous, and only a small portion of this space has been explored in research. There are many dimensions of neural function that may be relevant to cognition. Dimensions that have already been explored extensively in current models include the modifiable strength (or weight) of excitatory and inhibitory synaptic connections between neurons (Bliss & Collingridge, 1993), the magnitude of tonic depolarization of populations of neurons, and the threshold for firing of individual neurons. Synaptic dimensions of neural function that have not been explored as extensively as synaptic weight and bias include the nonlinear interactions of synapses on the dendritic tree. This could include nonlinear interactions of synapses because of N-methyl-D-aspartate receptors (Poirazi, Brannon, & Mel, 2003; Mel, 1993) or internal chemical waves (Bhalla, 2017, 2019), which have been used in sigma-Pi networks (Durbin & Rumelhart, 1989). These multiplicative interactions could play a role in gating the flow of information between populations of cortical neurons (Hasselmo & Stern, 2018). Gating of information flow can be supplemented by temporally coordinated firing of inhibitory interneurons providing cell-type- and cell-compartment-specific inhibition. Two examples of such microcircuits are axoaxonic inhibitory interneurons (Klausberger & Somogyi, 2008) targeting the axon initial segment of pyramidal neurons and the interneuron-specific vasoactive intestinal peptide-positive interneurons inhibiting parvalbumin-positive and somatostatin-positive neurons to cause disinhibition of excitatory principal neurons (Fu et al., 2014; Pi et al., 2013).

Another area that has not been explored are the intrinsic dynamics of neurons that include the spike frequency accomodation (or adaptation) caused by calcium-activated potassium currents (Barkai & Hasselmo, 1994; Connors, Gutnick, & Prince, 1982), the resonance and rebound spiking caused by hyperpolarization activated cation currents (Ferrante, Shay, Tsuno, Chapman, & Hasselmo, 2017; Klink & Alonso, 1993), and the persistent spiking caused by calcium-sensitive nonspecific cation currents (Fransén, Tahvildari, Egorov, Hasselmo, & Alonso, 2006; Fransén, Egorov, Hasselmo, & Alonso, 2003; Egorov, Hamam, Fransén, Hasselmo, & Alonso, 2002). These dimensions of function have been effectively simulated in reduced form in single neuron models (Izhikevich, 2003), and biophysical models have explored the memory function of channels such as adapation currents (Barkai, Bergman, Horwitz, & Hasselmo, 1994) and persistent spiking currents (Fransén, Alonso, & Hasselmo, 2002). However, implications for network dynamical function have not been fully explored.

An important step in the exploration of model space would be to systematize what we do and do not know. For example, it could be productive to characterize the functional attributes of known neural circuit models. The systemization of the properties of atomic elements provided an important framework for chemistry (Mendeleev, 1869), and mathematical systematization guided research in particle physics (Gell-Mann, 1964). As potential examples, the functional dynamics of second-order differential equations have been characterized along the dimensions of the trace and determinant of the Jacobian matrix (see page 96 of Hirsch & Smale, 1974). There have been efforts to provide a systematic categorization of an existing set of neural network architectures and their functional properties (Van Veen, 2016), but the field might benefit from a more systematic approach to understanding the large space of neural models that have not yet been explored.

This article reviews a theory of episodic memory and cognitive function in which spatio-temporal trajectories are coded on multiple scales of place and time. As proposed previously (Hasselmo et al., 2010), this coding can address detailed scales of the spatio-temporal trajectories of individual features of objects (a finger or an eyelid) as well as other dimensions such as the color or size of individual features. The previous model was extended by the proposal of dynamical matrices that can code trajectories and boundaries (Hasselmo, 2018), allowing interpolation and extrapolation of not only predicted place and time but also velocity (direction and speed) and acceleration. In a hierarchical manner, this framework can also represent the spatio-temporal trajectory of overall objects, barriers, and agents observed by an agent within an event (in egocentric or allocentric coordinates) as well as the spatio-temporal trajectory and features such as egocentric direction of the individual observing the episode.

This work was supported by the National Institutes of Health, Grant Numbers R01 MH60013 and R01 MH61492, and by the Office of Naval Research MURI N00014-16-1-2832 and F32 NS101836.

Reprint requests should be sent to Michael E. Hasselmo, Center for Systems Neuroscience, Boston University, 610 Commonwealth Ave., Boston, MA 02215, or via e-mail: hasselmo@bu.edu.