Episodic memories are constructed from sequences of events. When recalling such a memory, we not only recall individual events, but we also retrieve information about how the sequence of events unfolded. Here, we focus on the role of the hippocampal–entorhinal region in processing and remembering sequences of events, which are thought to be stored in relational networks. We summarize evidence that temporal relations are a central organizational principle for memories in the hippocampus. Importantly, we incorporate novel insights from recent studies about the role of the adjacent entorhinal cortex in sequence memory. In rodents, the lateral entorhinal subregion carries temporal information during ongoing behavior. The human homologue is recruited during memory recall where its representations reflect the temporal relationships between events encountered in a sequence. We further introduce the idea that the hippocampal–entorhinal region might enable temporal scaling of sequence representations. Flexible changes of sequence progression speed could underlie the traversal of episodic memories and mental simulations at different paces. In conclusion, we describe how the entorhinal cortex and hippocampus contribute to remembering event sequences—a core component of episodic memory.
Episodic memories are classically thought to comprise information about specific events as well as about where and when these events occurred (Tulving, 1972, 1983). Recalling such an episodic memory is characterized by a vivid feeling of recollection—famously referred to as “mental time travel” (Tulving, 1983, 2002). However, episodes do not occur in isolation but rather in a sequence. Think back to an event that happened earlier today. For example, you might remember meeting your colleague upon arriving at work this morning: You had just locked your bike and entered the building before exchanging a few words with her. Then, you climbed the stairs to the third floor and made your way to your office. The events that precede and follow an event are important for defining its temporal context. Sequence information such as relative times of occurrence can support memory for when events took place (Friedman, 1993). This can happen in concert with distance information like the decaying strengths of memory traces or differences in the accessibility of items that were stored in memory at different times. Furthermore, location information in the form of temporal tags or general contextual associations can be encoded with the event and underlie memory for when events occurred (Friedman, 1993). Here, we aim to review the role of the hippocampus and entorhinal cortex in the formation and retrieval of sequence memories.
The hippocampal–entorhinal region, situated in the medial temporal lobe of the human brain, is thought to be central for episodic memory. Theoretical accounts suggest that it forms networks of related experiences, for example, the different elements of an event sequence (Eichenbaum, Dudchenko, Wood, Shapiro, & Tanila, 1999; Eichenbaum & Cohen, 1988). Different events occurring in temporal proximity can be linked together in relational networks (Eichenbaum et al., 1999; Eichenbaum & Cohen, 1988). Associating event details with contextual information—constituted, for example, by the time and place at which an event takes place—has been suggested as a key contribution of the hippocampus to episodic memory (Diana, Yonelinas, & Ranganath, 2007; Eichenbaum, Yonelinas, & Ranganath, 2007). Other accounts have proposed the role of the hippocampus to lie in the generation of content-free sequences (Buzsáki & Tingley, 2018; Friston & Buzsáki, 2016). A representation of an event sequence might then emerge from associations of these sequential states with particular content in neocortical regions (Friston & Buzsáki, 2016).
How does the brain keep track of how events unfold over time to allow remembrance of event sequences? In keeping with its key role in episodic memory and sequence processing in general, we will center this review on the human hippocampal formation. Specifically, we will focus on recent neuroimaging work investigating the hippocampus and entorhinal cortex. We further incorporate insights from intracranial recordings in patients as well as theoretical work and findings from animal models of temporal processing. Where a comprehensive description of the different aspects of temporal coding and memory is beyond the scope of this article, we refer the reader to insightful reviews.
We begin by summarizing findings that implicate the hippocampus in the formation and retrieval of sequence memories. We argue that correlations of activity patterns reflect sequence information. Subsequently, we will briefly review recent studies suggesting that sequence representations in the hippocampal formation include information about the precise duration of the temporal structure of the sequence. Next, we shift our focus to the entorhinal cortex as a major cortical input to the hippocampus. Importantly, we highlight recent evidence suggesting that the anterior–lateral subregion of the entorhinal cortex is particularly relevant for temporal processing. We thus emphasize the role of the entorhinal cortex—in concert with the hippocampus—in sequence memory. Furthermore, we introduce the idea of temporal scaling, which has been described for sensory and motor timing and the corresponding cortical networks. Specifically, we suggest that the flexible adaptation of sequence progression speed, in episodic memory and mental simulation, is supported by the hippocampal–entorhinal region. Finally, we present outstanding questions for future research that emerge from these findings and ideas.
SEQUENCE MEMORY AND THE HIPPOCAMPUS
The role of the hippocampus in the formation and retrieval of memories for temporal and sequence information is well established (for reviews, see, e.g., Ranganath & Hsieh, 2016; Eichenbaum, 2014). Here, we primarily focus on work in humans, although the fundamental role of the hippocampus in sequence memory is also well documented in animals (Eichenbaum, 2017; Fortin, Agster, & Eichenbaum, 2002; Kesner, Gilbert, & Barua, 2002). In studies using fMRI, hippocampal activations have been observed when participants learned stimulus sequences (Ross, Brown, & Stern, 2009; Kumaran & Maguire, 2006a) or completed serial RT tasks (Schendan, Searl, Melrose, & Stern, 2003). Likewise, hippocampal activity is increased for temporal information that is later successfully recalled from memory (Tubridy & Davachi, 2011; Jenkins & Ranganath, 2010). One such study analyzed activity during the delay period of a serial order working memory task as a function of later memory for when individual stimuli were encountered. After the presentation of the stimulus sequence, greater hippocampal activity related to more accurate subsequent memory for the temporal position of stimuli (Jenkins & Ranganath, 2010). Consistently, hippocampal activity during encoding predicted serial recall for items encountered at event boundaries, defined by stimulus category switches, suggesting a role for the hippocampus in linking different event sequences (DuBrow & Davachi, 2016).
These reports of increased hippocampal activity during the encoding of temporal information that was later remembered are complemented by studies of brain activity during the retrieval of sequence order. For example, when participants recalled the order of scenes from a movie, the hippocampus was more strongly engaged than in a baseline condition or when participants logically inferred the order of scene stimuli (Lehn et al., 2009). Likewise, hippocampal involvement in retrieving the order of events during navigation (Lieberman, Kyle, Schedlbauer, Stokes, & Ekstrom, 2017; Ekstrom, Copara, Isham, Wang, & Yonelinas, 2011; Ekstrom & Bookheimer, 2007) or retrieving the next element to complete a learned sequence (Ross et al., 2009) has also been observed. Collectively, these findings suggest that both successful encoding and retrieval of event sequences recruit the hippocampus.
PATTERN CORRELATIONS REFLECT SEQUENCE INFORMATION
One way by which the hippocampus could support memory for temporal relationships is by segregating representations of events that were originally separated in time. For example, the activity of neuronal ensembles in the medial temporal lobe changes over time, resulting in similar population signals for nearby points in time and more dissimilar neural patterns at increased time lags (Folkerts, Rutishauser, & Howard, 2018; Howard, Viskontas, Shankar, & Fried, 2012). Slowly drifting signals might arise from decaying traces of prior experiences or because of changing internal states. In models of episodic memory, these signals serve as contextual representations that allow the tagging of individual memories (Howard, Fotedar, Datey, & Hasselmo, 2005; Howard & Kahana, 2002). The current state of these contextual representations could be incorporated into the mnemonic representation of an event occurring at a certain moment, putatively facilitating memory for how a sequence of events unfolded over time. Consistent with the notion that changing neural patterns relate to sequence memory, hippocampal activity patterns during the encoding of object sequences were less similar, when the order of object pairs was remembered correctly in a later memory test (Jenkins & Ranganath, 2016). Furthermore, increased correlations of hippocampal multivoxel patterns between stimuli were related to remembering these stimuli as being relatively close in time, compared to stimuli separated by the same temporal distance that were judged to be far apart (Ezzyat & Davachi, 2014; Figure 1A). However, when participants use associative strategies to encode stimulus sequences, increased similarity of hippocampal representations has been reported for items whose order was later discriminated correctly (DuBrow & Davachi, 2014, 2017). This points toward an influence of encoding strategies on the way hippocampal activity relates to memory for order. Together, these data and theoretical considerations suggest that hippocampal activity patterns at encoding support memory by providing a temporal context representation allowing the encoding of sequence relationships.
Interestingly, hippocampal activity patterns also carry information about temporal context and the temporal relations of events during retrieval. Paralleling behavioral contiguity effects during the recall of word lists, reinstatement of medial temporal lobe activity patterns observed during encoding accompanies the successful recall of information from memory (Folkerts et al., 2018; Yaffe et al., 2014; Howard et al., 2012; Manning, Polyn, Baltuch, Litt, & Kahana, 2011). In these studies, neural activity patterns during item recall correlated with those observed during encoding, not only of the recalled item but also of those preceding and following it during learning (Folkerts et al., 2018; Yaffe et al., 2014; Howard et al., 2012; Manning et al., 2011).
The analysis of fMRI multivoxel patterns further suggests that hippocampal activity in response to retrieval cues carries information about the temporal relationships of memories. In one study, participants navigated a virtual city along a fixed route and learned when they encountered certain objects (Deuker, Bellmund, Navarro Schröder, & Doeller, 2016; Figure 1B). In a postlearning fMRI scan, they were presented with images of these objects in random order. Pattern similarity in the anterior hippocampus correlated negatively with the temporal distances by which participants remembered the objects to be separated. Relative to a prelearning baseline scan, the representations of pairs of objects that were remembered as being close in time became more similar, whereas objects separated by large temporal distances became more dissimilar (Deuker et al., 2016). Consistent results were observed in a study in which participants judged temporal relations of location triads, visited in a delivery task (Kyle, Smuda, Hassan, & Ekstrom, 2015). Furthermore, the relationship between multivoxel pattern similarity in the anterior hippocampus and the temporal distances of events was also demonstrated in a study in which participants viewed photographs of real-life events, which were taken with life-logging devices over the course of a month. This provides encouraging evidence suggesting that the anterior hippocampus represents temporal relations also between naturalistic events encountered outside the controlled setting of laboratory studies (Nielson, Smith, Sreekumar, Dennis, & Sederberg, 2015). Furthermore, hippocampal activity patterns have been shown to reflect conjunctions of item information and sequence position (Hsieh, Gruber, Jenkins, & Ranganath, 2014). Collectively, these studies suggest that hippocampal activity patterns during retrieval carry information about the relative sequence position at which different events were encountered.
One possibility for how such effects could arise is that related events are reactivated when recalling a memory. In line with this, during recency judgments, lower classifier evidence for the visual category of two items has been reported, if these items were separated by items from a different compared to the same category in the encoding sequence (DuBrow & Davachi, 2014). Classifier evidence was correlated with multivoxel pattern similarity during retrieval and encoding of intervening items from the same category. Together with behavioral priming effects, this has been interpreted as reinstatement of associative links between items during memory recall of temporal relations (DuBrow & Davachi, 2014). Associative retrieval of related memories might explain why hippocampal representations at retrieval reflect temporal relationships.
Despite our lives progressing continuously, we typically remember segregated episodes from our experience. The Event Horizon Model describes how ongoing experience is segmented into sequences of discrete events (Radvansky & Zacks, 2014). Currently active event models influence mnemonic processing and are stored in long-term memory when event boundaries are encountered (Zacks, 2020; Radvansky & Zacks, 2014). Brain areas at different levels of the cortical hierarchy process event boundaries at varying timescales (Baldassano et al., 2017). Evidence suggests that event boundaries exert a strong influence on sequence memory, including memory for the order of events, as well as estimates of temporal distances and duration (Bangert, Kurby, Hughes, & Carrasco, 2019; Faber & Gennari, 2015, 2017; Ezzyat & Davachi, 2014; for a review, see Clewett, DuBrow, & Davachi, 2019). In a study where participants watched a movie consisting of two interleaved narratives, reflecting alternative storylines involving the same characters and locations, hippocampal representations differentiated the two event sequences. Specifically, hippocampal multivoxel patterns diverged gradually as the narratives unfolded (Milivojevic, Varadinov, Grabovetsky, Collin, & Doeller, 2016). Notably, regions of the default network can integrate narrative information on timescales of around 30 sec, even when the hippocampus is severely damaged (Zuo et al., 2020). However, successful memory encoding and retrieval of event sequences might require interactions with the hippocampus. Consistent with the notion that the hippocampus supports the grouping of related events from a sequence, data from statistical learning paradigms demonstrate that multivoxel pattern representations of events likely to occur close in time become more similar after repeated exposure (Schapiro, Turk-Browne, Norman, & Botvinick, 2016; Schapiro, Kustner, & Turk-Browne, 2012). Activity profiles of the dentate gyrus are sensitive to repetitions of the same spatio-temporal sequence, in line with a role of the hippocampus in discriminating different sequences (Azab, Stark, & Stark, 2014). Together, these data implicate the hippocampus in both relating elements of a sequence to each other and distinguishing different sequences separated by event boundaries.
INCORPORATING DURATION IN HIPPOCAMPAL SEQUENCE PROCESSING
The above findings establish a central role for the hippocampus in tracking sequences of events for episodic memory. However, the fidelity with which event sequences are represented in the hippocampus is less clear. Behavioral work suggests that the durations of sequence elements can be accurately remembered for events that can be recollected (Brunec, Ozubko, Barense, & Moscovitch, 2017). Under most circumstances, events separated by a longer temporal interval are also separated by a larger number of intervening events. This makes it difficult to study whether and how information about the relative duration of events or the precise intervals separating sequence elements influences hippocampal sequence processing. Nevertheless, recent evidence suggests that the hippocampus processes time intervals and that it potentially incorporates duration information in sequence representations.
The discovery of sequentially active cells in rodents is central to the notion that the hippocampus forms precise representations of intervals. These cells were described in animals running in place throughout the delay periods of working memory tasks (MacDonald, Lepage, Eden, & Eichenbaum, 2011; Pastalkova, Itskov, Amarasingham, & Buzsáki, 2008). Over the course of the delay, these cells fire in a fixed sequence. In different repetitions of the interval, the same cells become active at approximately the same points in time, with respect to the beginning of the delay. Hence, these cells are often referred to as time cells (Eichenbaum, 2014; MacDonald et al., 2011). Building on the fine-grained information these cell assemblies carry about elapsed time, time cells have been suggested as a potential mechanism to incorporate temporal information in episodic memory (for review, see Eichenbaum, 2014, 2017). Currently, this is a difficult idea to test as conscious recollection of past experiences is extremely difficult, if not impossible, to assess in animals. Theoretical accounts have linked properties of time cells to a scale-invariant compression of memory for time, which could allow them to serve as a mechanism for temporal coding at different timescales (Liu, Tiganj, Hasselmo, & Howard, 2019; Howard, 2018). Thus, time cells could equip hippocampal sequence representations with duration information.
In line with the idea that the hippocampus enables duration encoding, recent studies of amnesic patients with damage to the medial temporal lobe suggest that the hippocampus contributes to duration estimates (Palombo et al., 2020; Palombo & Verfaellie, 2017; Palombo, Keane, & Verfaellie, 2016). In one such study, patients watched videos and judged their durations in a forced-choice task. For videos of 4 min or more—but not for short videos of 90 sec or less—the patients responded less accurately than matched control participants (Palombo et al., 2016). Next to this contribution to prospective duration estimates in the context of long intervals, damage to the medial temporal lobe also impairs duration estimates in the range of seconds, if events are part of a sequence (Palombo et al., 2020). Here, participants watched pinwheels spin for variable durations. This was followed by test stimuli moving for the same or a different amount of time. Amnesic patients performed worse than matched controls. Notably, this effect only emerged for durations that were part of a sequence of two spins and not for the duration of individual events (Palombo et al., 2020). Together, these data suggest hippocampal damage can impair memory for the duration of event sequences or the individual elements of a sequence.
Converging evidence from neuroimaging studies further demonstrates that hippocampal responses to stimulus sequences are sensitive to duration information (Thavabalasingam, O'Neil, Tay, Nestor, & Lee, 2019; Thavabalasingam, O'Neil, & Lee, 2018; Barnett, O'Neil, Watson, & Lee, 2014). Specifically, these studies manipulated the duration of intervals separating the elements of image sequences. Contrasting hippocampal activity patterns from a study sequence with the subsequent test sequence revealed higher pattern similarity when stimulus order and ISIs at test matched. Pattern similarity was lower if test sequences consisted of the same stimuli in identical order, but with changed ISIs (Thavabalasingam et al., 2018; Figure 1C). Follow-up work demonstrated a similar effect in an adaptation of the paradigm to long-term memory (Thavabalasingam et al., 2019). Participants studied four sequences of three images each. The same or different stimuli were presented with the same or different ISIs. During later recall, individual sequences could be decoded from activity patterns in the anterior hippocampus, whereas the classification of interval duration and stimulus identity alone did not exceed chance levels. This suggests that the anterior hippocampus combines information about stimulus identity and duration (Thavabalasingam et al., 2019). Together, these studies suggest that the human hippocampus is sensitive to the duration of intervals separating the individual elements of sequences (Lee, Thavabalasingam, Alushaj, Çavdaroğlu, & Ito, 2020).
In summary, the hippocampus is involved in encoding and retrieving temporal information (Lieberman et al., 2017; DuBrow & Davachi, 2016; Ekstrom et al., 2011; Tubridy & Davachi, 2011; Jenkins & Ranganath, 2010; Lehn et al., 2009; Ross et al., 2009; Ekstrom & Bookheimer, 2007). This is in line with its well-established role in tracking sequences and its sensitivity to event boundaries that separate different sequences of events (Clewett et al., 2019). Hippocampal activity patterns reflect the temporal relations of different events (Deuker et al., 2016; Nielson et al., 2015). One interesting question concerns how faithful and precise hippocampal representations of temporal relations are. On the one hand, biases in pattern similarity for events with identical underlying temporal relationships have been linked to differences in memory for these relations (Jenkins & Ranganath, 2016; DuBrow & Davachi, 2014; Ezzyat & Davachi, 2014). On the other hand, recent work emphasizes that the hippocampus is also sensitive to the precise timing between events within a sequence (Lee et al., 2020; Palombo et al., 2020; Thavabalasingam et al., 2018, 2019).
ENTORHINAL CORTEX CONTRIBUTIONS TO SEQUENCE MEMORY
Most cortical inputs are relayed to the hippocampus via the entorhinal cortex (for reviews, see, e.g., Witter, Doan, Jacobsen, Nilssen, & Ohara, 2017; Witter, Kleven, & Flatmoen, 2017; van Strien, Cappaert, & Witter, 2009). This raises the possibility that the entorhinal cortex also contributes to sequence memory. However, its role has long been unclear. An early study observed that the entorhinal cortex is sensitive to violations of learned stimulus sequences. For example, it responded more strongly to the latter items of a four-item sequence, when these violated expectations based on the initial items of the sequence (Kumaran & Maguire, 2006b). Entorhinal activity also increased when encountering entirely unknown and novel sequences, whereas the hippocampus responded more selectively to violations of sequence expectations, rather than novelty per se (Kumaran & Maguire, 2006b). In line with overlapping representations of successive stimuli from probabilistic sequences, the entorhinal cortex might extract regularities from repeatedly encountered sequences (Garvert, Dolan, & Behrens, 2017).
However, how does the entorhinal cortex support the representation of event sequences for episodic memory? It is conceivable that the entorhinal cortex provides temporal information for sequence representations. Consistent with this, decorrelated activity patterns in the entorhinal cortex during sequence processing have been related to later memory for temporal intervals (Lositsky et al., 2016). Over the course of 25 min, participants listened to a science fiction story while undergoing fMRI scanning. In a surprise memory test, they later estimated the time that passed between pairs of events from the story, which were separated by a constant temporal distance. During encoding, multivoxel patterns changed more strongly for events that participants remembered to be separated by larger time intervals. Furthermore, activity patterns in the entorhinal cortex changed particularly slowly in comparison to other brain regions (Lositsky et al., 2016). These findings are in line with contextual representations in the entorhinal cortex that slowly change over time, for example, through the encounter of different events (Lositsky et al., 2016; Howard et al., 2005).
Anatomically, the entorhinal cortex is typically subdivided into two subregions: the lateral and medial entorhinal cortex in rodents (Witter, Doan, et al., 2017; van Strien et al., 2009). In humans, these correspond to the anterior-lateral and posterior-medial portions of the entorhinal cortex (Maass, Berron, Libby, Ranganath, & Düzel, 2015; Navarro Schröder, Haak, Zaragoza Jimenez, Beckmann, & Doeller, 2015). Do the entorhinal subregions differentially contribute to temporal processing or sequence memory? One recent study suggests that the rodent lateral entorhinal cortex in particular carries temporal information during ongoing behavior (Tsao et al., 2018). Neural activity was recorded from the entorhinal cortex of navigating rats. For a subset of cells in the lateral entorhinal cortex, activity patterns were characterized by ramping activity with firing rates increasing or decreasing with different time constants (Tsao et al., 2018). Temporal epochs could be decoded from population activity at multiple timescales, notably with greater accuracy in the lateral compared to the medial entorhinal cortex and hippocampus. Importantly, this temporal information was suggested to arise through the integration of experience from ongoing behavior and internal states, rather than from an explicit clocking mechanism (Tsao et al., 2018). The lateral entorhinal cortex might thus provide an inherent code for the temporal progression of experience, which could potentially subserve the formation of sequence representations for episodic memory (Sugar & Moser, 2019; Tsao et al., 2018; Tsao, 2017).
This potential role of the lateral entorhinal cortex in sequence memory has been investigated in humans. Importantly, recent neuroimaging studies indeed implicated the anterior-lateral subregion of the human entorhinal cortex in sequence memory (Bellmund, Deuker, & Doeller, 2019; Montchal, Reagh, & Yassa, 2019), thus offering a novel perspective on the role of the entorhinal subregions in episodic memory. In one study, participants were shown snapshots from an episode of a sitcom and were asked to indicate when, over the course of the episode, they had seen each scene (Montchal et al., 2019). The anterior-lateral entorhinal cortex, together with a network of brain regions including the hippocampus, the medial pFC, posterior cingulate cortex, and angular gyrus, was more engaged when more accurately recalling the temporal position of a scene (Figure 1D). Notably, the posterior-medial entorhinal cortex was not modulated by memory accuracy (Montchal et al., 2019). These findings suggest that anterior-lateral entorhinal cortex activity supports mnemonic precision for when specific events occurred during a sequence.
What contribution does the anterior-lateral entorhinal cortex make to sequence memory? Pattern similarity analyses suggest that it carries information about the temporal relationships between events of a sequence (Bellmund et al., 2019). In the experiment, originally described in Deuker et al. (2016) and briefly outlined above, participants learned a sequence of events defined by the objects encountered when navigating a route through a virtual city (Bellmund et al., 2019). Later, participants saw images of these objects in random order while undergoing fMRI scanning. Remarkably, the multivoxel pattern similarity between object pairs reflected the temporal distance between when the respective objects were encountered (Figure 1E). Objects encountered at nearby sequence positions became representationally more similar compared to objects at distant sequence positions. This effect was specific to the anterior-lateral entorhinal cortex and temporal distances. More specifically, it was neither observed for spatial distances, measured by Euclidean distances or the lengths of the shortest paths between positions, nor was it detectable in the posterior-medial subregion. Importantly, entorhinal pattern similarity was related to the order in which participants recalled the events during a subsequent and unexpected memory test. Participants, in whom entorhinal pattern similarity resembled the sequence structure more closely, tended to recall objects together, which were originally encountered in temporal proximity. Furthermore, the timeline of events could be reconstructed from multivoxel patterns in the anterior-lateral entorhinal cortex (Bellmund et al., 2019). Together, these findings suggest that anterior-lateral entorhinal cortex forms holistic representations of the temporal relations between different elements in an event sequence.
In sum, the evidence described above allows us to view the role of the entorhinal cortex in sequence memory from a new angle. Specifically, these findings suggest the lateral entorhinal cortex and its human homologue, the anterior-lateral entorhinal cortex, to be particularly relevant for memory of event sequences. Ramping cell activity and population drift in this region carry precise temporal information. Decaying traces of prior experience in the entorhinal cortex (Bright et al., 2019; Tsao et al., 2018) could provide information about how long ago prior events occurred. Such a mechanism is well suited for episodic memory because temporal information inherently arises from experience rather than requiring repeated exposure or training. Consequently, temporal relations can be incorporated in mnemonic representations of event sequences. At retrieval, representations of individual elements might thus reflect temporal interrelations of the events comprising a sequence.
TEMPORAL SCALING OF EPISODIC MEMORY SEQUENCES
The capacity to represent temporal information embedded in the environment extends far beyond sequence memory in the hippocampal–entorhinal region. Precise timing is crucial for many behaviors that do not centrally rely on the medial temporal lobe. For example, dancing tango requires the execution of a motor sequence consisting of different steps. Notably, research on sensory and motor timing has uncovered cellular and network mechanisms underlying temporal computations in different brain regions including the basal ganglia and the cerebellum as well as sensory and motor cortices (Paton & Buonomano, 2018). Once the temporal structure of such a sequence is well learned, “temporal scaling”—the flexible adaptation of the speed of a sequence in response to internal or external demands—becomes possible (Hardy, Goudar, Romero-Sosa, & Buonomano, 2018; Remington, Egger, Narain, Wang, & Jazayeri, 2018; Lerner, Honey, Katkov, & Hasson, 2014; Gütig & Sompolinsky, 2009). Analysis of population activity in the frontal cortex of macaques, performing a time-interval reproduction task, demonstrates the scaling effect on the neural level (Wang, Narain, Hosseini, & Jazayeri, 2018). There were two contrasting aspects of the scaling effect. Although the shape of the trajectories in neural space was similar for trials with the same target interval, the speed at which activity evolved along the trajectories depended on the produced duration (Figure 2A). However, it is unclear if temporal scaling is observed in episodic memory networks in a manner similar to what has been observed in other cognitive domains (Remington, Narain, Hosseini, & Jazayeri, 2018; Wang et al., 2018; Mello, Soares, & Paton, 2015; Lerner et al., 2014).
To support flexible temporal scaling, a neural ensemble in the hippocampal–entorhinal region would need to (1) reflect the temporal patterns elicited by externally or internally generated sequences of stimuli upon recall and (2) temporally scale the retrieved patterns to flexibly replay the sequence at different speeds (Figure 2B). Temporal scaling of a sequence of events can be advantageous in everyday life (Boyer, 2008; Schacter, Addis, & Buckner, 2007; Suddendorf & Corballis, 2007). Consider the way from your office to the exit of the building. Imagine realizing that you lost your keys on your way to the office. You might mentally traverse your memory of arriving at work that day, at a slow pace, to figure out where you may have dropped the keys, in between locking your bike and entering your office. Conversely, in case of a fire emergency, you need to quickly plan an escape route to leave the building and therefore need to replay the trajectory at a fast pace.
Temporal compression appears to occur automatically when event sequences are retrieved from memory. Behavioral experiments provide evidence that recalled sequences progress at a faster rate than the original experience. These studies suggest that recall is particularly compressed when few contextual changes are recalled, such as turns, or when spatially coherent images of a route can be mentally replayed (Arnold, Iaria, & Ekstrom, 2016; Bonasia, Blommesteyn, & Moscovitch, 2016). Conversely, compression is less pronounced when details of goal-directed actions—during which context is relatively constant—are remembered (Jeunehomme & D'Argembeau, 2019; Jeunehomme, Folville, Stawarczyk, Van der Linden, & D'Argembeau, 2018). Indeed, the temporal compression of episodic recall is reduced when experience is segmented into more fine-grained events (Jeunehomme & D'Argembeau, 2020). These findings are in line with the role of contextual boundaries in delineating event sequences (Bangert et al., 2019; Clewett et al., 2019; Faber & Gennari, 2015, 2017). Moreover, neuroimaging studies show variations in the amount of temporal compression between participants (Chen et al., 2017), and there is evidence suggesting that the variable duration of memory reactivation is because of participants skipping sequence elements (Michelmann, Staresina, Bowman, & Hanslmayr, 2019). In contrast, duration estimates for recalling a life-threatening situation are expanded (Stetson, Fiesta, & Eagleman, 2007).
Clues about a potential role of the hippocampus in the scaling of neural activity sequences for episodic memory are provided by studies of spatial coding. A sequence of place cells in the hippocampus will become active as an animal traverses its environment. Notably, comparable hippocampal sequences have been observed in various situations and experiments in rodents. Such studies have shown that time-varying activity of cell assemblies in the hippocampal–entorhinal region can be driven by either endogenous (self-organized patterning) or exogenous (environmental and/or self-motion cues) mechanisms (Buzsáki & Tingley, 2018; Tsao et al., 2018; Aronov, Nevers, & Tank, 2017; Friston & Buzsáki, 2016; Kay et al., 2016; Kraus et al., 2015; Modi, Dhawale, & Bhalla, 2014; Kraus, Robinson, White, Eichenbaum, & Hasselmo, 2013; MacDonald et al., 2011; Pastalkova et al., 2008). The sequential firing of cells during a trajectory through space has been related to interval timing (Issa, Tocker, Hasselmo, Heys, & Dombeck, 2020) as well as to the representation of a sequence of events (Friston & Buzsáki, 2016; Buzsáki, Peyrache, & Kubie, 2014; Buzsáki & Moser, 2013; Hasselmo, 2009; Byrne, Becker, & Burgess, 2007). In line with the latter, studies using fMRI demonstrate that hexadirectional activity modulations in the human entorhinal cortex, elicited during active virtual navigation (Doeller, Barry, & Burgess, 2010), are also present when participants simulate trajectories through space during imagined navigation (Bellmund, Deuker, Navarro Schröder, & Doeller, 2016; Horner, Bisby, Zotow, Bush, & Burgess, 2016). Hippocampal activity sequences might underlie memory for episodes unfolding in time.
To speed up or slow down the progress through an episodic memory sequence, a neural network needs to first learn the sequence and then reproduce its temporal structure. Rodent studies demonstrate that learned sequences of traveled paths, represented by an evolving assembly of hippocampal neurons, can be reactivated spontaneously during wakeful rest or sleep (Karlsson & Frank, 2009; Nádasdy, Hirase, Czurkó, Csicsvari, & Buzsáki, 1999; Wilson & McNaughton, 1994). An important feature of this replay of past experiences is the temporal compression of the reactivated activity patterns. In rodents, the average speed of a replayed hippocampal sequence at wakeful rest tends to be ∼20 times faster than the experienced one, ranging between 100 and 300 msec (Ólafsdóttir, Bush, & Barry, 2018; Pfeiffer & Foster, 2015; Davidson, Kloosterman, & Wilson, 2009; Diba & Buzsáki, 2007; Lee & Wilson, 2002). Notably, neural correlates of the replay of learned sequences have been observed in humans (Wimmer, Liu, Vehar, Behrens, & Dolan, 2020; Liu, Dolan, Kurth-Nelson, & Behrens, 2019; Michelmann et al., 2019; Kurth-Nelson, Economides, Dolan, & Dayan, 2016). Recent studies suggest these decoded sequential patterns, which reflect previous nonspatial experiences, may originate from hippocampal activity (Liu, Dolan, et al., 2019; Schuck & Niv, 2019).
The aforementioned evidence supports the idea that cell assemblies in the hippocampus can reproduce temporally structured activity patterns at faster speeds. However, they do not fully demonstrate that evolving neural ensembles in the hippocampal–entorhinal region are temporally scalable because it is unclear whether their speed can be flexibly adapted. During sensorimotor tasks, cell assembly sequences in the striatum of rats (Gouvêa et al., 2015), or the frontal cortex of monkeys (Remington, Narain, et al., 2018; Wang et al., 2018), are able to adapt their rate of change in response to external or internal demands. A similar observation has been reported in the medial entorhinal cortex of mice performing an instrumental timing task. Specifically, the animals' response timing correlated with the sequence progression speed of a population of neurons in the medial entorhinal cortex (Heys & Dombeck, 2018). Despite a growing amount of studies on human spontaneous replay (Liu, Dolan, et al., 2019; Michelmann et al., 2019; Schuck & Niv, 2019; Kurth-Nelson et al., 2016), systematic variations in the speed of neural activity sequences during replay and memory recall have, to the best of our knowledge, not been studied. Hence, it remains to be tested whether it is possible to alter the speed of progression through hippocampal activity sequences in response to internal or external demands. Investigating the correspondence between episodic memory retrieval and spontaneous replay could help researchers determine how memories are both voluntarily and implicitly temporally scaled.
How could the rate of change of an episodic memory sequence be controlled? One possibility is that, in the same way as running velocity adjusts the transition time between hippocampal place cell ensembles within the theta cycle (Maurer, Burke, Lipa, Skaggs, & Barnes, 2012), an internally generated signal could adjust the rate of change of this evolving neural population state (Buzsáki, 2019; Buzsáki & Tingley, 2018). Evidence consistent with this idea comes from studies showing that changes in internal states such as attention (Polti, Martin, & van Wassenhove, 2018), emotions (Droit-Volet, 2013), or even body temperature (Hancock, 1993) influence duration estimates. Interestingly, recent work on a time-cell model has demonstrated how the temporal scale of a sequence can be modified, resulting in sequences progressing at different speeds (Liu, Tiganj, et al., 2019). Similar temporal rescaling was observed in a subset of time cells after changes of the delay duration, although most cells unpredictably changed their activity profile in response to this alteration (MacDonald et al., 2011). Notably, temporal scaling of sequence memories might be reflected in changes of the speed at which mnemonic trajectories are traversed (Buzsáki et al., 2014; Buzsáki & Moser, 2013; Hasselmo, 2009; Byrne et al., 2007).
Taken together, the available experimental evidence suggests that the temporal scaling of episodic memory sequences can be supported by the population activity of neurons in the hippocampal–entorhinal region. Future research could build on this new perspective and aim to understand the mechanisms that drive the flexible dynamics of this mnemonic network. Furthermore, it would help to elucidate how humans are able to voluntarily stretch or compress episodic memories and which role the hippocampus and entorhinal cortex play in the temporal scaling of event sequences.
In this review, we have summarized the established role of the hippocampus in temporal processing and incorporated recent evidence demonstrating contributions of the entorhinal cortex to sequence memory. In the following, we outline some questions that emerge from this new perspective on the medial temporal lobe memory system. The first question concerns the way the hippocampus and entorhinal cortex interact when processing sequences of events. Theoretical work has, for instance, demonstrated that time cells in the hippocampus might arise from entorhinal time ramping cells or decaying traces of prior experience (Liu, Tiganj, et al., 2019; Rolls & Mills, 2019; Howard et al., 2014; Shankar & Howard, 2012), yielding testable predictions for future studies. With respect to human memory, elucidating commonalities and differences between hippocampal and entorhinal representations poses an intriguing question along with the challenge of understanding how they emerge during learning.
Although we focused mostly on the lateral entorhinal cortex, prior work has also reported temporal coding in the medial entorhinal cortex. For example, grid cells are sensitive to elapsed time as well as distance while running in place (Kraus et al., 2015). Furthermore, cell populations sensitive to specific time points of a delay, during which the animal was immobile, have also been described in the medial entorhinal cortex (Heys & Dombeck, 2018). Whether and how the medial entorhinal cortex contributes to human memory of temporal relations remains to be understood. With respect to the interplay between the hippocampus and medial entorhinal cortex, there is conflicting evidence concerning the relevance of the medial entorhinal cortex for hippocampal time cells. Whereas one study reported destabilized time cell sequences after transient optogenetic inactivation of the medial entorhinal cortex (Robinson et al., 2017), more recent work observed no effects of medial entorhinal cortex lesions on time cells (Sabariego et al., 2019). A better understanding of the influence of the medial entorhinal cortex on temporal codes in the hippocampus, in animal models, could help to generate more precise predictions to dissect their potential interplay in human sequence memory.
We have discussed evidence that the hippocampus supports the representation of temporal relations, particularly its anterior portion. However, a better understanding is needed of the role different hippocampal subfields might play beyond functional segregation along its long axis. Studies in rodents have uncovered changes in the activity patterns of cells in CA1 and CA2, which might provide temporal context information for memory (Mau et al., 2018; Mankin, Diehl, Sparks, Leutgeb, & Leutgeb, 2015; Mankin et al., 2012; Manns, Howard, & Eichenbaum, 2007). In contrast, CA3 activity patterns are comparatively stable over time (Mankin et al., 2012, 2015). Recent evidence in humans suggests an overlapping representation of items from the same episode in CA1, whereas CA3 differentiated these items (Dimsdale-Zucker, Ritchey, Ekstrom, Yonelinas, & Ranganath, 2018). Given ongoing improvements of high-resolution neuroimaging techniques, it will be interesting to further disentangle the roles of the different hippocampal subfields in human sequence memory.
Recent theoretical work has taken different perspectives on domain-general coding principles of the entorhinal cortex (Mok & Love, 2019; Behrens et al., 2018; Bellmund, Gärdenfors, Moser, & Doeller, 2018; Buzsáki & Tingley, 2018; Stachenfeld, Botvinick, & Gershman, 2017). In the medial entorhinal cortex, grid cells are thought to provide a distance function for cognitive spaces (Bellmund et al., 2018) and might extract structural information from transition statistics (Behrens et al., 2018). As alluded to above, this raises the question of how medial versus lateral entorhinal cortices are differentially involved in sequence memory. The lateral entorhinal cortex has been implicated in object processing (Knierim, Neunuebel, & Deshmukh, 2014; Tsao, Moser, & Moser, 2013; Deshmukh & Knierim, 2011), a function mirrored in some proposals (Behrens et al., 2018). Possibly, traces of prior events in the entorhinal cortex (Bright et al., 2019; Tsao et al., 2018) allow it to contribute information not only about item identities but also about temporal relationships between different sequence elements to memory. This might explain the involvement of the anterior-lateral entorhinal cortex in recent studies of episodic memory (Bellmund et al., 2019; Montchal et al., 2019). The medial entorhinal cortex has been suggested to play a central role in extracting structural information (Behrens et al., 2018). For example, the sequences of events unfolding on your way to work might share a similar structure across days. Notably, prior work has emphasized the role of a posterior-medial memory network in representing shared sequence structure (Cohn-Sheehy & Ranganath, 2017; Hsieh & Ranganath, 2015). Future research should aim to elucidate how different sequences, which might share a similar structure, are represented in the hippocampus and entorhinal cortex.
We have highlighted contributions of the hippocampus and the adjacent entorhinal cortex to sequence memory. Notably, the hippocampal–entorhinal system supports episodic memory in close connection with cortical networks, which can integrate episodic information from the past 30 sec even when the hippocampus is damaged (Zuo et al., 2020). How temporal integration processes in cortical areas interact with the hippocampal–entorhinal region to create sequence memories remains to be understood.
In this review, we have focused on memory for event sequences. We have summarized influential findings demonstrating a role of the hippocampus in both the encoding and retrieval of sequence information. We have incorporated novel evidence implicating the entorhinal cortex in sequence memory to take a new perspective on the hippocampal–entorhinal memory system. Population signals well suited to provide temporal information for episodic memory have been discovered in the rodent lateral entorhinal cortex, and consistent with that, studies in humans have demonstrated the involvement of its homologue region in memory recall. Furthermore, we have discussed the idea of temporal scaling in the context of event sequences and described how flexibly adapting the speed of sequence progression could benefit episodic memory and mental simulation.
The authors thank Raphael Kaplan and Jørgen Sugar for helpful comments on a previous version of this article and Iván Andrés Davidovich for insightful discussions. C. F. D.'s research was supported by the Max Planck Society, the Kavli Foundation, the European Research Council (ERC-CoG GEOCOG 724836), the Centre of Excellence scheme of the Research Council of Norway–Centre for Neural Computation (223262/F50), the Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, and the National Infrastructure scheme of the Research Council of Norway–NORBRAIN (197467/F50). I. P. conceived the idea of temporal scaling in episodic memory and drafted the corresponding section of the manuscript. J. B. drafted the sections on sequence memory in the hippocampus and entorhinal cortex as well as the abstract, introduction, outstanding questions and conclusion. C. D. supervised the project. All authors provided critical feedback and contributed to the final manuscript.
Reprint requests should be sent to Jacob L. S. Bellmund, Department of Psychology, Max Planck Institute for Human Cognitive and Brain Sciences, Stephanstraße 1a, 04103 Leipzig, Germany, or via e-mail: email@example.com or Ignacio Polti, Kavli Institute for Systems Neuroscience, Centre for Neural Computation, The Egil and Pauline Braathen and Fred Kavli Centre for Cortical Microcircuits, Norwegian University of Science and Technology, Olav Kyrres Gate 9, 7030 Trondheim, Norway, or via e-mail: firstname.lastname@example.org.
This article is part of a Special Focus deriving from a symposium at the 2019 annual meeting of Cognitive Neuroscience Society, entitled, “Mental Models of Time.”