Abstract

Numerous studies indicate the importance of the hippocampus to temporal order retrieval. However, behavioral studies suggest that there are different ways to retrieve temporal order information from encoded sequences, one involving an associative strategy (retrieving associations using neighboring items in a list) and another involving a recency strategy (determining which of two items came first). It remains unresolved, however, whether both strategies recruit the hippocampus or only associative strategies, consistent with the hippocampus's role in relational processing. To address this, we developed a paradigm in which we dissociated associative versus recency-based retrieval, involving the same stimulus presentation during retrieval. Associative retrieval involved an increase in RT (and decrease in performance) with greater distances between intervals, consistent with the need to retrieve intervening associations. Recency-based retrieval involved an increase in RT (and decrease in performance) with shorter distances between intervals, suggesting the use of a strength-based coding mechanism to retrieve information. We employed fMRI to determine the neural basis of the different strategies. Both strategies showed significant levels of hippocampal activation and connectivity that did not differ between tasks. In contrast, both univariate and connectivity pattern analyses revealed differences in extrahippocampal areas such as parietal and frontal cortices. A covariate analysis suggested that differences could not be explained by task difficulty alone. Together, these findings suggest that the hippocampus plays a role in both forms of temporal order retrieval, with neocortical networks mediating the different cognitive demands for associative versus recency-based temporal order retrieval.

INTRODUCTION

Episodic memory involves remembering details from events with temporal information providing a particularly important anchor to differentiate these details (Howard & Kahana, 2002; Tulving, 2002). For example, if we attempt to recall what we had for dinner last week, remembering what we had the night before or whether we were traveling earlier that week are both helpful temporal order cues for recalling that event. Numerous studies have implicated the hippocampus as central to episodic memory with damage to this important structure resulting in impairments in retrieving details of recent events (Rosenbaum, Gilboa, Levine, Winocur, & Moscovitch, 2009; Yonelinas et al., 2002). Indeed, a hallmark of medial-temporal lobe (MTL) damage involves fragmented, temporally disrupted memory (Rosenbaum et al., 2009; Downes, Mayes, MacDonald, & Hunkin, 2002), suggesting a fundamental connection between the hippocampus, episodic memory, and temporal order. In further support of the importance of the hippocampus to temporal processing, lesions to the rodent hippocampus produce impairments in memory for temporal order (Farovik, Dupont, & Eichenbaum, 2010). Together, these findings suggest important links between temporal order memory and the hippocampus, possibly as part of its role in episodic memory.

fMRI experiments in human participants also support the involvement of the MTL, particularly the hippocampus, in temporal order judgments (Wang & Diana, 2016; Kyle, Smuda, Hassan, & Ekstrom, 2015; Nielson, Smith, Sreekumar, Dennis, & Sederberg, 2015; Copara et al., 2014; Ezzyat & Davachi, 2014; Hsieh, Gruber, Jenkins, & Ranganath, 2014; Kalm, Davis, & Norris, 2013; Ekstrom, Copara, Isham, Wang, & Yonelinas, 2011; Kimura et al., 2010; Lehn et al., 2009). Several of these studies suggest that explicitly retrieving temporal order information recruits the hippocampus at levels significantly above baseline (Kyle et al., 2015; Ekstrom et al., 2011; Kimura et al., 2010). In a study by Ekstrom and colleagues (2011), participants retrieved either spatial or temporal distance information from recently experienced spatial environments. Although these two types of judgments were behaviorally dissociated in the experiment (and thus retrieval of spatial distance information could not be explained by temporal order retrieval; see also Gauthier & van Wassenhove, 2016), both conditions resulted in significant activation of the hippocampus (Ekstrom et al., 2011). In a similar vein, Kyle and colleagues (2015) had participants indicate whether two stores were the same or different temporal distance from a reference store following navigation, requiring detailed knowledge of the temporal order of landmarks within the environment (Kyle et al., 2015). Similar to Ekstrom et al., the authors found significant levels of hippocampal activation when participants correctly retrieved temporal order and spatial layout information. Together, these studies argue for the involvement of the human hippocampus in processing temporal order, particularly in paradigms requiring detailed knowledge of the temporal sequence (see also Wang & Diana, 2016; Kalm et al., 2013; Kimura et al., 2010).

In contrast to the relative importance of the hippocampus to temporal order memory, another important brain area for this process is the pFC, with damage to this structure also producing impairments in temporal order memory (Duarte, Henson, Knight, Emery, & Graham, 2010; Milner, Corsi, & Leonard, 1991). In one study comparing patients with lesions to frontal versus the MTL, Milner and colleagues (1991) showed frontal lobe lesions profoundly impaired temporal order recency judgments compared with healthy controls, whereas MTL patients showed little to no deficit (Milner et al., 1991; see also Craver, Kwan, Steindam, & Rosenbaum, 2014; Rosenbaum et al., 2005). In the task, patients viewed cards in a deck containing abstract pictures and concrete words. At a random point, a card would cue patients to indicate which of two items from earlier in the deck came first, termed recency judgments. Milner and colleagues found that patients with frontal lobe damage, especially in the left hemisphere, were severely impaired at these recency judgments. In contrast, patients with MTL damage showed essentially no impairment on the task. These findings thus argued that the frontal lobes, not the MTL, are most important for recency judgments.

Consistent with the findings from Milner et al. (1991) suggesting the importance of pFC to temporal order coding, Marshuetz, Smith, Jonides, DeGutis, and Chenevert (2000) scanned healthy participants performing a working memory task where they indicated which of two letters in a sequence came first. They compared this condition with a separate one in which participants indicated whether they had seen one of the two letters before. The authors found parietal cortex and pFC activation for recency judgments but reported no MTL activation, which argued for the importance of these areas rather than the MTL for recency judgments. Notably, participants in this study, like in the Milner et al. study, performed recency judgments, which would not require the same degree of detailed temporal order knowledge as the task used in Ekstrom et al. (2011) and Kyle et al. (2015; see also: Kimura et al., 2010; Konishi, Asari, Jimura, Chikazoe, & Miyashita, 2006). Participants also performed a working memory task, which differs somewhat from the more episodic tasks conducted with past work on temporal order memory discussed above.

More generally, understanding in what manner a sequence is retrieved is an important component in understanding temporal order processing and could possibly underlie some of the discrepancies in past findings. Indeed, past cognitive studies suggest two fundamentally different ways that human participants might retrieve a sequence. One of these, as discussed so far, can involve recency judgments, which involves the participant making a judgment about which of two stimuli occurred earlier in a sequence (Hintzman, 2005; Yntema & Trask, 1963). Current theoretical models of recency judgments suggest that they may involve strength-based mechanisms whereby items encoded at different points in the list may have different levels of familiarity. These different levels of familiarity thus endow items at different positions in the list with different “strengths,” particularly the first and last items, providing a means of discriminating their relative order (Hintzman, 2005; Howard & Natu, 2005). Congruous with this idea, participants typically show faster RTs for items that are further apart in the list, suggesting that the relative strength of the encoded items can serve as a temporal order cue (Hintzman, 1976; Yntema & Trask, 1963). These findings relate to the idea of classic “distance effects”; in other words, the larger the temporal spacing between two items, the faster the RT to decide which of the two is larger (Moyer & Landauer, 1967).

Evidence from behavioral studies involving serial order memory paradigms reveals another strategy for retrieving temporal order, an associative mechanism. Specifically, when participants learn a list of words or objects and are instructed to learn this in order, theoretical models of serial order memory suggest that participants form associations between neighboring items (Addis & Kahana, 2004; Murdock, 1968, 1974). For example, if a participant learns the list “A-B-C-D,” when they retrieve the item “A” this will serve as a cue for B, which will then serve as a cue for C, and so on. Several behavioral studies have supported this prediction, showing that adjacent items in a studied list show a strong bias to be recalled close together (Solway, Murdock, & Kahana, 2012; Klein, Addis, & Kahana, 2005; Kahana, 1996; Murdock & Okada, 1970). Furthermore, a detailed analysis of participant verbal response patterns during recall suggests that they are more likely and faster to retrieve the item “C” after recalling “B” compared with any other item, including the backward item “A.” Confirming this tendency, Kyle and colleagues (2015) showed faster RT and greater accuracy for correctly judging that items A and C are equal temporal distance from the item “B” in the triad “A-B-C” compared with the more distant items “A-C-E.” Together, these studies suggest that participants can also retrieve temporal sequences using an association-based strategy.

Given the divergence of past studies on temporal order processing, with some studies employing recency-based strategies showing little, if no, hippocampal involvement, and those using associative strategies demonstrating hippocampal involvement, an important issue to resolve is the extent of the involvement of the hippocampus in different temporal order retrieval strategies. Another important issue is to try to match the visual input shown to participants in the same paradigm so this is not a possible counterexplanation for the findings. Finally, we employed the same episodic-based learning paradigm in which participants encoded a temporal sequence and then later retrieved this information in the scanner. Thus, in our paradigm, participants first encoded temporal sequences of objects (similar to that used in Kyle et al., 2015) and then retrieved these using two different types of temporal order judgments. On half of the blocks, participants retrieved the recently encoded items using a task designed to evoke an associative mechanism identical to what was employed in Kyle et al. (are the two objects equal or unequal distances from a third probe object?). On the other half of blocks, participants performed a different temporal memory retrieval task. Participants judged which of two objects from a sequence was closer to a reference object that occurred at the beginning or end of the list. Importantly, the rendering of triads on the screen was exactly matched across conditions, so visual confounds could not account for any differences we found. If the hippocampus is involved in associative and not recency judgments, we would expect greater hippocampal activation for the contrast of associative versus recency trials. If the reverse were true, we would expect greater hippocampal activation in the recency versus associative contrast. A final hypothesis is the hybrid of the two: If the hippocampus is involved in both, we would expect little hippocampal activation in the direct contrasts but greater hippocampal activation when compared with baseline in both conditions.

METHODS

Participants

We tested a total of 23 participants (average age = 22.8 years, age range = 18–38 years, 12 women) from the University of California, Davis, and the surrounding communities. Five participants were excluded, leaving a total of 18 participants for analysis (average age = 22.6 years, age range = 18–34 years, 10 women). Two of the participants were excluded because of excess head motion, two were excluded because of failure to complete the scanning session, and one was excluded because of technical problems during scanning. All participants were right-handed with normal or corrected-to-normal vision and screened for neurological disorders. This study was approved by the institutional review board at the University of California, Davis, and written informed consent was obtained from each participant before starting the experiment.

Behavioral Design: Encoding

The study consisted of an encoding session (not scanned) and a retrieval session (scanned; see Figure 1). The encoding task was designed using the Unity video game engine (Unity Technologies, San Francisco, CA) and required participants to memorize two unique temporal sequences of stores, each of which was associated with one of the retrieval tasks. Participants learned a short list, containing 6 stores, and a long list, consisting of 15 stores. This was done to match difficulty, as closely as possible, for the associative versus recency tasks, as our pilot experiments indicated that the associative task was substantially harder than the recency task. Each list was associated with either the associative or recency-based retrieval task, which are described below.

Figure 1. 

Participants first learned two unique lists of stores by watching videos of the storefronts, then sorting the randomized lists into the correct order by dragging and dropping storefront images into the blank box corresponding to their temporal position. The upper row depicts a selection of three stores from the 15-item “recency” list as seen during encoding, followed by the test screen. After encoding, participants underwent functional imaging while retrieving temporal order information. They viewed a triad of stores consisting of one reference store and two probe stores and indicated either whether the distances from the probe stores to the reference store were equal (associative) or which probe store was closer to the reference store (recency). In between trials, participants performed a control task, where they saw X or O symbols and indicated which letter appeared via button press.

Figure 1. 

Participants first learned two unique lists of stores by watching videos of the storefronts, then sorting the randomized lists into the correct order by dragging and dropping storefront images into the blank box corresponding to their temporal position. The upper row depicts a selection of three stores from the 15-item “recency” list as seen during encoding, followed by the test screen. After encoding, participants underwent functional imaging while retrieving temporal order information. They viewed a triad of stores consisting of one reference store and two probe stores and indicated either whether the distances from the probe stores to the reference store were equal (associative) or which probe store was closer to the reference store (recency). In between trials, participants performed a control task, where they saw X or O symbols and indicated which letter appeared via button press.

During the encoding task, participants viewed a video of one of the lists in which the storefronts were presented in list order on a rotating cube against a black background. To try to reduce any explicit spatial information and discourage participants from using a spatial coding strategy, each store was visible one at a time, and the rotating cube was shown in the center of a black screen. We used store fronts to match as closely as possible our previous results using these stimuli showing hippocampal activation for retrieving temporal order (e.g., Kyle et al., 2015; Ekstrom et al., 2011). Following the end of the video, participants were presented with all stores from the current list in random order and instructed to sort them into the order shown during the video. They then received feedback on the percentage of stores placed correctly and performed the encoding task with the other list. Participants repeated the task, alternating between lists, until reaching criterion of 100% accuracy on both lists. The list order was counterbalanced across participants. Average encoding time to first reaching criterion for a given list was 1.8 trials for the 6-store list and 4.2 trials for the 15-store list.

Behavioral Design: Retrieval during fMRI

The scanned retrieval session took place immediately following encoding and employed an event-related design paradigm. To avoid participants confusing retrieval strategies, associative and recency retrieval were split into 10 separate blocks. This included five consecutive associative temporal retrieval blocks and five consecutive recency-based temporal retrieval blocks, with the order of the tasks (matched to the order of list learning during encoding, or reversed) counterbalanced across participants. Questions for associative and recency temporal retrieval tasks were also presented identically, but the question that participants answered was specific to each task (Figure 1). This was intended to “force” participants to use a distinct retrieval mechanism for each task while simultaneously controlling for visual input.

Before the beginning of each block, participants were instructed which task they would be performing and viewed a refresher video of the list associated with that task, which was identical to the video of that list used during encoding. For the associative temporal retrieval task, participants were tested on the 6-store list and determined whether the temporal distances between two probe stores and a reference store were equal or unequal. For the recency temporal retrieval task, participants were tested on the 15-store list and determined which of two probe stores was closer in the list order to a reference store.

During both associative and recency temporal order retrieval trials, participants viewed triads of one reference store at the top of the screen and two probe stores at the bottom of the screen for 6 sec and made temporal order judgments via an MRI-compatible button box. All three stores appeared on the screen with no accompanying text or additional details. Both associative and recency temporal retrieval blocks consisted of 36 trials each, which were broken down on the basis of the temporal intervals used in store triads. There were five blocks per task for a total of 180 trials per retrieval type.

Associative temporal retrieval blocks were split into 10 small-interval or “1-away” trials, in which there was one step between the reference and each probe store (e.g., stores 5, 4, and 6), 8 large-interval or “2-away” trials (e.g., stores 3, 1, and 5), and 18 “unequal” trials which were considered of no interest (e.g., stores 2, 1, and 5). Note that there were fewer “2-away” trials because the sequence length of six stores limited the possible number of triads. Because of this limitation on the available triads, we presented a subset of the associative triads twice per block rather than once to keep the number of stimulus presentations per block matched across tasks. On each trial, participants thus answered whether the probe stores were the same or different distances from the reference store. Recency temporal retrieval blocks were split into 18 small-interval (1 store) and 18 large-interval (2 stores) trials, although for this task, the intervals between the two probe stores were used. For example, a triad of stores 1, 12, and 13 (which has one step between the two probe stores) would be considered a small-interval trial. On each trial, participants answered which of the two probes was closer in the list order to the reference store. The reference store was either store 1, 2, 14, or 15 (randomly presented across trials with equal probability).

Presentation of stimuli was jittered using a geometric distribution of intertrial intervals. Participants performed an active baseline task in between trials to better model task-related hippocampal activations (Stark & Squire, 2001). During the baseline task, participants pressed “1” or “2” when an X or O appeared on the screen, respectively. Each letter appeared for 1 sec, and total intertrial interval time ranged from 1 to 12 sec.

Examination of Distance Effects

In addition to examining the differences between the two mechanisms, we sought to confirm the presence of numerical distance effects (Moyer & Landauer, 1967) during the recency temporal retrieval task. As opposed to associative effects, numerical distance effects predict that nearby items are retrieved slower than more distant items (Yntema & Trask, 1963), a behavioral finding that would be critical to showing that participants used different strategies for the recency versus associative tasks. Because our fMRI experiment involved only two intervals for the recency task in our efforts to match the tasks as closely as possible, we ran a separate behavioral study involving the recency tasks with more intervals of discrimination.

Classic distance effects findings predict a monotonic decrease in RT (and a monotonic increase in performance) for items of greater temporal distance (Moyer & Landauer, 1967; Yntema & Trask, 1963). In other words, the further apart two studied items are, the faster participants are to respond in terms of which one came first and the better they perform on these judgments, compared with items that were closer in the list. To address whether we could observe classic distance effects in our study, we therefore conducted a secondary behavioral experiment and tested a total of 20 participants, none of whom had participated in the imaging study. Participants encoded the 15-store list and performed the distance task identically to the imaging study. However, the stimulus set included store triads with larger temporal intervals between probe stores and was broken down into intervals of 1, 2, 3, and 4 stores (9 trials per group). The visual rendering and experimental structure was otherwise identical.

Imaging Methods

Imaging took place immediately following encoding at the UC Davis Imaging Research Center in Davis, CA, using a Siemens (Erlingen, Germany) 3T Skyra scanner with a 32-channel head coil. Retrieval testing generally began approximately 30 min after the completion of encoding to allow time for positioning in the scanner and collection of structural scans. Structural images were acquired using a whole-brain 1 × 1 × 1 mm MPRAGE sequence. Functional images were acquired using a whole-brain 2 × 2 × 2.2 mm multiband EPI sequence (repetition time = 1600 msec, echo time = 25 msec, slices = 52, field of view = 208 mm, flip angle = 65°, bandwidth = 1550 Hz/pixel).

Preprocessing and parameter estimation were conducted using Statistical Parametric Mapping (SPM12) software (Wellcome Centre for Neuroimaging, UCL, London, UK). Functional images were motion-corrected, and the mean realigned functional image was coregistered to participants' structural images. Structural images were segmented into gray and white matter images, and functional and structural images were spatially normalized into MNI space. Normalized functional images were high-pass filtered at 128 sec to remove scanner drift and cardiac/respiratory artifacts (Frackowiak et al., 2004) and spatially smoothed using a 4-mm FWHM Gaussian kernel.

In a subject-specific first-level modeling phase, individual stimulus onsets from each condition were convolved with the canonical hemodynamic response function (double gamma) and then entered into a general linear model (Friston, Frith, Frackowiak, & Turner, 1995). Each trial was modeled as the onset of the store triad with a duration of 6 sec. Associative sessions therefore contained three task regressors (1-away, 2-away, and unequal trials), whereas recency sessions contained two task regressors (small-interval and large-interval trials). Baseline periods, which involved indicating whether an “X” or an “O” appeared on the screen, were implicitly modeled to provide for mean activation “zero” baseline comparison (Stark & Squire, 2001). Six head motion regressors for each session were also included in the general linear model. Contrast images were then carried forward into a second-level analysis. Second-level group analyses utilized one-way t tests and paired t tests to evaluate unique and common patterns of activations within and between conditions. The active baseline task was implicitly modeled in all cases and not directly included in any contrasts. Trials not of interest (e.g., associative unequal trials) were explicitly modeled but not included in further analyses. Results were corrected for false positives by performing Monte Carlo simulations in AFNI's 2015 (updated) release of 3dClustSim. A cluster-corrected, whole-brain-corrected p < .05 value corresponded to uncorrected voxelwise p < .001 and a cluster threshold of k = 41. Areas of activation were further investigated using MarsBaR v. 0.44 (Brett, Anton, Valabregue, & Poline, 2002) by extracting the percent signal change from clusters that were identified during whole-brain analysis.

Graph Theory Analysis

Using the methods as described in Schedlbauer, Copara, Watrous, and Ekstrom (2014), we performed a functional connectivity analysis that probed the interregional variance in activation over all trials within a particular condition (Schedlbauer et al., 2014). Briefly, we employed a beta time series approach (Rissman, Gazzaley, & D'Esposito, 2004), where each voxel's BOLD response in the task was modeled in a general linear model as an individual regressor specifying the onset of each trial convolved with the canonical hemodynamic response function. The parameter or beta estimates derived for each trial for each voxel were then sorted by condition (associative, recency) into a beta series. The beta series of voxels belonging to an ROI (a 5 × 5 × 5 voxel cube located at the center of mass of a region defined by the AAL atlas) were subsequently averaged, culminating in 17 average beta series per condition. The cluster peak voxel coordinates were not used for the cube locations to avoid “double dipping” or the redundant use of analyzed data to obtain results (Kriegeskorte, Simmons, Bellgowan, & Baker, 2009). The Pearson product–moment correlation coefficients between all ROIs' beta series were computed, creating a correlation matrix of all pairwise combinations describing the strength of the functional relationship between two regions. Using a bootstrap technique, we controlled for false positives by deriving a distribution of correlation coefficient values calculated from randomly shuffled beta series (2,000×) pooled across participants and all pairwise ROIs within a condition. Any observed correlation value that was greater than the 99.9th percentile of the calculated distribution was considered significant. Significant and nonsignificant values between two nodes received a one and a zero, respectively, in the connectivity matrix. The ROIs in the network are denoted as nodes, whereas significant connections are called edges. To compare conditions, the set difference of significant connections for associative versus recency revealed the functional connections unique to a condition (Figure 4). Networks are visualized using Brain NetViewer (Xia, Wang, & He, 2013). The network topologies were characterized using measures of node degree, or the sum of all edges connected to a node, on binary, undirected networks. All computations were calculated using custom written MATLAB code (Mathworks, Natick, MA).

RESULTS

Behavioral Results

Average performance was significantly above chance for all conditions, as shown in Table 1. Performance on trials of interest differed according to associative or recency temporal retrieval and interval size. A two-way repeated-measures ANOVA revealed a significant Task × Interval interaction effect for both performance (F(1, 17) = 14.457, p < .001) and RT (F(1, 17) = 39.375, p < .001). Importantly, the tasks exhibited opposing patterns of behavior for both performance and RT with respect to interval size. Specifically, participants performed better and responded significantly faster for small intervals than large intervals during the associative temporal retrieval task (performance: t(17) = 3.6694, p = .0019, RT: t(17) = 7.5557, p < .001), replicating Kyle et al. (2015). In contrast, responses during the recency retrieval task were numerically faster and more accurate for large-interval trials than small-interval trials (although this difference did not reach statistical significance).

Table 1. 

Behavioral Performance during Associative and Recency-based Order Retrieval

TaskPerformance (%)RT (sec)
Imaging Study 
Chaining 
 Small interval 89.0 3.20 
 Large interval 75.0 3.61 
Distance 
 Small interval 94.5 2.89 
 Large interval 94.9 2.85 
 
Distance Effects Study 
Distance 
 1 store 84.3 3.27 
 2 stores 87.7 3.10 
 3 stores 91.2 3.02 
 4 stores 95.0 2.82 
TaskPerformance (%)RT (sec)
Imaging Study 
Chaining 
 Small interval 89.0 3.20 
 Large interval 75.0 3.61 
Distance 
 Small interval 94.5 2.89 
 Large interval 94.9 2.85 
 
Distance Effects Study 
Distance 
 1 store 84.3 3.27 
 2 stores 87.7 3.10 
 3 stores 91.2 3.02 
 4 stores 95.0 2.82 

Although the interaction effect for Task versus Interval suggested that participants employed different retrieval strategies for the associative versus recency task for small versus large intervals, the difference in performance and RT did not reach significance for large versus small intervals in the recency retrieval task. This could have occurred because the difference in 1-away versus 2-away intervals in the recency retrieval task was too small to show statistically reliable distance effects. To ensure that participants did indeed show decreasing RT and increasing performance for recency judgments of large versus small intervals, we analyzed our separate behavioral study involving 1–4 rather than 1–2 recency intervals. The presence of distance effects was confirmed: a one-way ANOVA identified a main effect of Interval for both performance (F(1, 19) = 21.0, p < .0005) and RT (F(1, 19) = 8.8, p < .005). Participants were significantly more accurate and responded more quickly for the longest interval than the shortest one (performance: t(19) = 4.8, p < .005, RT: t(19) = 4.8, p < .001), yet performance was largely comparable to the imaging study (Table 1). Thus, these findings suggest that participants likely employed a recency-based retrieval strategy in both imaging and behavioral studies.

Together, our behavioral data indicate that participants used different behavioral strategies to perform the recency versus associative retrieval tasks. For the associative retrieval task, participants were less accurate and took longer to respond for items further apart than closer together. These findings suggest that they used an associative mechanism to remember temporal distances in this task, consistent with our hypotheses and past findings from Kyle et al. (2015). In contrast, for recency judgments, participants performed better and responded faster for more distant than nearby temporal elements. This in turn suggests that participants used a qualitatively different strategy to retrieve temporal distance information when making a recency decision. Thus, our findings support our hypothesis that cognitively, at least, participants used different strategies to retrieve temporal order in these two instances based on the retrieval probes that we employed (i.e., were items the same vs. different distance from the reference item vs. which of the two items was closer to the reference item?).

Imaging Data

Task-specific Patterns of Activation

We first conducted a whole-brain analysis (reported at p < .05 cluster-corrected, see Methods) to identify task-specific patterns of brain activation and regions of activation that were common to the recency and associative temporal retrieval tasks. This analysis collapsed across all trials and included both correct and incorrect trials. We separately contrasted each task with the implicit baseline, as well as directly contrasting the retrieval tasks (e.g., associative > recency and recency > associative).

The Task versus Baseline contrasts exhibited several common areas of activation, including the bilateral posterior hippocampus (Figure 2; Tables 2 and 3). Within hemispheres, the hippocampal activation clusters from the two baseline contrasts largely overlapped. Examination of these clusters (extracted from one-sample t tests of each condition separately) revealed that hippocampal activation was significantly greater than baseline in both tasks, but there were no significant differences in activation between the associative and recency conditions (p = .1308). These data demonstrate that hippocampal activation was statistically indistinguishable during recency- and association-based temporal order retrieval.

Figure 2. 

Brain regions active during associative retrieval (A) or recency-based retrieval (B). Each task was contrasted with the implicit baseline. Analysis revealed clusters within the hippocampus for both tasks, as well as multiple additional brain regions. An ROI analysis of hippocampal clusters (C) revealed no significant differences in percent signal change between tasks. Percent signal change results are collapsed across hemispheres.

Figure 2. 

Brain regions active during associative retrieval (A) or recency-based retrieval (B). Each task was contrasted with the implicit baseline. Analysis revealed clusters within the hippocampus for both tasks, as well as multiple additional brain regions. An ROI analysis of hippocampal clusters (C) revealed no significant differences in percent signal change between tasks. Percent signal change results are collapsed across hemispheres.

Table 2. 

Clusters of Activation for Associative > Baseline Contrast

ConditionAssociative > Baseline
RegionxyzZCluster SizeCluster Peak
Medial-temporal lobe Posterior hippocampus −22 −30 −2 6.14 149 12.15 
24 −28 −4 6.04 218 11.16 
Occipital lobe Intracalcarine cortex −6 −76 5.92 9772 11.06 
Temporal lobe Temporal occipital fusiform cortex −26 −46 −16 5.47 445 9.30 
Inferior temporal gyrus 56 −54 −12 4.51 55 6.40 
Insula  32 22 −2 5.19 269 8.30 
−28 24 4.86 954 7.36 
Frontal lobe Middle frontal gyrus 32 52 5.12 453 8.09 
44 30 24 4.43 169 6.21 
−32 60 4.90 350 7.36 
−48 28 32 4.87 954 7.33 
Frontal pole −42 50 −4 3.97 68 5.20 
Cingulate Paracingulate gyrus 20 40 5.03 517 7.82 
Motor Precentral gyrus 42 28 4.42 134 6.19 
ConditionAssociative > Baseline
RegionxyzZCluster SizeCluster Peak
Medial-temporal lobe Posterior hippocampus −22 −30 −2 6.14 149 12.15 
24 −28 −4 6.04 218 11.16 
Occipital lobe Intracalcarine cortex −6 −76 5.92 9772 11.06 
Temporal lobe Temporal occipital fusiform cortex −26 −46 −16 5.47 445 9.30 
Inferior temporal gyrus 56 −54 −12 4.51 55 6.40 
Insula  32 22 −2 5.19 269 8.30 
−28 24 4.86 954 7.36 
Frontal lobe Middle frontal gyrus 32 52 5.12 453 8.09 
44 30 24 4.43 169 6.21 
−32 60 4.90 350 7.36 
−48 28 32 4.87 954 7.33 
Frontal pole −42 50 −4 3.97 68 5.20 
Cingulate Paracingulate gyrus 20 40 5.03 517 7.82 
Motor Precentral gyrus 42 28 4.42 134 6.19 
Table 3. 

Clusters of Activation for Recency > Baseline Contrast

ConditionRecency > Baseline
RegionxyzZCluster SizeCluster Peak
Medial-temporal lobe Posterior hippocampus −22 −28 6.53 205 14.32 
24 −28 −4 6.20 233 12.48 
Temporal lobe Temporal occipital fusiform cortex −26 −46 −14 6.01 528 11.51 
26 −50 −14 5.58 826 9.72 
Precuneus  −20 −66 26 5.89 8977 10.96 
Cingulate Posterior −8 −42 22 5.35 85 8.87 
Insula  −30 20 5.22 212 8.41 
Frontal lobe Middle frontal gyrus −46 28 32 5.00 574 7.75 
Orbitofrontal cortex 30 30 −2 4.71 150 6.91 
Superior frontal gyrus −20 56 3.76 69 4.80 
28 60 3.71 124 4.70 
Putamen  28 −16 4.13 103 5.53 
ConditionRecency > Baseline
RegionxyzZCluster SizeCluster Peak
Medial-temporal lobe Posterior hippocampus −22 −28 6.53 205 14.32 
24 −28 −4 6.20 233 12.48 
Temporal lobe Temporal occipital fusiform cortex −26 −46 −14 6.01 528 11.51 
26 −50 −14 5.58 826 9.72 
Precuneus  −20 −66 26 5.89 8977 10.96 
Cingulate Posterior −8 −42 22 5.35 85 8.87 
Insula  −30 20 5.22 212 8.41 
Frontal lobe Middle frontal gyrus −46 28 32 5.00 574 7.75 
Orbitofrontal cortex 30 30 −2 4.71 150 6.91 
Superior frontal gyrus −20 56 3.76 69 4.80 
28 60 3.71 124 4.70 
Putamen  28 −16 4.13 103 5.53 

We next sought to determine whether there were brain regions specific to either form of temporal order retrieval in a direct whole-brain contrast. Contrasting the retrieval conditions revealed unique activation patterns for each task. The associative > recency contrast revealed large activation clusters in the supramarginal gyri in parietal cortex (Figure 3A), as well as several smaller clusters outside these two primary areas, including occipital cortex and pFC, paracingulate and precentral gyri, and the insula (Table 4). In the recency > associative contrast, we found clusters of activation in the frontal poles and perirhinal cortex (Figure 3B, C). We did not find significant hippocampal activation in either recency > associative or associative > recency contrasts, supporting our previous ROI analysis. Therefore, these findings suggest that associative and recency tasks evoke unique patterns of activation across the cortex and insula. The lack of hippocampal activation in the direct contrasts suggests its involvement in both forms of retrieval, consistent with the baseline contrast analysis.

Figure 3. 

Brain regions uniquely active to either associative retrieval or recency-based retrieval, but not both. We conducted two analyses collapsed across all trials, which revealed three primary brain regions that were task-specific. Comparing associative > recency blocks showed clusters of activation in the superior parietal cortex (A). Comparing recency > associative blocks showed clusters of activation in the pFC (B) and the perirhinal cortex (C). All percent signal change results are collapsed across hemispheres. ***p < .001.

Figure 3. 

Brain regions uniquely active to either associative retrieval or recency-based retrieval, but not both. We conducted two analyses collapsed across all trials, which revealed three primary brain regions that were task-specific. Comparing associative > recency blocks showed clusters of activation in the superior parietal cortex (A). Comparing recency > associative blocks showed clusters of activation in the pFC (B) and the perirhinal cortex (C). All percent signal change results are collapsed across hemispheres. ***p < .001.

Table 4. 

Clusters of Activation for Associative > Recency and Recency > Associative Contrasts

ConditionAssociative > RecencyRecency > Associative
RegionxyzZCluster SizeCluster PeakxyzZCluster SizeCluster Peak
Parietal lobe Supramarginal gyrus −48 −40 50 5.21 409 8.38       
44 −38 46 5.09 810 8.00       
Motor Precentral gyrus 44 30 4.45 41 6.77       
34 −4 52 4.12 114 5.51       
Frontal lobe Frontal pole       −6 58 14 4.45 64 6.26 
      60 3.89 44 5.04 
Superior frontal gyrus −16 10 54 4.03 76 5.32       
Middle frontal gyrus −48 28 32 3.90 50 5.07       
Inferior frontal gyrus 54 12 28 3.86 65 4.98       
Medial-temporal lobe Perirhinal cortex       −28 −30 −14 4.28 60 5.86 
      32 −32 −8 4.04 49 5.35 
Occipital lobe Lateral occipital complex 32 −62 44 4.21 77 5.70       
Cingulate Paracingulate gyrus 22 46 3.99 48 5.24       
Insula  34 22 −2 3.67 57 4.63       
ConditionAssociative > RecencyRecency > Associative
RegionxyzZCluster SizeCluster PeakxyzZCluster SizeCluster Peak
Parietal lobe Supramarginal gyrus −48 −40 50 5.21 409 8.38       
44 −38 46 5.09 810 8.00       
Motor Precentral gyrus 44 30 4.45 41 6.77       
34 −4 52 4.12 114 5.51       
Frontal lobe Frontal pole       −6 58 14 4.45 64 6.26 
      60 3.89 44 5.04 
Superior frontal gyrus −16 10 54 4.03 76 5.32       
Middle frontal gyrus −48 28 32 3.90 50 5.07       
Inferior frontal gyrus 54 12 28 3.86 65 4.98       
Medial-temporal lobe Perirhinal cortex       −28 −30 −14 4.28 60 5.86 
      32 −32 −8 4.04 49 5.35 
Occipital lobe Lateral occipital complex 32 −62 44 4.21 77 5.70       
Cingulate Paracingulate gyrus 22 46 3.99 48 5.24       
Insula  34 22 −2 3.67 57 4.63       

Analysis Including a Performance-based Covariate

Because performance differed behaviorally between recency and associative temporal retrieval conditions, we conducted a whole-brain analysis across all trials in which participants' performance within task categories were included as covariates. Although average performance was significantly above chance for all task conditions, there was still a range of variance in performance in individual conditions, particularly for large interval trials during the associative task. We thus included each participant's performance for the four different possible conditions (associative-short interval, associative-long interval, recency-short interval, recency-long interval). As in previous analyses, we contrasted each task with baseline, as well as directly contrasting the two tasks.

In both the individual task and direct cross-task contrasts, activation patterns after controlling for effects of performance were largely consistent with the results of the all-trials analysis, suggesting that performance did not have a significant effect on brain activation for either form of retrieval. In both of the Task > Baseline contrasts, there were slight changes in activation within the temporal lobe. During the associative > baseline contrast, a small cluster within the right inferior temporal gyrus was not significantly activated after controlling for performance, whereas during recency > baseline, a cluster within the left superior temporal gyrus was active only during the covariate analysis. The cross-task contrasts showed small changes within the frontal lobe and insula; clusters within the left middle frontal gyrus and right insula (associative > recency) and right frontal pole (recency > associative) were no longer significant after including the performance covariate. Importantly, including the covariate did not affect larger clusters; the only variable clusters were those smaller than or equal to 57 voxels. Critically, when including performance covariates, all other activations from the primary series of contrasts remained significantly active during this analysis. This included the hippocampus, which was present in both baseline contrasts when including covariates for performance (Table 5).

Table 5. 

Clusters of Activation for Task > Baseline and Task Comparison Contrasts with Performance Covariate

ConditionAssociative > Baseline
RegionxyzZCluster SizeCluster Peak
Medial-temporal lobe Posterior hippocampus −22 −30 −2 5.90 137 12.92 
24 −28 −4 5.68 237 11.68 
Occipital lobe Intracalcarine cortex −6 −76 5.86 8464 12.67 
Cingulate Paracingulate gyrus 20 40 5.45 460 10.52 
Temporal lobe Temporal occipital fusiform cortex −32 −48 −14 5.22 372 9.50 
Insula  −32 22 5.06 202 8.85 
32 22 −2 4.76 234 7.77 
Frontal lobe Superior frontal gyrus 24 62 4.81 393 7.95 
Middle frontal gyrus −44 28 32 4.81 815 7.93 
 −30 50 4.64 265 7.36 
Frontal pole 48 40 24 4.19 143 6.08 
−40 44 −2 3.72 49 4.98 
Motor Precentral gyrus 42 28 4.20 142 6.12 
 
Condition Recency > Baseline 
Region x y z Z Cluster Size Cluster Peak 
Medial-temporal lobe Posterior hippocampus −22 −28 6.34 229 14.81 
24 −28 −4 5.91 218 12.20 
Occipital lobe Lateral occipital complex −28 −70 38 5.76 8633 11.44 
Cingulate Posterior −8 −42 22 5.03 71 8.36 
Temporal lobe Temporal occipital fusiform cortex −24 −44 −16 5.88 504 12.05 
26 −50 −14 5.38 783 9.70 
Superior temporal gyrus −52 −34 4.77 55 7.52 
Insula  −30 20 5.07 184 8.53 
Frontal Superior frontal gyrus −20 58 3.76 50 4.97 
Middle frontal gyrus −46 28 32 4.88 503 7.86 
40 60 3.75 106 4.94 
Orbitofrontal cortex 30 30 −2 4.47 113 6.64 
Putamen  28 12 3.96 56 5.38 
 
Condition Associative > Recency 
Region x y z Z Cluster Size Cluster Peak 
Parietal lobe Supramarginal gyrus 34 −38 38 5.13 637 10.19 
−48 −38 50 4.62 325 7.97 
Motor Precentral gyrus 52 18 4.62 81 7.97 
44 30 4.38 71 7.15 
Occipital lobe Lateral occipital complex 32 −64 48 4.50 134 7.53 
Frontal lobe Middle frontal gyrus 28 10 48 4.41 59 7.23 
Superior frontal gyrus −20 12 60 4.04 74 6.10 
 
Condition Recency > Associative 
Region x y z Z Cluster Size Cluster Peak 
Frontal lobe Frontal pole −6 56 14 4.49 63 7.52 
Medial-temporal lobe Parahippocampal gyrus −30 −28 −14 4.37 46 7.11 
28 −24 −14 4.24 53 6.69 
ConditionAssociative > Baseline
RegionxyzZCluster SizeCluster Peak
Medial-temporal lobe Posterior hippocampus −22 −30 −2 5.90 137 12.92 
24 −28 −4 5.68 237 11.68 
Occipital lobe Intracalcarine cortex −6 −76 5.86 8464 12.67 
Cingulate Paracingulate gyrus 20 40 5.45 460 10.52 
Temporal lobe Temporal occipital fusiform cortex −32 −48 −14 5.22 372 9.50 
Insula  −32 22 5.06 202 8.85 
32 22 −2 4.76 234 7.77 
Frontal lobe Superior frontal gyrus 24 62 4.81 393 7.95 
Middle frontal gyrus −44 28 32 4.81 815 7.93 
 −30 50 4.64 265 7.36 
Frontal pole 48 40 24 4.19 143 6.08 
−40 44 −2 3.72 49 4.98 
Motor Precentral gyrus 42 28 4.20 142 6.12 
 
Condition Recency > Baseline 
Region x y z Z Cluster Size Cluster Peak 
Medial-temporal lobe Posterior hippocampus −22 −28 6.34 229 14.81 
24 −28 −4 5.91 218 12.20 
Occipital lobe Lateral occipital complex −28 −70 38 5.76 8633 11.44 
Cingulate Posterior −8 −42 22 5.03 71 8.36 
Temporal lobe Temporal occipital fusiform cortex −24 −44 −16 5.88 504 12.05 
26 −50 −14 5.38 783 9.70 
Superior temporal gyrus −52 −34 4.77 55 7.52 
Insula  −30 20 5.07 184 8.53 
Frontal Superior frontal gyrus −20 58 3.76 50 4.97 
Middle frontal gyrus −46 28 32 4.88 503 7.86 
40 60 3.75 106 4.94 
Orbitofrontal cortex 30 30 −2 4.47 113 6.64 
Putamen  28 12 3.96 56 5.38 
 
Condition Associative > Recency 
Region x y z Z Cluster Size Cluster Peak 
Parietal lobe Supramarginal gyrus 34 −38 38 5.13 637 10.19 
−48 −38 50 4.62 325 7.97 
Motor Precentral gyrus 52 18 4.62 81 7.97 
44 30 4.38 71 7.15 
Occipital lobe Lateral occipital complex 32 −64 48 4.50 134 7.53 
Frontal lobe Middle frontal gyrus 28 10 48 4.41 59 7.23 
Superior frontal gyrus −20 12 60 4.04 74 6.10 
 
Condition Recency > Associative 
Region x y z Z Cluster Size Cluster Peak 
Frontal lobe Frontal pole −6 56 14 4.49 63 7.52 
Medial-temporal lobe Parahippocampal gyrus −30 −28 −14 4.37 46 7.11 
28 −24 −14 4.24 53 6.69 

Effects of Interval Distance

To further identify brain regions modulated by temporal distance, we examined the effects of interval size both within and across tasks. For example, it might be possible that effects of interval distance rather than task mediated many of our univariate effects. For within-task analyses, the respective small- and large-interval trials for a given task were directly contrasted, whereas the small- and large-interval trials for both tasks were pooled for across-task contrasts. One cluster of activation was identified in the posterior supramarginal gyrus when contrasting associative 1-away > associative 2-away, whereas the associative 2-away > associative 1-away contrast exhibited multiple activation clusters in several regions, including the superior and middle frontal gyri, insula, precentral gyrus, and precuneus, as well as other parts of the posterior supramarginal gyrus. However, no effects of interval were observed for the recency task; both the small > large interval and large > small interval analyses had no significant activations. These data suggest that associative retrieval did result in some differences in brain areas recruited, particularly within the parietal lobe. These clusters, however, were largely restricted to the parietal lobe, where we saw the majority of significant effects for the associative versus baseline contrast in our previous analysis.

Bolstering this impression, across both tasks, large > small intervals (analyzed across associative and recency tasks) showed activations in the superior parietal lobule, precentral gyrus, supplementary motor cortex, superior lateral occipital cortex, precuneus, middle frontal gyrus, and insula, whereas no significant activations were found for the small > large interval contrast. Although these results suggest that brain activation during temporal order retrieval generally scales upward with temporal distance, we note that they are largely driven by activation patterns present in the associative temporal task. These findings thus reinforce the idea that associative temporal order retrieval is modulated by interval size, but the effects were largely localized to differences in parietal cortex activation and were largely a result of associative temporal order retrieval. Thus, interval differences alone were unlikely to be driving the primary effects we observed for the associative versus recency retrieval tasks.

Graph Theory Analysis of Networks Underlying Associative and Recency Conditions

Because hippocampal activation did not significantly differ between recency versus associative temporal order retrieval strategies, differential connectivity patterns between the hippocampus and neocortical areas might instead explain differences in behavior. To explore this issue, we employed graph theory in combination with task-related functional connectivity using the beta-time series technique (Schedlbauer et al., 2014; Rissman et al., 2004). We included all nodes (a total of 17, including bilateral hippocampus) that showed activation in either of the direct contrasts or in both of the baseline contrasts to better understand the connectivity patterns among brain areas revealed in our previous analyses. As shown in Figure 4A and B, both the right and left hippocampus showed significant levels of connectivity in both contrasts that did not differ overall (recency > associative: total node degree = 6, associative > recency: total node degree = 5). Instead, the network differed primarily in frontal versus parietal connectivity patterns, with the recency > associative network showing greater interfrontal and frontal MTL connectivity patterns overall and the associative > recency network showing greater inter-parietal-occipital and parietal-occipital-MTL connectivity patterns (Fisher's exact test, two-tailed p < .05). Overall, these findings suggest the hippocampus did not differ in overall connectivity for associative versus recency temporal order strategies, although, similar to our univariate analyses, we did find differences in the neocortical connectivity patterns.

Figure 4. 

(A, B) Recency versus associative networks plotted in MNI space and overlaid on a transparent brain showed connectivity differences to frontal, parietal, and occipital regions. Edges, or red lines, connect nodes, or the multicolored spheres indicating ROIs. Node degree is indicated by color with warmer colors showing higher connectivity. Abbreviations: L: left, R: right, ACG: paracingulate gyrus, FFG: temporal occipital fusiform cortex, FrPole: frontal pole, HPC: hippocampus, IFGoperc: inferior frontal gyrus, opercularis, INS: insula, MFG: middle frontal gyrus, MOG: lateral occipital cortex, PHG: parahippocampal gyrus, PreCG: precentral gyrus, SFG: superior frontal gyrus, Smar: supramarginal gyrus.

Figure 4. 

(A, B) Recency versus associative networks plotted in MNI space and overlaid on a transparent brain showed connectivity differences to frontal, parietal, and occipital regions. Edges, or red lines, connect nodes, or the multicolored spheres indicating ROIs. Node degree is indicated by color with warmer colors showing higher connectivity. Abbreviations: L: left, R: right, ACG: paracingulate gyrus, FFG: temporal occipital fusiform cortex, FrPole: frontal pole, HPC: hippocampus, IFGoperc: inferior frontal gyrus, opercularis, INS: insula, MFG: middle frontal gyrus, MOG: lateral occipital cortex, PHG: parahippocampal gyrus, PreCG: precentral gyrus, SFG: superior frontal gyrus, Smar: supramarginal gyrus.

DISCUSSION

Because past work has provided conflicting accounts regarding the involvement of different brain regions in temporal order retrieval strategies, the goal of this study was to compare the neural correlates of a recency-based strategy (which of two items occurred closer to a third item?) versus an associative-based strategy (were the two items an equal or different distance from a third item?). The study presented here is the first, to the best of our knowledge, to contrast these different types of retrieval strategies in the same episodic memory experiment using the same encoding strategies. Thus, we could investigate three different competing hypotheses: (1) the hippocampus would only be involved in association-based temporal order retrieval, (2) the hippocampus would only be involved in recency-based temporal order retrieval, or (3) the hippocampus would be involved in both forms of temporal order retrieval and additional extrahippocampal regions would differentiate between the two strategies.

Comparison of both tasks against baseline revealed significant levels of hippocampal activation that did not differ statistically from each other. Similarly, direct contrasts of recency > associative and associative > recency retrieval did not reveal significant differences in hippocampal activation, supporting the argument that it was equally active in both conditions. These findings persisted when including performance as a covariate, suggesting they could not be accounted for by the greater relative difficulty of the associative versus recency-based task. Additionally, we did not find differences in overall hippocampal connectivity patterns using a beta-time series approach, suggesting that the hippocampus was involved to comparable extents in both forms of temporal order retrieval. Together, these finding support the involvement of the hippocampus in both recency and associative forms of temporal order retrieval.

Our findings are both consistent and at odds with some past studies involving temporal order retrieval. Importantly, our findings for associative temporal order retrieval replicate previous findings from work that showed significant levels of hippocampal activity when participants retrieved the order of stores from a recently navigated spatial environment (Kyle et al., 2015). In particular, just like our findings reported here, the Kyle et al. study involved participants retrieving temporal order using an associative strategy in which they had to indicate whether two stores were the same or equal distance from a third reference store. We note that, to be consistent with Kyle et al., our task did involve store fronts, although our instructions explicitly encouraged participants to encode the order of objects and in no way encouraged a spatial strategy. We thus think that it is unlikely that participants used a navigation-based strategy to encode temporal order. There also does not appear to be a basis for thinking that a spatial encoding strategy would change associative (or recency-based) temporal order retrieval, although future experiments would be needed to explicitly address this issue (i.e., compare recency judgments for store fronts and common objects). Overall, our finding of hippocampal involvement when participants used an associative strategy to retrieve temporal order is consistent with other studies using common objects and thus together argue for the importance of the hippocampus in temporal order retrieval, particularly involving the processing of relational information (Brown, Hasselmo, & Stern, 2014; Kalm et al., 2013; Kimura et al., 2010; Konishi et al., 2006).

There is less consensus regarding whether the hippocampus is involved in recency types of temporal order judgments. Several fMRI and lesion studies involving working memory tasks (studying a small set of items and then indicating which of two items came first) typically have not revealed significant levels of hippocampal activation and instead have revealed frontal and parietal involvement (Kimura et al., 2010; Konishi et al., 2006; Marshuetz et al., 2000; Milner et al., 1991). In addition, studies of amnestic patients with significant hippocampal damage have found preserved ability to place temporal events on a timeline and judge the relative recency of personal events (Craver et al., 2014; Rosenbaum et al., 2005; Milner et al., 1991). However, other studies have found inconsistent findings, instead supporting the involvement of the hippocampus in recency judgments, even when detailed knowledge of the event is not required. Naya and Suzuki (2011), recording from neurons in the non-human primate hippocampus, found changes in firing rate based on recency. Specifically, neurons in the primate hippocampus fired preferentially if an object was presented first or second during retrieval, suggesting the involvement of the hippocampus in order coding (Naya & Suzuki, 2011). Similarly, Lehn and colleagues (2009) found that accuracy of temporal order recency judgments from a movie correlated with the degree of hippocampal activation (Lehn et al., 2009). Thus, past studies provide diverging results on whether the hippocampus is involved in different forms of temporal order retrieval, particularly those involving recency-based retrieval strategies. Our data suggest hippocampal involvement when participants retrieve longer sequences encoded in a more episodic fashion akin to Lehn et al. Our findings here are also consistent with past studies from the lab in which participants made judgments about which of two stores came closer to a third reference store, although these studies did not control for the position of the reference store and thus may not have consistently involved recency judgments in terms of comparing with either the end or beginning of the list (Copara et al., 2014; Ekstrom et al., 2011).

One possible explanation for why our study found significant levels of hippocampal involvement during recency judgments may relate to a proposed difference in temporal order retrieval proposed by Kimura et al. (2010). Specifically, the authors contrasted recency judgments (for two items) for trials involving at least one end item (first two or last two in a 10-item sequence) or trials involving recency judgments from items in the middle of the list. They demonstrated hippocampal activation during recency judgments for items in the middle of the list but not for judgments involving at least one item from the end of the list. Kimura et al. argued that this could arise because memory for temporal order of items in the middle of the list might require more relational/associative processing whereas those involving judgments at the end of the list would involve more item-based familiarity processing (Eichenbaum, Otto, & Cohen, 1994). Recency judgments in our task were always in reference to a third store at the beginning or end of the list, although the items being compared with the reference store (probe stores) therefore did come from the middle of the list. Kimura and colleagues did not collect behavioral data to demonstrate that participants were using a more relational strategy for recency retrieval of items in the middle of the list, and thus, it seems difficult to conclude that participants in fact were using a different strategy from the end items, although the differential brain activation patterns might support this idea. Thus, we cannot rule out that participants were employing some relational processing to retrieve recency information about middle items from the list in our study, although the clear behavioral and neural dissociations between our recency and associative task suggests that these relational strategies were unlikely to be identical, either.

Outside the hippocampus, whole-brain and graph theory analyses revealed extrahippocampal brain regions that differed for associative versus recency-based forms of retrieval. Specifically, we found that prefrontal and parietal areas differed as a function of associative versus recency forms of retrieval. In particular, our univariate analysis that included subject performance as a covariate revealed higher activation in parietal and some prefrontal areas for associative > recency retrieval whereas the frontal pole showed higher activation for recency > associative than judgments. An analysis of differences in connectivity patterns and activation during different intervals (1-away vs. 2-away) suggested overall greater involvement of parietal areas in associative based retrieval and prefrontal areas in recency-based judgments. These findings suggest the importance of frontal areas, in particular, for recency-based judgments, possibly in relation to coding familiarity types of signals often associated with recency judgments (Kimura et al., 2010; Konishi et al., 2006; Duarte, Ranganath, & Knight, 2005; Marshuetz et al., 2000; Milner et al., 1991). In contrast, parietal cortex is often associated with numerical judgments, which might be more important for judging the distance between different words involving a more associative form of temporal order retrieval (Pinel, Dehaene, Riviere, & LeBihan, 2001; Menon, Rivera, White, Glover, & Reiss, 2000). One possibility for why associative retrieval might show greater activation than strength-based retrieval is that our associative retrieval task required increased numerical processing because of the equality judgment implicit in the associative task (is A–B the same distance as B–C in triad A–B–C?), consistent with the concept of Buridan's principle (Lamport, 2012). Another possibility is that associative chaining simply places more demand on a parietal “output buffer” (Wagner, Shannon, Kahn, & Buckner, 2005); future studies will be needed to adjudicate between these possibilities.

Another potential explanation that runs counter to our arguments above is that the differences between associative and recency judgments in our task were mainly driven by differences in list length or task difficulty rather than differences in retrieval strategy. Our pilot behavioral experiments indicated that associative retrieval was significantly harder for participants than indicating which of the two stores occurred closer in temporal order to either beginning or end list items. In fact, the difference was so pronounced, we could only approximate performance by using a 15-item list for recency judgments and a 6-item list for associative judgments. Unlike any previous studies on temporal order processing (to our knowledge), the behavioral data indicated a significant interaction effect for RT, indicating that large intervals took longer to retrieve during the associative task whereas the opposite was true for recency judgments (which we subsequently confirmed in a separate behavioral study with judgments of larger intervals). This allowed us to directly compare conditions involving cognitively dissociable retrieval strategies. Additionally, we employed a covariate analysis in which we regressed out variance explained by performance. If differences in task difficulty alone accounted for activation differences, we would expect these to dissipate when controlling for differences in performance. Instead, activation patterns largely persisted, suggesting that activation patterns were largely a result of differences in retrieval strategy rather than task difficulty.

Given that past lesion studies have suggested that the hippocampus is not necessary for recency judgments, could it be that the hippocampus is involved yet not central to processing this type of temporal order information (Craver et al., 2014; Rosenbaum et al., 2005; Milner et al., 1991)? For example, it could be that the hippocampus receives temporal order information and thus activates during recency judgments yet may not be necessary for these judgments (Sarter, Berntson, & Cacioppo, 1996). Indeed, because the hippocampus receives multimodal (particularly visual) input from both perirhinal and parahippocampal cortex, temporal order information likely enters the hippocampus and under normal conditions such as the healthy volunteers studied here, the hippocampus is indeed involved in recency judgments. However, other structures outside the hippocampus may have sufficient neural machinery for recency judgments, thus explaining how lesions to the hippocampus would not affect recency judgments. It is also possible that extrahippocampal structures can compensate for lost hippocampal function, given sufficient time after lesion, but under typical conditions, the hippocampus plays a necessary role in recency judgments. Because lesion patients are typically studied well after their lesion occurred, during which plasticity and compensatory activity can occur (Alstott, Breakspear, Hagmann, Cammoun, & Sporns, 2009), we cannot be sure. Our findings do support the idea, however, that the primary differentiating factor between recency and associative judgments in our task was the degree of extrahippocampal activation and not hippocampal involvement per se. Thus, it seems possible that under normal conditions the hippocampus may play roles in both forms of temporal order processing, with regions outside the hippocampus playing more critical and different roles in these two types of judgments.

In conclusion, our data suggest a generalized role of hippocampal involvement in temporal order retrieval. Meanwhile, distinct temporal retrieval strategies are further supported by neocortical regions, including the prefrontal, parietal, and extrahippocampal cortices. Our findings help to clarify the roles of brain regions that have been implicated in temporal processing, as well as their interactions.

Reprint requests should be sent to Dr. Arne D. Ekstrom, Center for Neuroscience and Department of Psychology, University of California, Davis, Davis, CA 95618, or via e-mail: adekstrom@ucdavis.edu.

REFERENCES

Addis
,
K. M.
, &
Kahana
,
M. J.
(
2004
).
Decomposing serial learning: What is missing from the learning curve?
Psychonomic Bulletin and Review
,
11
,
118
124
.
Alstott
,
J.
,
Breakspear
,
M.
,
Hagmann
,
P.
,
Cammoun
,
L.
, &
Sporns
,
O.
(
2009
).
Modeling the impact of lesions in the human brain
.
PLoS Computational Biology
,
5
,
e1000408
.
Brett
,
M.
,
Anton
,
J.-L.
,
Valabregue
,
R.
, &
Poline
,
J.-B.
(
2002
).
Region of interest analysis using the MarsBar toolbox for SPM 99
.
Neuroimage
,
16
,
S497
.
Brown
,
T. I.
,
Hasselmo
,
M. E.
, &
Stern
,
C. E.
(
2014
).
A high-resolution study of hippocampal and medial temporal lobe correlates of spatial context and prospective overlapping route memory
.
Hippocampus
,
24
,
819
839
.
Copara
,
M. S.
,
Hassan
,
A. S.
,
Kyle
,
C. T.
,
Libby
,
L. A.
,
Ranganath
,
C.
, &
Ekstrom
,
A. D.
(
2014
).
Complementary roles of human hippocampal subregions during retrieval of spatiotemporal context
.
Journal of Neuroscience
,
34
,
6834
6842
.
Craver
,
C. F.
,
Kwan
,
D.
,
Steindam
,
C.
, &
Rosenbaum
,
R. S.
(
2014
).
Individuals with episodic amnesia are not stuck in time
.
Neuropsychologia
,
57
,
191
195
.
Downes
,
J. J.
,
Mayes
,
A. R.
,
MacDonald
,
C.
, &
Hunkin
,
N. M.
(
2002
).
Temporal order memory in patients with Korsakoff's syndrome and medial temporal amnesia
.
Neuropsychologia
,
40
,
853
861
.
Duarte
,
A.
,
Henson
,
R. N.
,
Knight
,
R. T.
,
Emery
,
T.
, &
Graham
,
K. S.
(
2010
).
Orbito-frontal cortex is necessary for temporal context memory
.
Journal of Cognitive Neuroscience
,
22
,
1819
1831
.
Duarte
,
A.
,
Ranganath
,
C.
, &
Knight
,
R. T.
(
2005
).
Effects of unilateral prefrontal lesions on familiarity, recollection, and source memory
.
Journal of Neuroscience
,
25
,
8333
8337
.
Eichenbaum
,
H.
,
Otto
,
T.
, &
Cohen
,
N. J.
(
1994
).
Two functional components of the hippocampal memory system
.
Behavioral and Brain Sciences
,
17
,
449
472
.
Ekstrom
,
A. D.
,
Copara
,
M. S.
,
Isham
,
E. A.
,
Wang
,
W. C.
, &
Yonelinas
,
A. P.
(
2011
).
Dissociable networks involved in spatial and temporal order source retrieval
.
Neuroimage
,
2011
,
18
.
Ezzyat
,
Y.
, &
Davachi
,
L.
(
2014
).
Similarity breeds proximity: Pattern similarity within and across contexts is related to later mnemonic judgments of temporal proximity
.
Neuron
,
81
,
1179
1189
.
Farovik
,
A.
,
Dupont
,
L. M.
, &
Eichenbaum
,
H.
(
2010
).
Distinct roles for dorsal CA3 and CA1 in memory for sequential nonspatial events
.
Learning and Memory
,
17
,
12
17
.
Frackowiak
,
R. S.
,
Friston
,
K. J.
,
Frith
,
C. D.
,
Dolan
,
R. J.
,
Price
,
C. J.
,
Zeki
,
S.
, et al
(
2004
).
Human brain function
.
Cambridge, MA
:
Academic Press
.
Friston
,
K. J.
,
Frith
,
C. D.
,
Frackowiak
,
R. S.
, &
Turner
,
R.
(
1995
).
Characterizing dynamic brain responses with fMRI: A multivariate approach
.
Neuroimage
,
2
,
166
172
.
Gauthier
,
B.
, &
van Wassenhove
,
V.
(
2016
).
Cognitive mapping in mental time travel and mental space navigation
.
Cognition
,
154
,
55
68
.
Hintzman
,
D. L.
(
1976
).
Repetition and memory
. In
G.
Bower
(Ed.),
Psychology of learning and motivation
(
Vol. 10
, pp.
47
91
).
Cambridge, MA
:
Academic Press
.
Hintzman
,
D. L.
(
2005
).
Memory strength and recency judgments
.
Psychonomic Bulletin and Review
,
12
,
858
864
.
Howard
,
M. W.
, &
Kahana
,
M. J.
(
2002
).
A distributed representation of temporal context
.
Journal of Mathematical Psychology
,
46
,
269
299
.
Howard
,
M. W.
, &
Natu
,
V. S.
(
2005
).
Place from time: Reconstructing position from a distributed representation of temporal context
.
Neural Networks
,
18
,
1150
1162
.
Hsieh
,
L. T.
,
Gruber
,
M. J.
,
Jenkins
,
L. J.
, &
Ranganath
,
C.
(
2014
).
Hippocampal activity patterns carry information about objects in temporal context
.
Neuron
,
81
,
1165
1178
.
Kahana
,
M. J.
(
1996
).
Associative retrieval processes in free recall
.
Memory & Cognition
,
24
,
103
109
.
Kalm
,
K.
,
Davis
,
M. H.
, &
Norris
,
D.
(
2013
).
Individual sequence representations in the medial temporal lobe
.
Journal of Cognitive Neuroscience
,
25
,
1111
1121
.
Kimura
,
H. M.
,
Hirose
,
S.
,
Kunimatsu
,
A.
,
Chikazoe
,
J.
,
Jimura
,
K.
,
Watanabe
,
T.
, et al
(
2010
).
Differential temporo-parietal cortical networks that support relational and item-based recency judgments
.
Neuroimage
,
49
,
3474
3480
.
Klein
,
K. A.
,
Addis
,
K. M.
, &
Kahana
,
M. J.
(
2005
).
A comparative analysis of serial and free recall
.
Memory & Cognition
,
33
,
833
839
.
Konishi
,
S.
,
Asari
,
T.
,
Jimura
,
K.
,
Chikazoe
,
J.
, &
Miyashita
,
Y.
(
2006
).
Activation shift from medial to lateral temporal cortex associated with recency judgements following impoverished encoding
.
Cerebral Cortex
,
16
,
469
474
.
Kriegeskorte
,
N.
,
Simmons
,
W. K.
,
Bellgowan
,
P. S.
, &
Baker
,
C. I.
(
2009
).
Circular analysis in systems neuroscience: The dangers of double dipping
.
Nature Neuroscience
,
12
,
535
540
.
Kyle
,
C. T.
,
Smuda
,
D. N.
,
Hassan
,
A. S.
, &
Ekstrom
,
A. D.
(
2015
).
Roles of human hippocampal subfields in retrieval of spatial and temporal context
.
Behavioural Brain Research
,
278C
,
549
558
.
Lamport
,
L.
(
2012
).
Buridan's principle
.
Foundations of Physics
,
42
,
1056
1066
.
Lehn
,
H.
,
Steffenach
,
H. A.
,
van Strien
,
N. M.
,
Veltman
,
D. J.
,
Witter
,
M. P.
, &
Haberg
,
A. K.
(
2009
).
A specific role of the human hippocampus in recall of temporal sequences
.
Journal of Neuroscience
,
29
,
3475
3484
.
Marshuetz
,
C.
,
Smith
,
E. E.
,
Jonides
,
J.
,
DeGutis
,
J.
, &
Chenevert
,
T. L.
(
2000
).
Order information in working memory: fMRI evidence for parietal and prefrontal mechanisms
.
Journal of Cognitive Neuroscience
,
12(Suppl. 2)
,
130
144
.
Menon
,
V.
,
Rivera
,
S. M.
,
White
,
C. D.
,
Glover
,
G. H.
, &
Reiss
,
A. L.
(
2000
).
Dissociating prefrontal and parietal cortex activation during arithmetic processing
.
Neuroimage
,
12
,
357
365
.
Milner
,
B.
,
Corsi
,
P.
, &
Leonard
,
G.
(
1991
).
Frontal-lobe contribution to recency judgements
.
Neuropsychologia
,
29
,
601
618
.
Moyer
,
R. S.
, &
Landauer
,
T. K.
(
1967
).
Time required for judgements of numerical inequality
.
Nature
,
215
,
1519
1520
.
Murdock
,
B. B.
, Jr.
(
1968
).
Serial order effects in short-term memory
.
Journal of Experimental Psychology
,
76(Suppl.)
,
1
15
.
Murdock
,
B. B.
, Jr.
(
1974
).
Human memory: Theory and data
.
Potomac, MD
:
Lawrence Erlbaum Associates
.
Murdock
,
B. B.
, Jr.
, &
Okada
,
R.
(
1970
).
Interresponse times in single-trial free recall
.
Journal of Experimental Psychology
,
86
,
263
267
.
Naya
,
Y.
, &
Suzuki
,
W. A.
(
2011
).
Integrating what and when across the primate medial temporal lobe
.
Science
,
333
,
773
776
.
Nielson
,
D. M.
,
Smith
,
T. A.
,
Sreekumar
,
V.
,
Dennis
,
S.
, &
Sederberg
,
P. B.
(
2015
).
Human hippocampus represents space and time during retrieval of real-world memories
.
Proceedings of the National Academy of Sciences, U.S.A.
,
112
,
11078
11083
.
Pinel
,
P.
,
Dehaene
,
S.
,
Riviere
,
D.
, &
LeBihan
,
D.
(
2001
).
Modulation of parietal activation by semantic distance in a number comparison task
.
Neuroimage
,
14
,
1013
1026
.
Rissman
,
J.
,
Gazzaley
,
A.
, &
D'Esposito
,
M.
(
2004
).
Measuring functional connectivity during distinct stages of a cognitive task
.
Neuroimage
,
23
,
752
763
.
Rosenbaum
,
R. S.
,
Gilboa
,
A.
,
Levine
,
B.
,
Winocur
,
G.
, &
Moscovitch
,
M.
(
2009
).
Amnesia as an impairment of detail generation and binding: Evidence from personal, fictional, and semantic narratives in K.C.
Neuropsychologia
,
47
,
2181
2187
.
Rosenbaum
,
R. S.
,
Kohler
,
S.
,
Schacter
,
D. L.
,
Moscovitch
,
M.
,
Westmacott
,
R.
,
Black
,
S. E.
, et al
(
2005
).
The case of K.C.: Contributions of a memory-impaired person to memory theory
.
Neuropsychologia
,
43
,
989
1021
.
Sarter
,
M.
,
Berntson
,
G. G.
, &
Cacioppo
,
J. T.
(
1996
).
Brain imaging and cognitive neuroscience: Toward strong inference in attributing function to structure
.
American Psychologist
,
51
,
13
.
Schedlbauer
,
A. M.
,
Copara
,
M. S.
,
Watrous
,
A. J.
, &
Ekstrom
,
A. D.
(
2014
).
Multiple interacting brain areas underlie successful spatiotemporal memory retrieval in humans
.
Scientific Reports
,
4
,
6431
.
Solway
,
A.
,
Murdock
,
B. B.
, &
Kahana
,
M. J.
(
2012
).
Positional and temporal clustering in serial order memory
.
Memory & Cognition
,
40
,
177
190
.
Stark
,
C. E.
, &
Squire
,
L. R.
(
2001
).
When zero is not zero: The problem of ambiguous baseline conditions in fMRI
.
Proceedings of the National Academy of Sciences, U.S.A.
,
98
,
12760
12766
.
Tulving
,
E.
(
2002
).
Episodic memory: From mind to brain
.
Annual Review of Psychology
,
53
,
1
25
.
Wagner
,
A. D.
,
Shannon
,
B. J.
,
Kahn
,
I.
, &
Buckner
,
R. L.
(
2005
).
Parietal lobe contributions to episodic memory retrieval
.
Trends in Cognitive Sciences
,
9
,
445
453
.
Wang
,
F.
, &
Diana
,
R. A.
(
2016
).
Temporal context processing within hippocampal subfields
.
Neuroimage
,
134
,
261
269
.
Xia
,
M.
,
Wang
,
J.
, &
He
,
Y.
(
2013
).
BrainNet Viewer: A network visualization tool for human brain connectomics
.
PLoS ONE
,
8
,
e68910
.
Yntema
,
D. B.
, &
Trask
,
F. P.
(
1963
).
Recall as a search process
.
Journal of Verbal Learning and Verbal Behavior
,
2
,
65
74
.
Yonelinas
,
A. P.
,
Kroll
,
N. E.
,
Quamme
,
J. R.
,
Lazzara
,
M. M.
,
Sauve
,
M. J.
,
Widaman
,
K. F.
, et al
(
2002
).
Effects of extensive temporal lobe damage or mild hypoxia on recollection and familiarity
.
Nature Neuroscience
,
5
,
1236
1241
.