Abstract

It is well established that the hippocampus is critical for memory. Recent evidence suggests that one function of hippocampal memory processing is to optimize how people actively explore the world. Here we demonstrate that the link between the hippocampus and exploration extends even to the moment-to-moment use of eye movements during visuospatial memory encoding. In Experiment 1, we examined relationships between study-phase eye movements in healthy individuals and subsequent performance on a spatial reconstruction test. In addition to quantitative measures of viewing behaviors (e.g., how many fixations or saccades were deployed during study), we used the information–theoretic measure of entropy to assess the amount of randomness or disorganization in participants' scanning behaviors. We found that the use of scanpaths during study that were lower in entropy (e.g., more organized, less random) predicted more accurate spatial reconstruction both within and between participants. Scanpath entropy was a better predictor of reconstruction accuracy than were the quantitative measures of viewing. In Experiment 2, we found that individuals with hippocampal amnesia tended to engage in viewing patterns that were higher in entropy (less organized) relative to healthy comparisons. These findings reveal a critical role of the hippocampus in guiding eye movement exploration to optimize visuospatial relational memory.

INTRODUCTION

The discovery that damage to the hippocampus results in profound anterograde amnesia (Scoville & Milner, 1957) was a watershed moment in the study of human memory. This finding spurred decades of research examining hippocampal contributions to human memory abilities. Although theories regarding the precise nature of these contributions vary, there is now relative agreement that the hippocampus is particularly important for memory that involves associations or relations among elements of experience (e.g., Moscovitch et al., 2005; Norman & O'Reilly, 2003; Yonelinas, 2002; Cohen & Eichenbaum, 1993).

Around the same time, another landmark finding occurred in the form of “place cells” or neurons in the rodent hippocampus that respond to specific spatial locations (O'Keefe & Dostrovsky, 1971) and code for future navigational goals (Johnson & Redish, 2007). A second, somewhat independent literature emerged from this discovery, in which the hippocampus is positioned primarily as a cognitive mapping tool that underlies spatial learning and promotes the ability to navigate efficiently in large-scale space (Hartley, Lever, Burgess, & O'Keefe, 2013; McNaughton, Battaglia, Jensen, Moser, & Moser, 2006; McNaughton et al., 1996). For example, hippocampal place cells have been linked to path integration, or the process of tracking location from self-motion cues (McNaughton et al., 1996). In addition, neuronal ensembles in navigating rodents exhibit “prospective coding” or the preemptive firing of patterns of activity representing possible future routes (Pfeiffer & Foster, 2013; Johnson & Redish, 2007). This phenomenon of hippocampal “preplay” correlates with the use of deliberative behaviors such as repetitive head turning (Johnson & Redish, 2007), indicating a process by which upcoming navigation choices are simulated and evaluated before action.

In recent years, interest has grown in synthesizing the body of research linking the hippocampus to relational memory processing in humans with that connecting it to spatial learning and navigation in rodents (Eichenbaum & Cohen, 2014). However, considerable methodological differences make it difficult to draw parallels between these two literatures. One key difference relates to the qualitatively different dependent measures that have traditionally been used in studies of humans versus nonhuman animals. Rodent studies in particular tend to assess the process by which information is studied by observing the use of exploratory behaviors, such as running, head turning, whisking, and sniffing. As such, the effects of hippocampal damage on learning and memory are evident primarily insofar as they impact the volitional use of such behaviors (e.g., Johnson, Varberg, Benhardus, Maahs, & Schrater, 2012). By contrast, the overwhelming majority of human memory studies are designed such that participants passively receive to-be-remembered information with minimal opportunities to influence how a given study episode unfolds. Said differently, memory in humans is treated more as an endpoint than as an active, exploratory process. As a consequence, we know relatively little about exploratory learning behaviors in humans and even less about the role of the hippocampus in shaping exploration.

A small number of studies have taken steps to bring paradigms used to study human memory in closer alignment with animal work by including “active” learning conditions, in which human participants navigate virtual environments (Daugherty et al., 2015; Kaplan et al., 2012) or otherwise exert control over the flow of to-be-remembered information (Yee et al., 2014; Voss, Gonsalves, Federmeier, Tranel, & Cohen, 2011; Voss, Warren, et al., 2011). Importantly, a consistent and illuminating finding from these initial studies is that the hippocampus appears to be critical for humans to use active learning opportunities to their own advantage. In one set of studies in particular (Voss, Gonsalves, et al., 2011; Voss, Warren, et al., 2011), participants were asked to study dense spatial arrays of objects that were occluded so that only one could be viewed at a time through a moving window. Successful subsequent memory correlated with the volitional use of a specific exploratory study behavior, namely the spontaneous “revisitation” of recently studied items via back-and-forth movements of the window. In an fMRI experiment included in the same article, the use of spontaneous revisitation was associated with enhanced hippocampal activity, as well as increased connectivity of the hippocampus to a network of frontal and cerebellar regions. Moreover, patients with amnesia due to hippocampal damage rarely engaged in spontaneous visitation, suggesting that the hippocampus played a causal role in implementing this behavior.

In summary, the above findings suggest that, as with rodents, the human hippocampus may promote memory formation precisely by way of optimizing exploration patterns. Several researchers (Voss, Bridge, Cohen, & Walker, 2017; Redish, 2016; Palombo, Keane, & Verfaellie, 2015; Wang, Cohen, & Voss, 2015; Johnson et al., 2012) have thus proposed that memory and spatial exploration may share neuroanatomical substrates because they are manifestations of an iterative process by which (1) memory is used to constrain choices about what to explore and when and (2) exploration leads to the further strengthening and refinement of relevant memory representations. Of note, this account posits an expanded role of the hippocampus in aspects of online decision-making (e.g., the monitoring and adjusting of goal-directed behavior from moment to moment) that have been traditionally assigned to medial and lateral prefrontal cortex (for reviews, see Rubin, Schwarb, Lucas, Dulas, & Cohen, 2017; Wang et al., 2015). However, this aspect of hippocampal function can be viewed as resulting from—rather than additional to—its well-established role in memory processing.

Importantly, to account for the bulk of the literature on human memory, it is necessary to adopt an operational definition of exploration that extends well beyond the navigation of large-scale space. As previously mentioned, the use of paradigms that permit overt exploration of this type in humans has been the exception rather than the rule, and it is clear that the hippocampus makes important contributions to memory even when the flow of information is entirely deterministic. It has thus been proposed that the link between hippocampal–cortical interactions and exploration may broadly include the purposive use of sensory sampling (Voss et al., 2017; Arkley, Grant, Mitchinson, & Prescott, 2014; Jutras, Fries, & Buffalo, 2013) or even the purely cognitive “exploration” of internal memory stores (Wang et al., 2015; Johnson et al., 2012). A key challenge going forward, then, will be to find ways to assess and quantify exploration across this wide range of circumstances so as to empirically determine the boundaries of its dependence on the hippocampus.

Of particular interest to this study is the intuitive, but largely unconfirmed, notion that the hippocampus promotes the adaptive use of eye movements during otherwise passive memory tasks. Saccadic eye movements are the most ubiquitous form of information gathering in primates and indeed seem to enjoy a privileged role in modulating ongoing neural activity in the primate hippocampus (Liu, Shen, Olsen, & Ryan, 2017; Voss et al., 2017; Hoffman et al., 2013; Jutras et al., 2013). In this study, we provide the first direct test of the notion that this relationship is reciprocal, in that the hippocampus also provides input into how eye movements are deployed from moment to moment during individual encoding events. Such evidence would significantly extend the ability of exploration-based models of hippocampal function to account for the extant literature in humans.

A prerequisite for linking the hippocampus to effective saccadic exploration is to identify specific viewing patterns or behaviors that are predictive of subsequent memory in healthy individuals (e.g., to understand what effective saccadic exploration looks like in the context of a given memory task). As such, an additional goal of this study is to identify eye movement predictors of accurate performance on a test of visuospatial relational memory. Although this type of “subsequent memory” analysis has been widely used in neuroimaging investigations, only a small number of studies have applied this approach to eye movement data (for a review, see Meister & Buffalo, 2016). These studies—most of which have tested recognition memory for single items, scenes, or paired associates—have generally suggested a simple monotonic relationship between eye movements and memory formation, by which memory representations strengthen as fixations accrue. However, there is reason to believe that, in more complex memory situations (i.e., for spatial layouts involving relationships among several items), the relationship between eye movements and subsequent memory will likewise become more complex and nuanced. Indeed, research on spontaneous exploration in rodents suggests that decisions about how long to explore a stimulus depend on processes that are distinct from those that inform decisions about where to explore within a complex environment (Robertson, Eacott, & Easton, 2015; Johnson et al., 2012; Easton, Zinkivskay, & Eacott, 2009). Similarly, it is possible that, when the amount and complexity of information present in even a single visual scene is sufficiently large, the hippocampus will be called upon to constrain saccadic exploration patterns so as to promote efficient information gathering and optimize memory.

In two experiments, we test these ideas by (1) investigating whether relational memory for a series of complex visuospatial displays in healthy individuals can be predicted based on the number and/or constraint of eye movements made during study and (2) examining whether individuals with hippocampal damage exhibit suboptimal viewing in the course of individual study episodes, which would link these patients' memory deficits to impaired exploration. Both experiments utilized a spatial reconstruction task, in which participants studied multi-item spatial displays and then attempted to reconstruct each display from memory. This task was chosen because it has been shown to be exquisitely sensitive to hippocampal integrity (Horecka et al., 2018; Schwarb et al., 2017; Schwarb, Johnson, McGarry, & Cohen, 2016; Monti et al., 2015; Watson, Voss, Warren, Tranel, & Cohen, 2013) and yet affords no opportunities for self-directed learning aside from the ability to freely view the displays.

Note that, unlike many tasks used in human memory experiments (e.g., recognition and recall tests), spatial reconstruction tasks assess memory along a continuous response space, producing information-rich data for which accuracy can be “scored” or quantified in a myriad of ways. Early studies relied on a global measure of “misplacement” error, or the mean distance between each item's original and reconstructed location (e.g., Huttenlocher & Presson, 1979). However, a limitation of this measure is that it is highly nonspecific, in that it can reflect memory for any combination of relational information (e.g., memory for arbitrary relations among aspects of the display), information about the Gestalten “shape” formed by the display, and/or memory for a series of independent Cartesian coordinates. Accordingly, researchers interested in linking task performance to hippocampal function have moved toward the use of alternative metrics that more selectively index relational memory processing (Horecka et al., 2018; Watson et al., 2013). In this study, we focus on one such metric, which has been referred to as a “swap” error. A swap error refers to a reconstruction error in which a pair of objects is transposed in space, such that the x and y coordinates of the vector comprising the spatial relationship between two objects are reversed.1

Several prior studies have substantiated the sensitivity of this type of error to hippocampal integrity. Relative to global misplacement error, swap errors are disproportionately present in patients with hippocampal amnesia (Watson et al., 2013). Individual differences in swap error rates among healthy individuals have also been found to correlate with hippocampal volume (Monti et al., 2015) and with the microstructural integrity of hippocampal tissue (Schwarb et al., 2016, 2017). Moreover, because swap errors reflect failures of memory for the relative positioning among multiple items in the display, they may be particularly sensitive to properties of interitem scanpaths. Thus, insofar as a functional relationship exists between scanpath constraint and hippocampal-dependent relational memory processing, this relationship should manifest in the number of swap errors committed during spatial reconstruction.

To assess study-phase eye movement constraint, we used the information–theoretic measure of entropy to quantify the level of randomness or disorganization inherent in participants' item-to-item gaze transition patterns (scanpaths; see also Althoff & Cohen, 1999). Lower-entropy scanpaths correspond to viewing patterns that are more structured and orderly, whereas higher-entropy scanpaths more closely resemble random patterns of viewing. Thus, we predict that scanpath entropy at study will negatively relate to subsequent relational memory performance in healthy participants, reflecting the beneficial effects of constrained and organized exploration strategies. Moreover, we predict that scanpath entropy will be elevated in patients with amnesia due to hippocampal damage, reflecting a necessary role of hippocampal memory processing in optimizing saccadic exploration.

EXPERIMENT 1

Methods

Participants

Forty young adult participants (18–30 years old, 33 women) from the University of Illinois Urbana-Champaign and surrounding community completed the study. An additional five participants completed the experiment but were excluded from analyses because of poor eye-tracking data quality (see Eye-tracking Methods), and two were excluded because of an error rate on the spatial reconstruction task that was greater than 3 standard deviations above the mean. Participants were paid $8 per hour. All procedures were approved by the institutional review board at the University of Illinois Urbana-Champaign, and written consent was obtained from all participants.

Stimuli

Stimuli consisted of 40 visual displays, each containing six novel abstract line drawings (“objects”) presented in a different spatial configuration. Each object was 80 × 80 pixels in size and either blue or yellow in color. Object locations for each trial were assigned randomly, with the constraint that the distance between any two objects within a display had to be at least 280 pixels.

Procedure

Participants were given 16 sec to study each display, followed by a 4-sec delay in which a blank screen was presented (see Figure 1). A self-paced reconstruction test followed in which the items from the previous display appeared at the top of the screen, and participants used a mouse to click and drag each object in an attempt to reconstruct the original display as accurately as possible. Eye tracking occurred throughout the study and test phases. Displays were presented in eight blocks of five, and eye movement calibration was performed at the beginning of each block. Participants completed six practice trials before beginning the task. All stimuli were presented on a 21-in. monitor using the Presentation software (Neurobehavioral Systems). Participants were positioned in a chin rest 60 cm from the screen.

Figure 1. 

Sample trial from the spatial construction task. Arrows in the reconstruction panel denote an example of a “swap” error, in which the directionality of both the x and y coordinates relating two objects were reversed.

Figure 1. 

Sample trial from the spatial construction task. Arrows in the reconstruction panel denote an example of a “swap” error, in which the directionality of both the x and y coordinates relating two objects were reversed.

Eye-tracking Methods

Eye position was recorded at a rate of 1000 Hz using an Eyelink 1000 eye-tracking system (SR Research), which uses an online parsing system to segment continuous eye movement data into saccades and fixations. Motion (0.15° displacement), velocity (30°/sec), and acceleration (8000°/sec) thresholds were used to identify saccades. Events with very small pupils were classified as blinks. All other events that did not meet the threshold for saccades were classified as fixations.

A k-means clustering procedure was used to assign fixations to specific items (ROIs) individually for each trial and participant. The geographic centers of the locations of the six items contained in the study display served as the initial cluster centroid positions, and the k-means function in MATLAB 2014a (The MathWorks) was used to partition each participant's fixations into six clusters and recompute cluster centroids. Fixations that were located more than 2 standard deviations away from the centroid of their assigned cluster were then eliminated to remove the influence of fixations that were not clearly directed at any of the ROIs. A second iteration of the clustering procedure was implemented after the removal of the outliers, followed by a second round of outlier removal. In total, this procedure resulted in the removal of an average of 6% of fixations from each participant's data set.

Only trials meeting an objective set of criteria for eye-tracking quality were analyzed. Criteria were as follows: (1) total gaze time recorded accounted for at least 67% of the total trial duration; (2) of the total gaze time recorded, the participant spent at least 67% of that time looking at one of the six objects; and (3) the geographical centers of the clusters assigned to each object via the k-means algorithm were each no more than 200 pixels away from the center of the object's actual location. Participants (n = 5) for whom more than 50% of trials did not meet these criteria were excluded from analysis. On average, 38/40 trials were included in each participant's eye-tracking analyses (range = 21–40).

Analyses of eye movement data focused on the following variables (see Figure 2 for illustration): (1) the number of fixations made to the ROIs across the course of a trial; (2) the number of transitions made across the course of the trial (e.g., the number of times that each participant shifted gaze from one item to another); (3) the number of within-item fixations in between each transition (e.g., fixations dedicated to a given item before gaze transitioned to a different item); and (4) scanpath entropy, which measures the amount of entropy or randomness in the pattern of item-to-item transitions.

Figure 2. 

(A) Example of a hypothetical pattern of fixations across a display. Each circle represents a fixation, and the numbers indicate the order in which they occurred. The green circles denote transition fixations, or fixations in which gaze was transferred from one item in the display to another. The yellow circles denote within-item fixations. The first fixation is considered neither a transition fixation nor a within-item fixation. This hypothetical example includes seven transition fixations, which were preceded by 1, 0, 2, 0, 1, 0, and 0 within-item fixations, respectively. Thus, the average within-item fixations value for this trial would be 0.57. (B) Examples of scanpaths characterized by levels of entropy (H1) that are relatively high (top) versus relatively low (bottom). Entropy calculations were based on patterns of transition fixations (see text). The thickness of the green arrows drawn between each pair of objects is proportional to the number of times that the participant's gaze transitioned from one item to the other over the course of the study trial. To the right of each scanpath is the transition matrix created from the corresponding trial, which contains the proportion of gaze transitions that occurred from each item in the display to each other item. The numbers on the x and y axes correspond to the numeric labels of each of the six items on the displays to the left. Both of the scanpaths depicted here contain 24 transition fixations. However, note that in the case of the high-entropy scanpath, the transition fixations were distributed relatively evenly, resulting in a transition matrix that resembles a random distribution. By contrast, the bottom scanpath is characterized by the repeated sampling of a smaller number of transition patterns, resulting in a transition matrix that is lower in entropy.

Figure 2. 

(A) Example of a hypothetical pattern of fixations across a display. Each circle represents a fixation, and the numbers indicate the order in which they occurred. The green circles denote transition fixations, or fixations in which gaze was transferred from one item in the display to another. The yellow circles denote within-item fixations. The first fixation is considered neither a transition fixation nor a within-item fixation. This hypothetical example includes seven transition fixations, which were preceded by 1, 0, 2, 0, 1, 0, and 0 within-item fixations, respectively. Thus, the average within-item fixations value for this trial would be 0.57. (B) Examples of scanpaths characterized by levels of entropy (H1) that are relatively high (top) versus relatively low (bottom). Entropy calculations were based on patterns of transition fixations (see text). The thickness of the green arrows drawn between each pair of objects is proportional to the number of times that the participant's gaze transitioned from one item to the other over the course of the study trial. To the right of each scanpath is the transition matrix created from the corresponding trial, which contains the proportion of gaze transitions that occurred from each item in the display to each other item. The numbers on the x and y axes correspond to the numeric labels of each of the six items on the displays to the left. Both of the scanpaths depicted here contain 24 transition fixations. However, note that in the case of the high-entropy scanpath, the transition fixations were distributed relatively evenly, resulting in a transition matrix that resembles a random distribution. By contrast, the bottom scanpath is characterized by the repeated sampling of a smaller number of transition patterns, resulting in a transition matrix that is lower in entropy.

The procedure for calculating scanpath entropy is illustrated in part in Figure 2B (see also Althoff & Cohen, 1999). We first entered each participant's item-to-item transition patterns from each trial into a Markov transition matrix, which was constructed such that the columns represented the current focus of gaze and the rows represented the previously fixated object. The resulting matrix contains the proportion of total transitions that occurred from each item to each other item in the display. The amount of entropy contained within each matrix was calculated using the following formula:
H=i=1nPilog21Pi
where n is the number of cells in the matrix and P(i) is the relative probability of the event in a given cell. The summed H measure represents the total amount of entropy or randomness contained within the cells. We then performed a second calculation to control for the amount of entropy in the matrix that could be predicted based simply on the fact that certain items may have attracted more viewing time than others (thus making transitions between them more likely). The row and column totals represent the amount of entropy attributable to these zero-order effects. Thus, our correction procedure used the following formula:
H1=1rowtotals+columntotalsHmeanofrowandcolumntotals
The resulting measure of entropy ranges from 0 to 1, with 0 being complete constraint and 1 being complete entropy.

Analysis Strategy

Subsequent memory was operationalized as the number of times participants committed “swap” errors during spatial reconstruction (see Figure 1). A swap error was coded anytime the signs of both the x and y coordinates of the vector comprising the spatial relationship between two objects were reversed.

Statistical analyses were conducted using R software Version 3.2.1 (R Core Team, 2015). In a first set of analyses, we used Pearson's correlations to examine across-subject relationships between the aforementioned eye movement behaviors and subsequent memory. Eye movement variables that showed a significant relationship to accuracy were then subject to (1) paired t tests examining within-subject relationships between each individual eye movements and accuracy and (2) analyses that used mixed effects modeling to characterize the relationship between multiple of these eye movement variables and subsequent memory using each individual trial as an outcome. All statistical tests reported are two-tailed and use alpha = .05.

Results

Across-subject Correlational Analyses

Analyses of study-phase eye movements included the total number of fixations made to the six objects on each trial (“total fixations”), the number of fixations that specifically represented shifts of gaze from one object to another (“transition fixations”), the average number of fixations on a given item that were made before each transition fixation (“within-item fixations”), and the amount of entropy contained in object-to-object transition patterns (“scanpath entropy”). Figure 3 depicts across-participant relationships between the mean number of swap errors per trial (an inverse measure of accuracy) and per-trial averages of each eye movement measure. As shown, the average number of swap errors made by each participant correlated positively with both the mean number of transitions made at study, r(38) = .34, p = .03, and with mean scanpath entropy, r(38) = .37, p = .02. In addition, the mean number of within-item fixations correlated negatively with swap errors, r(38) = −.41, p = .01. Said differently, participants who made more swap errors also tended to engage in viewing patterns at study that were characterized by (1) more frequent transitions of eye gaze from one item to another, (2) fewer repeated fixations on the same item before each transition, and (3) item-to-item transition patterns that were less constrained and more random. The correlation between total fixation count and swap errors was not significant, r(38) = .04, p = .82.

Figure 3. 

Scatterplots depicting across-participant correlations between three aspects of study-phase eye movements (within-item fixations per transition, transition fixations per trial, scanpath entropy) and the average number of swap errors made per trial.

Figure 3. 

Scatterplots depicting across-participant correlations between three aspects of study-phase eye movements (within-item fixations per transition, transition fixations per trial, scanpath entropy) and the average number of swap errors made per trial.

Within-subject Analyses

Across all participants, 60% of trials contained no swap errors, and only 13% of trials contained more than two such errors. For this reason, we treat swap errors as a binary outcome in within-subject analyses and used paired t tests to compare trials that did and did not contain at least one such error. On average, 15 swap-present trials (range = 3–33) and 23 swap-absent trials per participant (range = 6–37) were included in these analyses. The results are shown in Figure 4. Consistent with the across-participant analyses, study-phase eye movements of trials associated with later swap errors were characterized by higher scanpath entropy, t(39) = 4.02, p < .001, Cohen's d = 0.64, and more transition fixations, t(39) = 3.78, p < .001, Cohen's d = 0.60, relative to trials that did not result in swap errors. A small but significant effect was also present for within-item fixations, by which trials associated with later swap errors had fewer within-item fixations per transition relative to trials not associated with later swap errors, t(39) = 2.11, p = .04, Cohen's d = 0.33.

Figure 4. 

Within-subject analyses comparing mean within-item fixations per transition, mean transition fixations per trial, and mean scanpath entropy between trials with and without subsequent swap errors (labeled “Swap-present” and “Swap-absent,” respectively). Error bars show ±1 standard error with between-subject variance removed (Morey, 2008).

Figure 4. 

Within-subject analyses comparing mean within-item fixations per transition, mean transition fixations per trial, and mean scanpath entropy between trials with and without subsequent swap errors (labeled “Swap-present” and “Swap-absent,” respectively). Error bars show ±1 standard error with between-subject variance removed (Morey, 2008).

Hierarchical Mixed-effect Modeling

The above analyses identified three eye movement behaviors—infrequent item-to-item transitions, more within-item fixations in between transitions, and highly constrained scanpaths—that predicted subsequent relational memory accuracy in both across-subject and within-subject analyses. However, these behaviors are very highly correlated across subjects (rs > .80), making it difficult to measure their relative contributions based on per-subject averages alone. We therefore supplemented the above subject-level analyses by using logit mixed effects modeling to model trial type likelihood on the individual trial level. Analyses were carried out using the lme4 software package Version 1.1-12 (Bates, Maechler, Bolker, & Walker, 2015). All variables were grand-mean-centered and converted to z scores before statistical analysis.

We first constructed three statistical models, each of which included only one of the three predictors (transitions, within-transition fixations, and entropy) as a fixed effect. Each model also included a random intercept for participants and a random slope for the fixed effect under examination. To determine whether each predictor accounted for significant variance, we used likelihood ratio tests to compare each model to a model that retained the random slope and intercept but held out the fixed effect. As shown in Models 1–3 in Table 1, each variable was significantly associated with swap errors when considered in isolation (ps ≤ .01), consistent with the trial-aggregated analyses.

Table 1. 
Parameter Estimates for Fixed Effects of Number of Transition Fixations, Average Within-item Fixations between Transition Fixations, and Scanpath Entropy on Reconstruction Accuracy in Mixed Effects Logistic Regression
ParameterEstimateSEWald Zp
Model 1: Total Transition Fixations χ2(1) = 12.79, p < .001, logl = −951.50 
Intercept −0.46 0.13 −3.65 <.001 
Transition fixations 0.31 0.08 4.00 <.001 
  
Model 2: Average Within-item Fixations χ2(1) = 7.68, p = .006, logl = −952.19 
Intercept −0.48 0.12 −3.91 <.001 
Within-item fixations −0.26 0.09 −3.00 .003 
  
Model 3: Scanpath Entropy χ2(1) = 18.95, p < .001, logl = −949.22 
Intercept −0.47 0.13 −3.67 <.001 
Scanpath entropy 0.31 0.07 4.49 <.001 
  
Model 4: All Three Predictors χ2(1) = 7.16, p = .007, logl = −947.90 
Intercept −0.47 0.13 −3.69 <.001 
Transition fixations 0.16 0.12 1.28 .20 
Within-item fixations 0.00 0.12 0.00 .99 
Scanpath entropy 0.23 0.09 2.67 .008 
ParameterEstimateSEWald Zp
Model 1: Total Transition Fixations χ2(1) = 12.79, p < .001, logl = −951.50 
Intercept −0.46 0.13 −3.65 <.001 
Transition fixations 0.31 0.08 4.00 <.001 
  
Model 2: Average Within-item Fixations χ2(1) = 7.68, p = .006, logl = −952.19 
Intercept −0.48 0.12 −3.91 <.001 
Within-item fixations −0.26 0.09 −3.00 .003 
  
Model 3: Scanpath Entropy χ2(1) = 18.95, p < .001, logl = −949.22 
Intercept −0.47 0.13 −3.67 <.001 
Scanpath entropy 0.31 0.07 4.49 <.001 
  
Model 4: All Three Predictors χ2(1) = 7.16, p = .007, logl = −947.90 
Intercept −0.47 0.13 −3.69 <.001 
Transition fixations 0.16 0.12 1.28 .20 
Within-item fixations 0.00 0.12 0.00 .99 
Scanpath entropy 0.23 0.09 2.67 .008 

Chi-square values reflect a comparison of the model with and without the final fixed effect. logl = log likelihood.

We then constructed a fourth model that included all three predictor variables as fixed effects and evaluated the significance of each variable by comparing this model to models that held out each fixed effect while retaining the other two. Random slopes were removed to facilitate model convergence, although their removal did not affect the results of significance testing. As shown in Model 4 in Table 1, entropy remained a significant predictor after accounting for the other two fixed effects, x2(1) = 7.16, p = .007, whereas the effect of transition fixations was nonsignificant, x2(1) = 1.63, p = .20, as was the effect of within-item fixations, x2(1) = 0.00, p = .99. We checked for collinearity using mer-utils.R (https://github.com/aufrank/R-hacks/blob/master/mer-utils.R). Variable inflation factors were <2.75 for all predictors, indicating an acceptable level of collinearity. Thus, the relationship between scanpath entropy and subsequent swap errors is not simply a byproduct of the relationship between scanpath entropy and either of the quantitative measures of eye movement behaviors.

Early- versus Late-trial Eye Movements

A final set of analyses examined whether scanpath entropy, transition fixations, within-item fixations per transition, and/or the relationship of these variables to subsequent memory differed between the first half and the second half of each 16-sec trial. These analyses were exploratory and intended to provide additional information about the timescale along which relationships between the aforementioned viewing behaviors and subsequent reconstruction accuracy emerges. We calculated each of the three eye movement variables separately for seconds 0–8 (early bin) and seconds 8–16 (late bin) and conducted 2 Time bin (early/late) × 2 Swap error (present/absent) repeated-measures ANOVAs with each variable as the dependent measure.

The analysis for entropy2 revealed a significant main effect of Time bin, F(1, 39) = 20.15, p < .001, ηp2 = .34, indicating that entropy was slightly but reliably lower in the second half relative to the first half of the trial (means = 0.27 and 0.29, respectively). As expected, the main effect of Swap error was also significant, F(1, 39) = 4.93, p = .03, ηp2 = .11. However, the Time bin × Swap interaction was not significant, F(1, 39) = 0.12, p = .73, ηp2 = .003.

The parallel analysis for transition fixations likewise revealed a main effect of Time bin, F(1, 39) = 5.43, p = .03, ηp2 = .12, indicating that participants tended to make fewer transition fixations in the second half relative to the first half of the trial (means = 10.22 and 9.73, respectively). The main effect of Swap error was also significant, F(1, 39) = 15.08, p < .001, ηp2 = .28. In addition, the Time bin × Swap error interaction approached significance, F(1, 39) = 3.45, p = .07, ηp2 = .08. Follow-up paired t tests revealed that swap-present and swap-absent trials differed in transition counts during the early bin (means = 10.0 and 10.7 transitions for swap-absent and swap-present trials, respectively, t(39) = 3.91, p < .001, Cohen's d = 0.62), but not the late bin (means = 9.6 and 9.9, t(39) = 1.28, p = .21, Cohen's d = 0.20).

Finally, the analysis for within-item fixations revealed a significant main effect of Swap error, F(1, 39) = 5.05, p = .03, ηp2 = .12, as well as a significant Time bin × Swap error interaction, F(1, 39) = 10.75, p = .002, ηp2 = .22. The main effect of Time bin was nonsignificant, F(1, 39) = 0.01, p = .91, ηp2 < 0.01. Follow-up paired t tests revealed that the average number of within-item fixations per transition was larger for swap-absent than swap-present trials during the early bin means = 1.30 and 1.14, respectively, t(39) = 4.00, p < .001, Cohen's d = 0.63). However, no significant relationship between within-item fixations and subsequent memory was present in the late time bin (means = 1.20 and 1.23 for swap-absent and swap-present trials, respectively, t(39) = 0.63, p = .53, Cohen's d = 0.10).

In summary, although the relationship between scanpath organization (entropy) and the tendency to commit swap errors was stable across both time windows, both of the quantitative measures of eye movements (number of transition fixations per trial; number of within-item fixations per transition) showed relationships to subsequent memory that were selective to the beginning of each trial. Specifically, trials that began with a smaller number of transition fixations and a larger number of within-item fixations in between transitions were associated with fewer swap errors. Though exploratory, this pattern could indicate that suboptimal initial patterns of information sampling—for example, engaging in extensive display-level exploration before sufficient item-level exploration has occurred—may have cascading negative effects on memory formation. We will return to this possibility after Experiment 2.

Discussion

Although the link between active exploration and memory has featured prominently in work on nonhuman animals, far less is known about how exploration supports memory formation in humans. Saccadic eye movements are the primary means by which humans explore their environments, and several initial studies have reported positive, monotonic relationships between the number of fixations made during encoding and subsequent memory (Olsen et al., 2015, 2016; Molitor, Ko, Hussey, & Ally, 2014; Kafkas & Montaldi, 2011). Fixation count has thus been proposed as a “currency of memory” (Meister & Buffalo, 2016), in that memory representations strengthen as fixations accrue. By contrast, spatial reconstruction accuracy in this study was best predicted by the way in which participants initially viewed information, rather than how many fixations or transitions were deployed. In both across- and within-subjects analyses, scanpaths that were lower in entropy (e.g., were more organized and less random) were associated with more accurate subsequent memory, as operationalized by fewer instances of “swapping” the locations of items when reconstructing the array.

Several aspects of our experimental design may explain this departure from prior findings. For instance, although previous studies have examined eye movement predictors of subsequent memory for individual objects (Molitor et al., 2014; Kafkas & Montaldi, 2011; Loftus, 1972), faces (Olsen et al., 2015, 2016), or paired associates (Kamp & Zimmer, 2015), the spatial reconstruction task used here emphasized the binding of arbitrary relations among complex sets of items and locations. We suggest that the nature and/or complexity of the memory representations that support performance on this task necessitated that eye movements be used in a selective and strategic manner (see Johnson et al., 2012, for a review of parallel findings from the animal literature).

Disentangling the relationships among—and contributions to memory of—type versus amount of eye movements in various situations will be a fruitful topic for additional research. In addition to direct contrasts of tasks with higher versus lower demands on relational memory, future research could examine whether memory for certain types of relations are more tightly tied to viewing constraint than others. As previously discussed, we chose to examine swap errors because they provide a relatively straightforward measure of the ability to bind arbitrary spatial relations and because they have been linked to hippocampal integrity in both healthy and patient populations (Schwarb et al., 2016, 2017; Monti et al., 2015; Watson et al., 2013). However, this measure is unable to separate various types of relational memory failures, such as those pertaining to item–item relational binding (e.g., remembering that Item 1 is located above and to the right of Item 2) versus item–location binding (remembering that Item 1 is located toward the upper right corner of the display). Interestingly, Horecka et al. (2018) recently developed a procedure to tease apart the contributions of these and a large number of other, highly specific types of errors to overall reconstruction performance, which may prove useful in future investigations of the role of viewing behaviors in memory formation.

Regardless, the finding that low-entropy viewing promoted successful performance on a task with well-established dependence on the hippocampus provides a novel opportunity to interrogate the nature of the relationships among relational memory, saccadic exploration, and hippocampal integrity. To this end, Experiment 2 examines study-phase eye movements and spatial reconstruction performance in patients with amnesia due to bilateral hippocampal damage, as well as healthy comparison participants. A finding that, rather than simply making more swap errors at test, amnesic patients also engage in suboptimal viewing patterns during initial study would indicate a causal role of hippocampal processing in optimizing memory formation by guiding exploratory viewing behaviors.

EXPERIMENT 2

Methods

Participants

Participants were three patients with bilateral hippocampal damage and profound declarative memory impairment (HC group), along with nine healthy comparison participants free of neurological and psychiatric disease (NC group) matched to the HC group on sex, age, and level of education. All participants gave written consent before participating in the study. Institutional review boards at both the University of Iowa and the University of Illinois Urbana-Champaign approved all study procedures. Etiologies within the HC group included anoxia/hypoxia resulting in bilateral damage that was confined to the hippocampus (1846 and 2563) and herpes simplex encephalitis (1951), resulting in more extensive damage affecting the bilateral hippocampus, amygdala, and surrounding cortices.

Structural MRI examinations completed on two of the three patients confirmed bilateral hippocampal damage and showed hippocampal volumes significantly decreased for each patient, with the studentized residual differences in hippocampal volume relative to a matched comparison group down by at least 4.2 (1846) and as much as 8.10 (1951) z scores (Allen, Tranel, Bruss, & Damasio, 2006; Buchanan, Tranel, & Adolphs, 2005). Participant 2563 wears a pacemaker and was unable to undergo MRI examination. Thus, this patient's damage was confirmed by computerized tomography; damage was confined to the hippocampus.

Neuropsychological examination confirmed profound declarative memory impairment in all hippocampal patients (M = 63; Wechsler Memory Scale-III General Memory Index, more than 2 standard deviations below population norms) in the context of within normal performance on standardized measures of intelligence, language, visual perception, working memory, and executive functioning (see Table 2).

Table 2. 
Demographic, Neuroanatomical, and Neuropsychological Characteristics of Participants with Hippocampal Amnesia (HC), along with Demographic Information for the Comparison Participants with No Brain Damage (NC)
Sub.DemographicNeuroanatomyNeuropsychological
SexHAgeEdEtLesionHippWAIS-FSIQWMS-GMIWMS-WMIBNTTTCFT CopyWCS Cat
1846 52 14 An Bilateral HC −4.23 84 57 85 43 41 28 
2563 60 16 An Bilateral HC N/A 102 75 99 52 44 36 
1951 63 16 HSE Bilateral HC + MTL −8.10 106 57 108 49 44 32 
HC mean 1 F 2 R 58 15 –     97 63 97 48 43 32 
2 M 1 L (±6) (±1)       (±12) (±10) (±12) (±5) (±2) (±2)   
NC mean 3 F 7 R 60 16 –                   
6 M 2 L (±5) (±2)                     
Sub.DemographicNeuroanatomyNeuropsychological
SexHAgeEdEtLesionHippWAIS-FSIQWMS-GMIWMS-WMIBNTTTCFT CopyWCS Cat
1846 52 14 An Bilateral HC −4.23 84 57 85 43 41 28 
2563 60 16 An Bilateral HC N/A 102 75 99 52 44 36 
1951 63 16 HSE Bilateral HC + MTL −8.10 106 57 108 49 44 32 
HC mean 1 F 2 R 58 15 –     97 63 97 48 43 32 
2 M 1 L (±6) (±1)       (±12) (±10) (±12) (±5) (±2) (±2)   
NC mean 3 F 7 R 60 16 –                   
6 M 2 L (±5) (±2)                     

Bolded scores indicate deficit greater than 2 SD below population or comparison norms. Participant 2563 wears a pacemaker and was unable to undergo MRI examination. H = handedness; Ed = years of education; Et = etiology; An = anoxia; HSE = herpes simplex encephalitis; HC = damage confined to the hippocampus; + MTL = damage extending into the greater medial temporal lobes; Hipp = hippocampal volumetric z scores as measured through high-resolution volumetric MRI and compared to a matched healthy comparison group (Allen et al., 2006; Buchanan et al., 2005); WAIS-FSIQ = Wechsler Adult Intelligence Scale–III Full Scale Intelligence Quotient; WMS-GMI = Wechsler Memory Scale–III General Memory Index; WMS-WMI = Wechsler Memory Scale–III Working Memory Index; BNT = Boston Naming Test; TT = Token Test; CFT = Complex Figure Test; WCT = Wisconsin Card Sorting Task; Cat = number of categories achieved out of six; N/A = not available.

Stimuli

Stimuli were 32 visual displays similar to those used in Experiment 1, but which were composed of grayscale images of common objects instead of abstract images. Images came from the publicly available BOSS stimulus set (Brodeur, Guérard, & Bouras, 2014). The six objects in each display were either all members of the same basic category (e.g., musical instruments) or contained three items each from two categories. The categories were initially included for exploratory analyses concerning the role of semantic relatedness in constraining eye movement exploration. None of the eye movement variables differed between single-category and double-category trials for either the hippocampal patients or the comparisons nor did the number of swap errors committed. Thus, we collapsed across trial types for all analyses.

Procedure

The procedure for Experiment 2 was the same as in Experiment 1, except that displays were presented in four blocks of eight trials. Eye-tracking quality criteria for trial inclusion were the same as in Experiment 1, and an average of 30/32 trials were included from each participant (range = 21–32).

As in Experiment 1, spatial reconstruction performance was operationalized as the number of swap errors committed on each trial, and analyses of study-phase eye movements included the number of total fixations, transition fixations, average within-item fixations per transition, and scanpath entropy.3 Across-group comparisons used Welch's t tests, in which degrees of freedom are adjusted for unequal variance. A one-tailed test was used to compare swap error rates between the HC and NC groups, because this pattern was only expected to differ in one direction. All other statistical tests are reported as two-tailed.

In addition to the eye movement variables of interest, we performed several additional comparisons to confirm that the amount, recording quality, and other global characteristics of the eye movement data did not differ between the patients and comparisons. As in several other studies of patients with hippocampal amnesia (Kurczek, Brown-Schmidt, & Duff, 2013; Warren, Duff, Tranel, & Cohen, 2011; Ryan, Althoff, Whitlow, & Cohen, 2000), these analyses confirm that eye movements were largely similar between the groups. These data are summarized in Table 3.

Table 3. 
Additional Comparisons of Patients' and Comparisons' Eye Movement Data
MeasureNC Mean (SE)HC Mean (SE)tp
Mean number of trials included in analyses (out of a possible 32) 28.9 (1.5) 29 (1.2) 0.06 .96 
Mean proportion of each 16-sec trial spent fixating on any part of the screen 0.86 (0.01) 0.81 (0.03) 1.62 .22 
Mean percentage of fixations per trial that were assigned to an ROI by the k-means clustering procedurea 0.93 (0.005) 0.93 (0.002) 0.06 .96 
Mean fixation duration in milliseconds 294 (15) 255 (17) 1.75 .13 
Total number of fixations, including between-item transitions and within-item fixations 46.1 (1.6) 49.5 (1.9) 1.33 .24 
MeasureNC Mean (SE)HC Mean (SE)tp
Mean number of trials included in analyses (out of a possible 32) 28.9 (1.5) 29 (1.2) 0.06 .96 
Mean proportion of each 16-sec trial spent fixating on any part of the screen 0.86 (0.01) 0.81 (0.03) 1.62 .22 
Mean percentage of fixations per trial that were assigned to an ROI by the k-means clustering procedurea 0.93 (0.005) 0.93 (0.002) 0.06 .96 
Mean fixation duration in milliseconds 294 (15) 255 (17) 1.75 .13 
Total number of fixations, including between-item transitions and within-item fixations 46.1 (1.6) 49.5 (1.9) 1.33 .24 

No significant group differences were present for any of the variables listed below. HC = hippocampal patient; NC = normal comparison participant; SE = standard error.

a

Fixations that were not assigned to an ROI represent looking to “blank” sections of the screen in which no object was present. These fixations were excluded from analyses (see Experiment 1 Methods). The nonsignificant result indicates that patients and comparisons directed an equal proportion of their fixations to the ROIs as opposed to elsewhere on the screen.

Results

Across-group Analyses

As expected, the HC group showed a greater preponderance of swap errors relative to the NC group, t(2.67) = 3.06, p = .03, Cohen's d = 2.45 (Figure 4A). Figure 5BC depicts the results of the group comparisons of study-phase eye movement behaviors. As shown, the HC group exhibited significantly higher levels of scanpath entropy relative to the NC group, t(10.00) = 2.94, p = .015, Cohen's d = 1.21. By contrast, the total number of transition fixations per trial did not differ by group, t(8.82) = 0.83, p = .43, Cohen's d = 0.38, nor did the mean number of within-item fixations that preceded each transition, t(4.51) = 0.01, p = .99, Cohen's d = 0.003.

Figure 5. 

Swap errors (A), scanpath entropy (B), transition fixations/trial (C), and within-item fixations/transition (D) for hippocampal patients (HCs) and normal comparison participants (NCs). Hippocampal patients made significantly more swap errors than did the comparison participants and had significantly higher amounts of scanpath entropy during study. By contrast, the groups did not differ in the number of transitions made per study nor in the number of within-item fixations. Patients' scores are individually labeled for reference.

Figure 5. 

Swap errors (A), scanpath entropy (B), transition fixations/trial (C), and within-item fixations/transition (D) for hippocampal patients (HCs) and normal comparison participants (NCs). Hippocampal patients made significantly more swap errors than did the comparison participants and had significantly higher amounts of scanpath entropy during study. By contrast, the groups did not differ in the number of transitions made per study nor in the number of within-item fixations. Patients' scores are individually labeled for reference.

We then compared scanpath entropy between trials with and without swap errors in both the NC and HC group to determine (1) whether the within-subject relationship between entropy and swap errors shown in Experiment 1 was replicated in Experiment 2 and (2) whether this relationship is also present in the HC group. For the NC group, an average of nine swap-present trials (range = 2–24) and 20 swap-absent trials per participant (range = 8–29) were included in these analyses. For the HC group, the trial counts were as follows: 23 swap-present trials and four swap-absent trials for Patient 1846, 19 swap-present trials and 12 swap-absent trials for Patient 2563, and 28 swap-present trials and one swap-absent trial for Patient 1951. Note that the number of swap-present trials was significantly greater in the HC group relative to the NC group, t(5.3) = 4.23, p = .01, Cohen's d = 2.30.

The results are depicted in Figure 6. A 2 × 2 ANOVA with swap error (present/absent) as a within-subjects factor and group (HC/NC) as a between-subject factor revealed only a nonsignificant trend toward an interaction, F(1, 10) = 3.18, p = .11. However, a ηp2 value of .24 indicated a large effect size. Indeed, paired t tests indicated that, as in Experiment 1, NC participants had significantly higher entropy for trials with swap errors (mean entropy = 0.46, SE = 0.02) relative to trials without swap errors (mean = 0.41, SE = 0.03, t(8) = 2.47, p = .04, Cohen's d = 0.82). By contrast, a marginal trend in the opposite direction was present for HC participants (swap-present trial mean = 0.50, SE = 0.01, swap-absent mean = 0.52, SE = 0.02, t(2) = 3.09, p = .09, Cohen's d = 1.78).

Figure 6. 

Within-subject analyses comparing scanpath entropy between trials with and without subsequent swap errors for both normal comparison participants (NCs) and hippocampal patients (HCs). Error bars show ±1 standard error with between-subject variance removed (Morey, 2008). Patients' scores are individually labeled for reference.

Figure 6. 

Within-subject analyses comparing scanpath entropy between trials with and without subsequent swap errors for both normal comparison participants (NCs) and hippocampal patients (HCs). Error bars show ±1 standard error with between-subject variance removed (Morey, 2008). Patients' scores are individually labeled for reference.

Early- versus Late-trial Eye Movements

The results of Experiment 1 suggested that swap errors were particularly associated with (1) a higher number of item-to-item transitions early as opposed to late in the course of a trial and (2) a lower number of fixations preceding each transition early as opposed to late, potentially suggesting inadequacies in early stages of information gathering. Thus, despite the fact that neither overall transition fixation count nor the average number of within-item fixations per transition differed between the patients and comparisons, we performed separate 2 × 2 ANOVAs for each of these variables with factors Time bin (early/late) and Group (HC/NC). For the analysis of transition fixations, the Group × Trial bin interaction was significant, F(1, 10) = 6.83, p = .03, ηp2 = .41. Follow-up comparisons revealed that the HC group made significantly more transitions relative to the NC group during the first half of the trial (means = 12.1 vs. 10.3, t(9.92) = 2.27, p = .047, Cohen's d = 0.96), whereas transition counts did not differ later in the trial (means = 10.8 and 11.4 for HCs and NCs, respectively, t(8.13) = 0.63, p = .55, Cohen's d = 0.29). No significant Group × Time bin interaction emerged for the analysis of average within-item fixations, F(1, 10) = 1.81, p = .21, ηp2 = .15.

Discussion

As expected, patients with bilateral hippocampal damage made more errors on the spatial reconstruction task relative to healthy comparison participants (Horecka et al., 2018; Monti et al., 2015; Watson et al., 2013). Most interestingly, analyses of eye-tracking data revealed effects of hippocampal damage that were operative during the study phases, while the participants were in the process of attempting to commit the displays to memory. Specifically, the hippocampal patients engaged in item-to-item viewing patterns at study that were more disorganized (e.g., higher in entropy) relative to the viewing patterns of healthy comparison participants, despite the fact that patients and comparisons engaged in a similar number of fixations and item-to-item transitions. This finding, taken together with data from Experiment 1, provides causal support for the notion that hippocampal contributions to relational memory occur, in part, by way of organizing information sampling, including sensory sampling (Wang et al., 2015; Johnson et al., 2012).

In addition to showing different amounts of scanpath entropy, the patient and comparison groups also differed in the functional significance of trial-by-trial variability in entropy. As in Experiment 1, healthy participants in Experiment 2 showed a significant within-subject relationship between study-phase entropy and subsequent swap errors, such that trials that contained one or more errors at test were preceded by higher-entropy viewing at study relative to trials that did not. By contrast, this relationship was not present in the hippocampal patients, suggesting an inability to benefit from the use of constrained viewing. Below we discuss the implications of these findings, together with those of Experiment 1, for theoretical accounts of the relationships among memory, exploration, and the hippocampus.

GENERAL DISCUSSION

In contrast to traditional accounts of hippocampal function that emphasize the ability to remember information after delays, a growing body of research suggests that the hippocampus also plays a critical role in the rapid use of memory to inform and optimize ongoing, moment-to-moment behaviors. One prediction of this account, already borne out in initial studies (Yee et al., 2014; Kaplan et al., 2012; Voss, Gonsalves, et al., 2011; Voss, Warren, et al., 2011), is that hippocampal contributions to memory outcomes will be greatest when participants are actively involved in directing the course of each study episode and, therefore, must make constant decisions about how best to explore to-be-remembered information. The present findings cohere with this prediction and further extend the link between self-directed exploration, memory, and the hippocampus to a study task that would typically be characterized as passive, in that participants had no measure of control over the course of the study episode aside from their use of eye movements.

The assertion that eye movements should influence hippocampal memory formation is relatively intuitive and is supported by work in both humans and nonhuman primates (Meister & Buffalo, 2016). For example, research using hippocampal depth electrodes indicates that saccades made during exploratory viewing serve to modulate theta-band oscillatory activity in the hippocampus by triggering phase alignment (Hoffman et al., 2013; Jutras et al., 2013), the reliability of which has been shown to predict subsequent recognition memory (Jutras et al., 2013). As such, the finding that the relationship between low-entropy viewing and subsequent memory in healthy participants did not extend to hippocampal patients is not entirely surprising and can be attributed to the disruption of these anatomical and functional pathways (see also Olsen et al., 2016).

By contrast, prior evidence for a causal role of the hippocampus in guiding study-related eye movements—as suggested by the overall higher levels of entropy in patients' viewing behaviors—is scarce and largely indirect (for a review, see Voss et al., 2017). A recent investigation combining eye movement recordings with functional neuroimaging during a difficult visual discrimination task (Voss & Cohen, 2017) found that activity in a network of brain regions, including the hippocampus, correlated with the use of a specific eye movement pattern linked to better task performance. However, neuroimaging methods are insufficient to establish causality, and this study was not designed to link viewing to the process of memory formation per se. Another study examined eye movements during face gender judgments in both healthy participants and a patient with developmental amnesia (Olsen et al., 2015) and revealed a tendency in the patient toward a more exclusive gaze focus on the eye region with fewer transitions between the eye and the nose/mouth area. However, the functional significance of this pattern is unclear.4 As such, the present findings constitute the first evidence for a causal relationship between hippocampal memory processing and the ability to use moment-to-moment eye movements effectively during study.

As previously noted, the tendency toward greater scanpath entropy that was found in the hippocampal patients in Experiment 2 cannot be attributed to a simple effect of hippocampal damage on the amount of visual exploration, because neither the number of fixations nor the frequency of item-to-item transitions differed between the patients and comparison participants. Rather, the present findings link the hippocampus to the ability to enact the more sophisticated (and, in the context of this task, more effective) strategy of imposing structure and order on patterns of saccadic exploration. That said, patients did make more frequent item-to-item transitions during the early period (the first 8 sec) of each trial. Interestingly, a related pattern was present in Experiment 1, by which making frequent transitions early, but not late, in each trial was associated with a higher instance of swap errors.

Both of these patterns emerged from exploratory analyses and should be interpreted with caution. Nonetheless, it bears mention that some models of memory-guided exploration that originate from rodent studies (e.g., Johnson et al., 2012) suggest that exploration patterns in complex, multi-item environments follow a predictable sequence during learning that begins with an item familiarization process (manifest as a series of “stay/go” exploration decisions for each item) and then shifts toward a more directed use of memory to maximize sampling efficiency across the space. Given that saccadic eye movements have been proposed as a primate homologue to the sensory processes by which rodents explore their environment (whisking, sniffing, etc.; Meister & Buffalo, 2016; Jutras et al., 2013), future studies should further investigate the notion that the types of saccadic “foraging” afforded by the hippocampus may follow a similar iterative progression, such that the “weights” of different information-sampling behaviors change as representations form and evolve.

Related to this point, a limitation of the present research pertains to our use of relatively long analysis windows (16-sec trials or 8 sec in the exploratory half-trial analyses). The use of longer windows is beneficial for obtaining stable and meaningful estimates of scanpath entropy, which by definition must be calculated over a series of item-to-item transitions. However, a downside of this choice is that it provides only very coarse insight into when differences in viewing behaviors first emerge between trials with versus without subsequent swap errors, or between hippocampal patients and healthy comparisons. In both experiments, significant differences were either selective to the earlier analysis window or were invariant across time windows, consistent with the notion that visual exploration is tightly and perhaps ubiquitously interconnected with the process of visual memory formation. However, future studies could harness high-temporal resolution techniques, such as intracranial EEG recording, to provide far more precise information about the nature and time course of hippocampal involvement. For example, research in primates (Jutras et al., 2013) has shown that recognition memory for novel images is predicted by (1) the reliability with which exploratory saccades produced a theta-band phase reset in hippocampal neurons and (2) the magnitude of theta-band power that was evident before stimulus onset. Future work might examine whether these phenomena also predict the extent to which scanning over the course of a visuospatial encoding attempt unfolds in a more constrained versus entropic manner and/or the frequency with which item-to-item transitions occur at different points in time.

Other outstanding questions pertain to the boundaries of the present findings. It is important to keep in mind that rarely, if ever, is there a one-to-one relationship between a specific pattern of viewing and a given set of neurocognitive processes. For this reason, our ability to draw conclusions about the functional significance of the elevated scanpath entropy in HC patients—most notably, the conclusion that it contributed to suboptimal relational memory formation—depends critically on the findings of Experiment 1, in which we established a relationship between scanpath entropy and subsequent performance in the context of the very same task. As previously discussed, different relationships between eye movements and memory formation have been obtained in other situations, for example, when testing memory for single items (Molitor et al., 2014; Kafkas & Montaldi, 2011; Loftus, 1972), faces (Olsen et al., 2015, 2016), or paired associates (Kamp & Zimmer, 2015). Thus, to the extent that the hippocampus contributes to the optimization of exploratory study behaviors in those circumstances, such contributions would likewise be expected to manifest differently. fMRI evidence from healthy individuals concords with this notion. For example, one recent study found a positive relationship between the strength of hippocampal activity and the number of fixations made while participants viewed faces (Liu et al., 2017), a stimulus category for which the relationship between fixation count and memory has been shown to be positive and monotonic. A greater use of causal methods in this research will be informative, both in detailing the ways in which hippocampal memory processing informs exploratory behaviors and, in turn, how specific patterns of exploration contribute to the creation and refinement of memory representations.

In summary, the present findings bolster recent suggestions that the processes by which the hippocampus promotes memory encoding are fundamentally, perhaps even indelibly interconnected with the processes by which the hippocampus promotes exploration (Voss et al., 2017; Wang et al., 2015; Johnson et al., 2012). However, given the many forms by which exploration occurs—not only externally via sensorimotor processing, but also internally, as in the sampling of memory or knowledge stores—much work remains to be done to empirically probe the boundaries of this account. For example, mounting evidence suggests that the hippocampus contributes to performance on a variety of tasks, such as divergent creative thinking (Duff, Kurczek, Rubin, Cohen, & Tranel, 2013), that require the ability to flexibly explore and combine existing knowledge in response to task demands. It is possible that this form of exploration might also contribute to the ability to learn and study new information under certain circumstances (Addis, Giovanello, Vu, & Schacter, 2014). The relationship between memory and exploration in all its forms constitutes a fertile area of study that promises to inform our understanding of both healthy and disordered forms of memory.

Acknowledgments

This work was supported by the National Institute of Mental Health (R01-MH062500 to N. J. C.), the National Institute on Deafness and other Communication Disorders (R01-DC011755 to M. C. D.), and a Beckman Institute Postdoctoral Fellowship to H. D. L. We thank Michael Dulas, Rachel Gonzalez, Faizan Khawaja, Nirav Patel, and Hillary Schwarb for assistance with data collection; Dan Kleinman for guidance on the multilevel models; and Kevin Horecka and Patrick Watson for helpful discussion.

Reprint requests should be sent to Heather D. Lucas, Department of Psychology, Louisiana State University, 236 Audubon Hall, Baton Rouge, LA 70803, or via e-mail: hdlucas@gmail.com.

Notes

1. 

It bears mention that our use of the term “swap” error in this context refers to any instance in which both the right/left and top/bottom relationships between two objects are reversed to be consistent with its use in multiple prior studies, rather than referring to the more limited instance in which each item is assigned the precise coordinates previously occupied by the other, as used in Horecka et al. (2018). Note that true compound coordinate-matching swaps are, in fact, less diagnostic of hippocampal damage than the measure used here, likely because such errors require participants to maintain a relatively accurate representation in memory of where pairs of items are located while simultaneously failing to bind specific items to locations within each pair.

2. 

One trial from one participant was excluded from these analyses because only one transition was made during the early window, and the entropy of a Markov transition matrix with only one data point cannot be calculated. Note that at least three transitions within a given time window are necessary for variability in entropy to occur, because the entropy of a matrix containing two transitions will always be zero. Excluding trials with only two transitions in either time window (1.6% of all trials) did not change the results.

3. 

A more comprehensive examination of ways in which behavioral performance differed between patients and controls on the spatial reconstruction task can be found in Horecka et al. (2018).

4. 

Note, however, that this represents a situation in which hippocampal damage was associated with viewing that was more constrained (lower in entropy), opposite to the pattern found for the present spatial reconstruction task. This finding (see also Warren et al., 2011) suggests that eye movement disorganization is not a ubiquitous consequence of hippocampal damage but depends on the extent to which hippocampal memory processing is called upon to constrain viewing in a given situation. See subsequent text for additional discussion.

REFERENCES

REFERENCES
Addis
,
D. R.
,
Giovanello
,
K. S.
,
Vu
,
M.-A.
, &
Schacter
,
D. L.
(
2014
).
Age-related changes in prefrontal and hippocampal contributions to relational encoding
.
Neuroimage
,
84
,
19
26
.
Allen
,
J. S.
,
Tranel
,
D.
,
Bruss
,
J.
, &
Damasio
,
H.
(
2006
).
Correlations between regional brain volumes and memory performance in anoxia
.
Journal of Clinical and Experimental Neuropsychology
,
28
,
457
476
.
Althoff
,
R. R.
, &
Cohen
,
N. J.
(
1999
).
Eye-movement-based memory effect: A reprocessing effect in face perception
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
25
,
997
1010
.
Arkley
,
K.
,
Grant
,
R. A.
,
Mitchinson
,
B.
, &
Prescott
,
T. J.
(
2014
).
Strategy change in vibrissal active sensing during rat locomotion
.
Current Biology
,
24
,
1507
1512
.
Bates
,
D.
,
Maechler
,
M.
,
Bolker
,
B.
, &
Walker
,
S.
(
2015
).
Fitting linear mixed-effects models using lme4
.
Journal of Statistical Software
,
67
,
1
48
.
Brodeur
,
M. B.
,
Guérard
,
K.
, &
Bouras
,
M.
(
2014
).
Bank of Standardized Stimuli (BOSS) phase II: 930 New normative photos
.
PLoS One
,
9
,
e106953
.
Buchanan
,
T. W.
,
Tranel
,
D.
, &
Adolphs
,
R.
(
2005
).
Emotional autobiographical memories in amnesic patients with medial temporal lobe damage
.
Journal of Neuroscience
,
25
,
3151
3160
.
Cohen
,
N. J.
, &
Eichenbaum
,
H.
(
1993
).
Memory, amnesia, and the hippocampal system
.
Cambridge, MA
:
MIT Press
.
Daugherty
,
A. M.
,
Yuan
,
P.
,
Dahle
,
C. L.
,
Bender
,
A. R.
,
Yang
,
Y.
, &
Raz
,
N.
(
2015
).
Path complexity in virtual water maze navigation: Differential associations with age, sex, and regional brain volume
.
Cerebral Cortex
,
25
,
3122
3131
.
Duff
,
M. C.
,
Kurczek
,
J.
,
Rubin
,
R.
,
Cohen
,
N. J.
, &
Tranel
,
D.
(
2013
).
Hippocampal amnesia disrupts creative thinking
.
Hippocampus
,
23
,
1143
1149
.
Easton
,
A.
,
Zinkivskay
,
A.
, &
Eacott
,
M. J.
(
2009
).
Recollection is impaired, but familiarity remains intact in rats with lesions of the fornix
.
Hippocampus
,
19
,
837
843
.
Eichenbaum
,
H.
, &
Cohen
,
N. J.
(
2014
).
Can we reconcile the declarative memory and spatial navigation views on hippocampal function?
Neuron
,
83
,
764
770
.
Hartley
,
T.
,
Lever
,
C.
,
Burgess
,
N.
, &
O'Keefe
,
J.
(
2013
).
Space in the brain: How the hippocampal formation supports spatial cognition
.
Philosophical Transactions of the Royal Society, Series B, Biological Sciences
,
369
,
20120510
.
Hoffman
,
K. L.
,
Dragan
,
M. C.
,
Leonard
,
T. K.
,
Micheli
,
C.
,
Montefusco-Siegmund
,
R.
, &
Valiante
,
T. A.
(
2013
).
Saccades during visual exploration align hippocampal 3–8 Hz rhythms in human and non-human primates
.
Frontiers in Systems Neuroscience
,
7
,
43
.
Horecka
,
K. M.
,
Dulas
,
M. R.
,
Schwarb
,
H.
,
Lucas
,
H. D.
,
Duff
,
M.
, &
Cohen
,
N. J.
(
2018
).
Reconstructing relational information
.
Hippocampus
,
28
,
164
177
.
Huttenlocher
,
J.
, &
Presson
,
C. C.
(
1979
).
The coding and transformation of spatial information
.
Cognitive Psychology
,
11
,
375
394
.
Johnson
,
A.
, &
Redish
,
A. D.
(
2007
).
Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point
.
Journal of Neuroscience
,
27
,
12176
12189
.
Johnson
,
A.
,
Varberg
,
Z.
,
Benhardus
,
J.
,
Maahs
,
A.
, &
Schrater
,
P.
(
2012
).
The hippocampus and exploration: Dynamically evolving behavior and neural representations
.
Frontiers in Human Neuroscience
,
6
,
216
.
Jutras
,
M. J.
,
Fries
,
P.
, &
Buffalo
,
E. A.
(
2013
).
Oscillatory activity in the monkey hippocampus during visual exploration and memory formation
.
Proceedings of the National Academy of Sciences, U.S.A.
,
110
,
13144
13149
.
Kafkas
,
A.
, &
Montaldi
,
D.
(
2011
).
Recognition memory strength is predicted by pupillary responses at encoding while fixation patterns distinguish recollection from familiarity
.
Quarterly Journal of Experimental Psychology
,
64
,
1971
1989
.
Kamp
,
S.-M.
, &
Zimmer
,
H. D.
(
2015
).
Contributions of attention and elaboration to associative encoding in young and older adults
.
Neuropsychologia
,
75
,
252
264
.
Kaplan
,
R.
,
Doeller
,
C. F.
,
Barnes
,
G. R.
,
Litvak
,
V.
,
Düzel
,
E.
,
Bandettini
,
P. A.
, et al
(
2012
).
Movement-related theta rhythm in humans: Coordinating self-directed hippocampal learning
.
PLoS Biology
,
10
,
e1001267
.
Kurczek
,
J.
,
Brown-Schmidt
,
S.
, &
Duff
,
M.
(
2013
).
Hippocampal contributions to language: Evidence of referential processing deficits in amnesia
.
Journal of Experimental Psychology. General
,
142
,
1346
1354
.
Liu
,
Z.-X.
,
Shen
,
K.
,
Olsen
,
R. K.
, &
Ryan
,
J. D.
(
2017
).
Visual sampling predicts hippocampal activity
.
Journal of Neuroscience
,
37
,
599
609
.
Loftus
,
G. R.
(
1972
).
Eye fixations and recognition memory for pictures
.
Cognitive Psychology
,
3
,
525
551
.
McNaughton
,
B. L.
,
Barnes
,
C. A.
,
Gerrard
,
J. L.
,
Gothard
,
K.
,
Jung
,
M. W.
,
Knierim
,
J. J.
, et al
(
1996
).
Deciphering the hippocampal polyglot: The hippocampus as a path integration system
.
Journal of Experimental Biology
,
199
,
173
185
.
McNaughton
,
B. L.
,
Battaglia
,
F. P.
,
Jensen
,
O.
,
Moser
,
E. I.
, &
Moser
,
M.-B.
(
2006
).
Path integration and the neural basis of the “cognitive map.”
Nature Reviews Neuroscience
,
7
,
663
678
.
Meister
,
M. L. R.
, &
Buffalo
,
E. A.
(
2016
).
Getting directions from the hippocampus: The neural connection between looking and memory
.
Neurobiology of Learning and Memory
,
134
,
135
144
.
Molitor
,
R. J.
,
Ko
,
P. C.
,
Hussey
,
E. P.
, &
Ally
,
B. A.
(
2014
).
Memory-related eye movements challenge behavioral measures of pattern completion and pattern separation
.
Hippocampus
,
24
,
666
672
.
Monti
,
J. M.
,
Cooke
,
G. E.
,
Watson
,
P. D.
,
Voss
,
M. W.
,
Kramer
,
A. F.
, &
Cohen
,
N. J.
(
2015
).
Relating hippocampus to relational memory processing across domains and delays
.
Journal of Cognitive Neuroscience
,
27
,
234
245
.
Morey
,
R. D.
(
2008
).
Confidence intervals from normalized data: A correction to Cousineau (2005)
.
Tutorials in Quantitative Methods for Psychology
,
4
,
61
64
.
Moscovitch
,
M.
,
Rosenbaum
,
R. S.
,
Gilboa
,
A.
,
Addis
,
D. R.
,
Westmacott
,
R.
,
Grady
,
C.
, et al
(
2005
).
Functional neuroanatomy of remote episodic, semantic and spatial memory: A unified account based on multiple trace theory
.
Journal of Anatomy
,
207
,
35
66
.
Norman
,
K. A.
, &
O'Reilly
,
R. C.
(
2003
).
Modeling hippocampal and neocortical contributions to recognition memory: A complementary-learning-systems approach
.
Psychological Review
,
110
,
611
646
.
O'Keefe
,
J.
, &
Dostrovsky
,
J.
(
1971
).
The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat
.
Brain Research
,
34
,
171
175
.
Olsen
,
R. K.
,
Lee
,
Y.
,
Kube
,
J.
,
Rosenbaum
,
R. S.
,
Grady
,
C. L.
,
Moscovitch
,
M.
, et al
(
2015
).
The role of relational binding in item memory: Evidence from face recognition in a case of developmental amnesia
.
Journal of Neuroscience
,
35
,
5342
5350
.
Olsen
,
R. K.
,
Sebanayagam
,
V.
,
Lee
,
Y.
,
Moscovitch
,
M.
,
Grady
,
C. L.
,
Rosenbaum
,
R. S.
, et al
(
2016
).
The relationship between eye movements and subsequent recognition: Evidence from individual differences and amnesia
.
Cortex
,
85
,
182
193
.
Palombo
,
D. J.
,
Keane
,
M. M.
, &
Verfaellie
,
M.
(
2015
).
How does the hippocampus shape decisions?
Neurobiology of Learning and Memory
,
125
,
93
97
.
Pfeiffer
,
B. E.
, &
Foster
,
D. J.
(
2013
).
Hippocampal place-cell sequences depict future paths to remembered goals
.
Nature
,
497
,
74
79
.
R Core Team
. (
2015
).
R: A language and environment for statistical computing
.
Vienna, Austria
:
R Foundation for Statistical Computing
.
Redish
,
A. D.
(
2016
).
Vicarious trial and error
.
Nature Reviews Neuroscience
,
17
,
147
159
.
Robertson
,
B.-A.
,
Eacott
,
M. J.
, &
Easton
,
A.
(
2015
).
Putting memory in context: Dissociating memories by distinguishing the nature of context
.
Behavioural Brain Research
,
285
,
99
104
.
Rubin
,
R.
,
Schwarb
,
H.
,
Lucas
,
H.
,
Dulas
,
M.
, &
Cohen
,
N. J.
(
2017
).
Dynamic hippocampal and prefrontal contributions to memory processes and representations blur the boundaries of traditional cognitive domains
.
Brain Sciences
,
7
,
82
.
Ryan
,
J. D.
,
Althoff
,
R. R.
,
Whitlow
,
S.
, &
Cohen
,
N. J.
(
2000
).
Amnesia is a deficit in relational memory
.
Psychological Science
,
11
,
454
461
.
Schwarb
,
H.
,
Johnson
,
C. L.
,
Daugherty
,
A. M.
,
Hillman
,
C. H.
,
Kramer
,
A. F.
,
Cohen
,
N. J.
, et al
(
2017
).
Aerobic fitness, hippocampal viscoelasticity, and relational memory performance
.
Neuroimage
,
153
,
179
188
.
Schwarb
,
H.
,
Johnson
,
C. L.
,
McGarry
,
M. D. J.
, &
Cohen
,
N. J.
(
2016
).
Medial temporal lobe viscoelasticity and relational memory performance
.
Neuroimage
,
132
,
534
541
.
Scoville
,
W. B.
, &
Milner
,
B.
(
1957
).
Loss of recent memory after bilateral hippocampal lesions
.
Journal of Neuropsychiatry and Clinical Neurosciences
,
12
,
103
113
.
Voss
,
J. L.
,
Bridge
,
D. J.
,
Cohen
,
N. J.
, &
Walker
,
J. A.
(
2017
).
A closer look at the hippocampus and memory
.
Trends in Cognitive Sciences
,
21
,
577
588
.
Voss
,
J. L.
, &
Cohen
,
N. J.
(
2017
).
Hippocampal–cortical contributions to strategic exploration during perceptual discrimination
.
Hippocampus
,
27
,
642
652
.
Voss
,
J. L.
,
Gonsalves
,
B. D.
,
Federmeier
,
K. D.
,
Tranel
,
D.
, &
Cohen
,
N. J.
(
2011
).
Hippocampal brain-network coordination during volitional exploratory behavior enhances learning
.
Nature Neuroscience
,
14
,
115
120
.
Voss
,
J. L.
,
Warren
,
D. E.
,
Gonsalves
,
B. D.
,
Federmeier
,
K. D.
,
Tranel
,
D.
, &
Cohen
,
N. J.
(
2011
).
Spontaneous revisitation during visual exploration as a link among strategic behavior, learning, and the hippocampus
.
Proceedings of the National Academy of Sciences, U.S.A.
,
108
,
E402
E409
.
Wang
,
J. X.
,
Cohen
,
N. J.
, &
Voss
,
J. L.
(
2015
).
Covert rapid action-memory simulation (CRAMS): A hypothesis of hippocampal-prefrontal interactions for adaptive behavior
.
Neurobiology of Learning and Memory
,
117
,
22
33
.
Warren
,
D. E.
,
Duff
,
M. C.
,
Tranel
,
D.
, &
Cohen
,
N. J.
(
2011
).
Observing degradation of visual representations over short intervals when medial temporal lobe is damaged
.
Journal of Cognitive Neuroscience
,
23
,
3862
3873
.
Watson
,
P. D.
,
Voss
,
J. L.
,
Warren
,
D. E.
,
Tranel
,
D.
, &
Cohen
,
N. J.
(
2013
).
Spatial reconstruction by patients with hippocampal damage is dominated by relational memory errors
.
Hippocampus
,
23
,
570
580
.
Yee
,
L. T. S.
,
Warren
,
D. E.
,
Voss
,
J. L.
,
Duff
,
M. C.
,
Tranel
,
D.
, &
Cohen
,
N. J.
(
2014
).
The hippocampus uses information just encountered to guide efficient ongoing behavior
.
Hippocampus
,
24
,
154
164
.
Yonelinas
,
A. P.
(
2002
).
The nature of recollection and familiarity: A review of 30 years of research
.
Journal of Memory and Language
,
46
,
441
517
.