Sustained Attention and Spatial Attention Distinctly Influence Long-term Memory Encoding

In our daily lives, we fail to remember many of the items that we encounter, largely because we were not paying sufficient attention. Although attention and long-term memory are clearly intertwined (Decker & Duncan, 2020; Hannula, 2018; Aly & Turk-Browne, 2017; Chun & Turk-Browne, 2007), past work investigating the relationship between attention and long-term memory often treats attention as a monolithic cognitive construct. However, attention has numerous subcomponents, any one of which could underlie the relationship with memory (Hakim, Adam, Gunseli, Awh, & Vogel, 2019; Robison & Brewer, 2019; Chun, Golomb, & Turk-Browne, 2011; Poole & Kane, 2009; Fan, McCandliss, Fossella, Flombaum, & Posner, 2005). For example, sustained attention fluctuates from trial to trial, and spatial attention can be oriented to different locations in space. However, studies investigating long-term memory have generally lacked the ability to disentangle whether memory failures reflect poor sustained attention, misallocated spatial attention, or both. This raises two potential explanations for the relationship between attention and long-term memory: In a unified model of attention and memory, memory failures could be attributable to convergent failures of multiple forms of attention. That is, lapses of sustained attention coincide with moments when spatial attention is oriented to the wrong location and together influence memory encoding. Alternatively, in a composite model of attention and memory, memory failures arise from failures of any particular attentional subcomponent. That is, lapses of sustained attention and misoriented spatial attention could each account for distinct failures of memory encoding. Thus, the goal of this study was to determine whether sustained attention and spatial attention exert overlapping or distinct influences on long-term memory.

Spatial attention and sustained attention are each important for long-term memory encoding. Spatial attention has been traditionally studied using spatial cues, in which participants orient spatial attention to a particular location. The influence of spatial cues can be measured behaviorally and via multivariate analyses of brain data (Foster, Sutterer, Serences, Vogel, & Awh, 2017; Sprague & Serences, 2013). Spatially cueing an item has further been shown to improve long-term memory (Ziman, Lee, Martinez, & Manning, 2019; LaRocque et al., 2015; Turk-Browne, Golomb, & Chun, 2013; Uncapher, Hutchinson, & Wagner, 2011). A largely distinct literature has studied the influence of sustained attention on behavior, traditionally using continuous performance tasks in which participants are required to repeatedly respond (deBettencourt, Keene, Awh, & Vogel, 2019; Rosenberg, Noonan, DeGutis, & Esterman, 2013). The hallmark signature of sustained attention is that behavioral and brain systems fluctuate between advantageous and disadvantageous states across trials over time (Rosenberg et al., 2016; deBettencourt, Cohen, Lee, Norman, & Turk-Browne, 2015; Esterman, Noonan, Rosenberg, & DeGutis, 2013). Trial-by-trial fluctuations of sustained attention have been shown to influence which items will be later remembered (deBettencourt, Norman, & Turk-Browne, 2018). Furthermore, sustained attention and working memory are closely linked cognitive constructs, although not synonymous (Hakim, deBettencourt, Awh, & Vogel, 2020; Adam & deBettencourt, 2019; deBettencourt et al., 2019; Hakim et al., 2019). Extant work, however, has not examined whether these fluctuations of sustained attention could be explained by trial-by-trial differences in the deployment of spatial attention. Thus, the central goal of the present work was to determine whether sustained and spatial attention have related or distinct impacts on long-term memory performance.

In this study, we examine whether sustained and spatial attention exert distinct or common influences on long-term memory encoding using behavioral and neural signatures. To measure long-term memory behavior with high sensitivity, we employed a continuous report task, in which participants report a particular dimension of a stimulus along a continuous space (Tompary, Zhou, & Davachi, 2020; Xie, Park, Zaghloul, & Zhang, 2020; Biderman, Luria, Teodorescu, Hajaj, & Goshen-Gottstein, 2019; Fan & Turk-Browne, 2016; Richter, Cooper, Bays, & Simons, 2016; Sutterer & Awh, 2016). To resolve the moment-by-moment influence of attention on long-term memory, we analyze multivariate EEG signals during time intervals before encoding. We manipulate spatial attention via spatial cues and observe the influence of sustained attention via trial-by-trial fluctuations of working memory performance for spatially attended and unattended items. In Experiment 1, we present a behavioral paradigm that captures how sustained and spatial attention distinctly influence long-term location memory on a continuous report task. In Experiment 2, we identify a multivariate EEG signature of sustained attention that predicts long-term memory performance independent of variations in spatial attention. In Experiment 3, we extend these findings to show that spatial and sustained attention influence color memory. Finally, collapsing across all studies, we show that individual differences of the influence of sustained and spatial attention predict unique variance in long-term memory performance.

The goal of this experiment was to examine whether signatures of sustained and spatial attention predict long-term memory. We hypothesized that long-term memory would reflect trial-by-trial fluctuations of sustained attention, as well as the prioritization of cued stimuli by spatial attention. We obtained a sensitive measure of long-term memory accuracy by asking participants to report their memory for the spatial location of trial-unique objects using a continuous report task.

In Experiments 1a and 1b, a combined 52 adults participated for University of Chicago course credit or $20 payment ($10/hr). In all studies, we targeted data collection from 25 participants before exclusion. In Experiment 1a, 25 adults (15 female, mean age = 23.2 years) participated, and in Experiment 1b, 27 adults participated (17 female, mean age = 24.2 years). We excluded any participants whose performance exceeded 3 SDs from the population mean (n = 2 in Experiment 1a; n = 1 in Experiment 1b) and participants who were outliers in terms of study completion (n = 2 in Experiment 1b completed 50% of the study in the allotted time). Therefore, the final sample of participants was 23 for Experiment 1a and 24 for Experiment 1b. All participants in this experiment and the following experiments reported normal or corrected-to-normal color vision and provided informed consent to a protocol approved by the University of Chicago institutional review board.

Participants were seated facing an LCD monitor (120-Hz refresh rate) in a testing room. In Experiment 1a, participants were approximately 70 cm from the monitor, and in Experiment 1b, they were approximately 88 cm from the monitor, because of a reconfiguration of the behavioral testing rooms. Stimuli were presented in Python using PsychoPy (Peirce, 2007).

Trial-unique real-world object pictures were presented on a gray background (Brodeur, Guérard, & Bouras, 2014; Brady, Konkle, Alvarez, & Oliva, 2008). At encoding, these images (subtending 3° visual angle) were presented along a light gray ring (at 5° eccentricity). A black fixation dot (0.5°) appeared at the center of the screen. Peripheral spatial cues (black dots, 0.5°) appeared along the gray ring.

In Experiment 1a, on each working memory trial, a peripheral spatial cue briefly appeared (250 msec) along the ring and participants were instructed to covertly attend to the cued spatial location (Figure 1A). After an extended prestimulus interval (2–4 sec), four items (trial-unique object pictures) briefly appeared along the ring (250 msec). Participants were instructed to hold the items in mind over a retention interval (2 sec). Then, one of the items reappeared at the center, and the mouse cursor was initialized to a random position along the ring. Participants reported the original location of the probed item by clicking along the ring with the mouse. On valid trials (75%), the location of the cue matched the location of the probed item. On invalid trials (25%), one of the other three uncued items was probed. After the response, there was a blank intertrial interval (1–2 sec). Participants were instructed to maintain central fixation throughout the trial. Participants completed 16 blocks, each consisting of 24 working memory trials, and the cue position was counterbalanced across trials within a block.

In the long-term memory phase, we examined memory items that had appeared in the working memory phase. In each block, these items could be probed (18 cued items from valid trials, 6 uncued items from invalid trials) or unprobed (6 items from invalid trials). By definition, there was no measure of working memory performance for unprobed items. A balanced number of new object pictures appeared (30 per block); these items otherwise never appeared during the experiment. First, participants completed a recognition memory rating for each item. Below the image, a 4-point confidence rating scale appeared below the image. When participants made their responses (using the keys 1–4), it was briefly displayed on the scale (0.5 sec). Then, for all old items, participants completed a location memory continuous report. The fixation dot turned white, and the participant was instructed to retrieve the original location of that item for 1 sec. Then, the fixation dot turned black, the mouse cursor appeared randomly along the ring, and the participant reported the original location for that object along the ring. After each item, there was a blank intertrial interval (0.5 sec).

In Experiment 1b, half of the blocks were identical to Experiment 1a with a single cue that was either valid or invalid (cued/uncued). In the other half of the blocks, the encoding arrays were preceded by four dots (neutral). These four dots indicated the location of each of the items but provided no information about which item was most likely to be tested. As such, this condition controlled for the visual presentation of peripheral dots and provided the same temporal information about an upcoming memory array. The order of single cued blocks and neutral cued blocks were randomized, such that there were two of each type every four blocks.

Participants performed this task for 2 hr or until they completed 16 blocks in total (384 trials). In Experiment 1a, participants completed, on average, 345 (89.95 %) of the maximum 384 trials, ranging from 240 to 384. In Experiment 1b, participants completed 360 (93.75 %) of the maximum 384 trials, ranging from 312 to 384.

Location memory was measured via continuous report and analyzed as response error, or the angular difference between the original minus reported location (−180° to 180°). We primarily conducted subsequent memory analyses, by focusing our analyses on long-term location memory. We examined whether long-term memory reflected differences in spatial attention (cued vs. uncued items) and differences in sustained attention (operationalized as trial-by-trial fluctuations, for cued, uncued, and unprobed items). For recognition memory, high-confidence old responses were treated as remembered and all other responses as forgotten to calculate item recognition memory hit rate. Recognition memory performance was summarized as a single nonparametric measure of sensitivity (A′).

Descriptive statistics are reported as the mean and 95% confidence interval (CI) of the bootstrapped distribution. If the hypothesis was directional, one-sided tests were performed. Trial-wise correlations were computed using the Pearson correlation coefficient for each participant. The reliability of the correlations was assessed across participants. Because some of the data violated the assumption of normality, nonparametric statistics were performed by resampling participants with replacement 100,000 times. The p value corresponds to the proportion of the iterations in which the bootstrapped mean was in the opposite direction (Efron & Tibshirani, 1986). Any p values smaller than one in 1000 were approximated as p < .001.

We obtained a sensitive measure of long-term location memory as the absolute response error from the original location (d_ltm = 61.00°, 95% CIs [54.36, 66.83]). We conducted a subsequent memory analysis, sorting trials according to long-term memory performance (Figure 2A).

We were interested in whether trial-by-trial fluctuations of sustained attention influenced long-term memory performance. We examined performance in the working memory phase (d_wm = 18.86°, 16.11, 23.17), as a more proximal assay of attentional state. Across all probed trials, working memory response error was reliably correlated with long-term memory response error (r = .19, .15–.23, one-tailed p < .001). To quantify this relationship, we calculated a linear fit to relate long-term memory (bin number, 1–8) to working memory response error. We observed a reliably positive slope relating long-term memory to working memory (m = 2.71, 2.11–3.46; one-tailed p < .001; Figure 2B). This correlation between working memory and long-term memory performance is consistent with prior work that working memory serves as a “gateway” that determines the bandwidth of encoding into long-term memory (Fukuda & Vogel, 2019).

The correlation between working memory and long-term memory could be explained in two distinct ways. On the one hand, successful prioritization of the cued item over the other items in the display (spatial attention) could influence long-term memory. On the other hand, memory outcomes might reflect a broader fluctuation of attentional state (sustained attention) that could impact the quality of memory for all items, regardless of whether they were spatially attended or not. We examined each of these factors in turn:

First, we examined whether long-term memory reflected differences in spatial attention. Spatial attention benefitted long-term memory, as cued items were better remembered (d_cued = 57.95°, 50.69–64.40°; d_uncued = 70.14°, 65.03–74.78°; one-tailed p < .001). We also calculated a linear fit to relate long-term memory (bin number, 1–8) to the proportion of trials that had been cued in each bin. Indeed, we observed a reliably negative slope (m = −0.02, −.03 to −0.01; one-tailed p < .001; Figure 2C). Thus, spatially attending to items enhanced long-term memory fidelity.

To examine whether sustained attention shaped long-term memory performance independently of spatial attention, we separately analyzed cued and uncued items. There were reliably positive correlations between trial-by-trial fluctuations of working memory response error with long-term memory, for cued items (r = .15, .12–.20, one-tailed p < 001) and uncued items (r = .19, .13–.26, one-tailed p < .001). We further quantified this relationship as a positive slope between long-term memory bin and working memory response error for cued items and uncued items (m_cued = 2.02, 1.49–2.78; m_uncued = 3.53, 2.24–4.72; one-tailed ps < .001; Figure 2B). This revealed the influence of trial-by-trial fluctuations of sustained attention on long-term memory performance, regardless of whether the items were spatially attended.

We further posited that sustained attention could broadly impact memory for multiple items from the same display, reflecting synchronous fluctuations among all simultaneously displayed items. We generally measured long-term memory for a single item per display, which is unable to speak to these broader fluctuations. However, on trials when the cue was invalid, we also measured long-term location memory of the item that was initially cued but unprobed during the working memory phase of the task. As these items were unprobed, we did not obtain a working memory response for these unprobed items. We therefore examined whether the long-term memory for the uncued item was correlated with long-term memory for the unprobed item from the same display. For each participant, we correlated the long-term memory response error for the unprobed and uncued items from the same display. Across participants, the correlation between unprobed and uncued items was reliably positive (r = .6, .03–.10, one-tailed p < .001). We further quantified this relationship as a slope relating the long-term memory bin for the uncued item to long-term memory for the unprobed item (m = 1.49, 0.64–2.49; one-tailed p < .001; Figure 2B). This provides evidence that trial-by-trial fluctuations of sustained attention have broad influences on long-term memory for multiple items presented simultaneously.

We conducted this experiment to replicate and extend the previous findings. Half of the blocks were the same as Experiment 1a (single cues, valid or invalid). In the other half of the blocks (neutral blocks), all images were preceded by four cues, which indicated the spatial location of each item but provided no information about which item was likely to be tested. We again operationalized performance as absolute response error in the long-term memory phase (d_ltm = 63.72°, 58.43–68.52°) and in the working memory phase (d_wm = 23.22°, 19.79–26.90°). We observed a reliably positive correlation between long-term memory and working memory response error across all probed items (r = .19, .16–.23; one-tailed p < .001). We further quantified this relationship as a slope relating long-term memory bins to working memory response error (m = 3.54, 2.84–4.44; one-tailed p < .001; Figure 3A and 3B). This replicated the finding of a general correlation between working memory and long-term memory performance.

We next examined whether this general correlation reflected distinct contributions of spatial and sustained attention. Indeed, spatial attention influenced long-term memory: Neutrally cued items were remembered worse than cued items (d_cued = 60.39°, 54.60–65.78; d_neutral = 64.32°, 58.66–69.27; one-tailed p < .001) and better than uncued items (d_uncued = 71.12°, 66.87–75.82; one-tailed p = .004). Moreover, we quantified the effect of spatial attention on long-term memory, by repeating our binning analyses from Experiment 1a within the single cued blocks. We replicated the finding from Experiment 1a that the proportion cued decreased across bins (m = −0.02, −0.01 to −0.02, one-tailed p < .001; Figure 3C). These findings confirm the strong influence of spatial attention on long-term memory.

We predicted that long-term memory would also reflect trial-by-trial fluctuations of sustained attention, as observed in Experiment 1a. First, we correlated long-term memory and working memory response error for all probed items within each condition (r_cued = .18, .14–.24; r_uncued = .21, .14–.29; r_neutral = .23, .17–.28; one-tailed ps < .001). Next, we calculated the slope that related long-term memory (bin number, 1–8) to working memory response error (Figure 3B). We observed a reliably positive slopes for cued, uncued, and neutral items (m_cued = 2.43, 1.74–3.29, m_uncued = 4.04, 2.91–5.29; m_neutral = 3.80, 2.94–4.90; one-tailed ps < .001). Finally, we examined whether long-term memory was related across items from the same display using the unprobed items. Long-term memory was reliably correlated between items from the same display (r = .05, 0.00–0.10, one-tailed p = .03), which replicates our finding from Experiment 1a. However, this correlation was reliable between the uncued and unprobed in the blocks with single cues (r = .08, .02–.17, one-tailed p = .01), and not in the blocks with neutral cues (r = .02, −.04–.08, one-tailed p = .26). We further quantified this relationship between items from the same display by calculating the slope across bins (m = 0.98, –0.15 to 2.22, one-tailed p = .05).

We designed this experiment to measure long-term location memory via continuous report, in order to obtain a sensitive measure of memory fidelity. However, we also measured item recognition memory. Overall recognition memory sensitivity was well above chance in Experiment 1 (A′ = 0.83; 0.80–0.85; one-tailed p < .001 vs. chance = 0.5). We examined whether recognition memory correlated with working memory performance. Working memory response error was lower for items that were later recognized (d_recog = 17.39, 15.61–19.42; d_unrecog = 24.09, 21.29–27.22; one-tailed p < .001).

To examine the effect of spatial attention on long-term item recognition memory, we calculated the proportion of items that had been initially cued, separately for items that were later recognized versus not. A greater proportion of items that were later recognized were initially cued (q_recog = 0.77, 0.76–0.78; q_unrecog = 0.72, 0.71–0.73; one-tailed p < 001).

To examine the effect of trial-by-trial fluctuations of sustained attention on long-term item recognition memory, we calculated the working memory response error for items that were later recognized versus not. Working memory response error was lower for items that were later recognized, for cued items (d_recog = 11.83, 10.40–13.45; d_unrecog = 17.04, 14.63–19.83; one-tailed p < .001) and uncued items (d_recog = 31.16, 27.40–35.81; d_unrecog = 35.90, 31.16–41.29; one-tailed p = .006). We also observed evidence that recognition memory was correlated between items from the same display: The memory hit rate for unprobed items was higher when the item from the same display was recognized (h_recog = 0.18, 0.15–0.20; h_unrecog = 0.13, 0.11–0.16; one-tailed p < .001). In summary, recognition memory corroborated the findings from long-term location memory obtained via continuous report.

This experiment demonstrates a robust influence of two distinct attentional factors, spatial attention and sustained attention, on the encoding of visual information into long-term memory. Long-term memory reflected whether participants were spatially attending the memoranda, as cued items were better remembered. We also found evidence for strong trial-by-trial fluctuations of sustained attention that influenced both spatially attended and spatially unattended items. We also observed that long-term memory was correlated for cued and uncued items within the same display, which argues against spatial attention as the source of these trial-to-trial fluctuations. Thus, this experiment provided preliminary evidence for the composite model of attention and long-term memory.

However, these behavioral results alone cannot speak to whether these results are a function of stimulus encoding (e.g., increased pattern separation of the memory array items) or retrieval (e.g., stronger reinstatement of the memory array item during the probe). Therefore, in Experiment 2, we extended these findings by measuring EEG activity while participants performed a similar task. This provided the opportunity to identify neural signals that tracked fluctuations in sustained attention and the current locus of covert spatial attention. To anticipate the findings, differences in sustained attention were detectable based on neural activity even before stimulus onset, and this appears to be separate from fluctuations in the quality of spatial orienting of attention.

The goal of this experiment was to characterize the neural signals of sustained and spatial attention that predict long-term memory. We collected eye-tracking and EEG data while participants performed the task from Experiment 1a.

Forty-two adults (23 female; mean = 23.5 years) completed Experiment 2 for $60 payment ($15/hr). A larger number of participants were chosen so as to have an adequate sample size after excluding participants who had excessive EEG or eye artifacts (six participants who all had fewer than half of the trials remaining after artifact rejection of the prestimulus period) or problems with EEG or eye-tracking equipment during the recording session (six participants). The final sample size was 30 participants. These exclusion criteria were determined a priori and are consistent with prior studies from our laboratory.

Participants were seated approximately 75 cm from an LCD monitor (120-Hz refresh rate) in a shielded booth.

Stimuli and procedure are the same as in Experiment 1a.

We monitored gaze position using a desk-mounted infrared eye-tracking system (EyeLink 1000 Plus, SR Research). Gaze position was sampled at 1000 Hz, and head position was stabilized with a chin rest.

We recorded EEG activity using 30 active Ag/AgCl electrodes mounted in an elastic cap (Brain Products actiCHamp). We recorded from International 10–20 sites: FP1, FP2, F3, F4, F7, F8, Fz, FC1, FC2, FC5, FC6, C3, C4, Cz, CP1, CP2, CP5, CP6, P3, P4, P7, P8, Pz, PO3, PO4, PO7, PO8, O1, O2, and Oz. Two electrodes were placed on the left and right mastoids, and a ground electrode was placed at position FPz. All sites were recorded with a right-mastoid reference, and were rereferenced off-line to the algebraic average of the left and right mastoids. Eye movements and blinks were recorded with passive electrodes using horizontal and vertical EOG. Data were filtered on-line (0.01–250 Hz) and were digitized at 1000 Hz using BrainVision Recorder.

We extracted data relative to the onset of spatial attention cues (−300 to 1500 msec relative to cue onset). We used an automatic pipeline implemented in prior publications by our laboratory following a suggested EEG artifact rejection procedure (Luck, 2014). Data were automatically examined for EEG artifacts (amplifier saturation, drifting, dropout, and excessive high-frequency noise) and EOG artifacts (blinks and saccades). We also extracted the same time period (−300 to 1500 msec relative to cue onset) from the eye-tracking data. Using the automatic pipeline implemented in prior publications by our laboratory, these data were automatically examined for eye artifacts (blinks and eye movements exceeding 0.5° from fixation). On average, 11% of trials were rejected because of eye artifacts, labeled as blinks (8%) and/or eye movements (10%). After artifact rejection, we manually inspected all trials using EEGLAB in MATLAB. Participants were excluded if fewer than half of the trials remained after discarding those with artifacts. On average, 295 (77%) trials remained per participant after artifact rejection.

For all artifact-free trials, we used multivariate pattern classification to predict long-term memory from the prestimulus time window. We decomposed EEG ERPs into oscillatory bands (4–7 Hz, 8–12 Hz, 13–16 Hz, 16–20 Hz, 20–25 Hz, 25–30 Hz) by bandpass filtering the data and applying the Hilbert transform (Kikumoto & Mayr, 2020; Weidemann & Kahana, 2020; Manning, Polyn, Baltuch, Litt, & Kahana, 2011; Guderian, Schott, Richardson-Klavehn, & Düzel, 2009). To equate for power differences across frequency bands without removing sustained prestimulus signals, we demeaned the signal based on the global average power within each band.

Each trial therefore provided a multifrequency vector as an input to the multivariate classification. Each trial was assigned a label for multivariate classification, corresponding to the binary categorization of whether that trial was accurately or inaccurately remembered. Accurate trials were trials where the long-term memory response error was below the median response error in the long-term memory phase. Inaccurate trials were trials where the long-term response error was greater than the median response error in the long-term memory phase. To control for spatial imbalances, we calculated a separate median response error for each quadrant. Therefore, accurate and inaccurate trials contained the same number of items per quadrant. These labels were not redundant with the accurate/inaccurate labels that would have been obtained from the working memory phase (only 56% of trials would have received the same label).

We split all accurate trials into two bins and all inaccurate trials into two bins. We averaged all trials within each bin to maximize the signal to noise of our EEG data. One accurate bin and one inaccurate bin served as training data, and the other accurate and inaccurate bins served as test data. The assignment of trials to bins was random, and we repeated this random assignment 1000 times for each participant. In the end, the classifier was trained on two vectors of length 180 (30 electrodes, six frequencies) with binary labels. We trained a multivariate classifier using L2-penalized logistic regression (C = 1) with the scikit-learn package in Python. We tested the classifier on two vectors of length 180 (30 electrodes, six frequencies) with binary labels.

To examine sustained attention over time, we trained separate classifiers at different moments of time following cue presentation. For each 100-msec time window, we averaged the power in each frequency band for each electrode. We repeated this entire procedure for 10-msec time steps. We conducted statistical analyses across participants to compare classification accuracy at each time point to theoretical chance (50%) as well as a shuffled null, for which we permuted the labels. Code that describes the entire multivariate decoding procedure of sustained attention is available on-line.

For all artifact-free trials, we used multivariate pattern classification to predict the quadrant where the cue had appeared. We repeated the same general procedure as what was used to decode sustained attention. We decomposed EEG ERPs into alpha (8–12 Hz) power. We chose to focus our analysis on the alpha frequency band a priori based on extensive work showing classification of spatial attention in this band (e.g., Foster et al., 2017).

Each valid trial therefore provided a vector input to the multivariate classification. Each trial was assigned a label for multivariate classification, corresponding to the quadrant where the cue had appeared (1–4). We downsampled trials such that there was an equal number of trials per quadrant.

We determined for each trial which quadrant was cued and whether long-term memory for that trial was accurate or inaccurate. We split all trials within each quadrant according to the median response error for that quadrant. Within each quadrant, we split all accurate trials into two bins and all inaccurate trials into bins. Therefore, in total, we had 16 bins of data (8 bins of accurate trials, 8 bins of inaccurate trials). We averaged all trials within each bin to maximize the signal to noise of our EEG data. We trained a classifier using a combined training set (one bin per quadrant of accurate trials, one bin per quadrant of inaccurate trials). We tested the classifier separately on accurate (one bin per quadrant) and inaccurate data (one bin per quadrant). The assignment of trials to bins was random, and we repeated this random assignment 1000 times for each participant. In the end, the classifier was trained on eight vectors of length 30 (30 electrodes, one frequency) with Labels 1–4. We trained a multivariate classifier using L2-penalized logistic regression (C = 1) with the scikit-learn package in Python. We tested the classifier twice, each time on four vectors of length 30 (30 electrodes, one frequency) with Labels 1–4.

To examine spatial attention over time, we trained separate classifiers at different moments of time following cue presentation. We averaged the alpha power in each electrode over a 100-msec time window. We repeated this entire procedure for 10-msec time steps. We conducted statistical analyses across participants to compare classification accuracy at each time point to theoretical chance (25%) as well as a shuffled null, for which we permuted the labels. Code that describes the entire multivariate decoding procedure of spatial attention is available on-line.

Statistics are the same as in Experiment 1.

We again operationalized long-term memory as absolute response error (d_ltm = 61.10°, 95% CIs [56.14, 65.92]; Figure 4A). We replicated our findings from Experiment 1 of a general correlation between working memory and long-term memory. We observed a reliably positive correlation between long-term memory and working memory response error (r = .17, .13–.22, one-tailed p < 001). We further quantified this as a positive slope relating long-term memory (bin number, 1–8) to working memory response error (m = 2.21, 1.70–2.89; one-tailed p < .001; Figure 4B).

We next disentangled how long-term memory reflected both spatial and sustained attention. We replicated the finding that long-term memory reflected spatial attention, as items that were cued to be spatially attended were better remembered (d_cued = 57.69°, 52.35–62.98; d_uncued = 71.34°, 66.19–75.86; one-tailed p < .001). This relationship between spatial attention and long-term memory was further quantified as a negative slope of the proportion cued across bins (m = −0.023, −0.031 to −0.017; one-tailed p < .001; Figure 4C).

We also replicated the finding that long-term memory was affected by trial-by-trial fluctuations in sustained attention (Figure 4B). We observed a positive correlation between long-term memory and working memory response error for cued and uncued items (r_cued = .11, .08–.16; r_uncued = .18, .13–.24; one-tailed ps < .001). We further quantified this relationship as a slope relating long-term memory (bin number, 1–8) to working memory response error (m_cued = 1.09, 0.74–1.77; m_uncued = 3.18, 2.27–4.26; one-tailed ps < .001). Then, we examined the long-term memory response error for unprobed items. We observed a positive correlation between long-term memory for the uncued items and long-term memory for the unprobed items from the same display (r = .05, .00–.10, p = .02) and a positive slope (m = 1.05, −0.23–2.17; one-tailed p = .045). In summary, behavioral evidence replicated the observation that long-term memory reflects distinct signatures of spatial and sustained attention.

We observed from behavioral evidence the importance of trial-to-trial fluctuations of sustained attention for later memory. However, based on these behavioral findings, we were unable to specify when and how these fluctuations emerged. By examining EEG activity that tracks the participants’ current attentional state, our goal was to investigate the role of prestimulus fluctuations of attention on subsequent encoding into long-term memory. Thus, we tested whether ongoing neural activity could predict long-term memory success even before item presentation. Given that participants had to sustain attention over 2–4 sec following cue presentation, we targeted this prestimulus time window for analyses. We used multivariate classification to decode whether a trial was remembered accurately or inaccurately, relative to the median long-term memory response error per quadrant (Figure 5A). We reliably decoded long-term memory accuracy based on EEG patterns in the time window following the cue (mean accuracy = 57.23%, 95% CIs [52.87%, 62.58%]; n = 30, one-tailed p < .001, chance = 50%; t = 500–1000 msec; Figure 5B). That is, we could predict whether an upcoming item would be better remembered, even before it appeared. These decoding results are consistent with trial-by-trial fluctuations of sustained attention that occur before stimulus presentation and influence memory encoding.

The behavioral evidence suggested that sustained attention is distinct from spatial attention, as it broadly impacts both cued and uncued items from the same display. Therefore, we predicted that fluctuations of sustained attention should be independent from fluctuations of spatial attention. If this is the case, then our classifier of sustained attention should be robust, even when we collapse data across all spatial locations. Alternatively, fluctuations of sustained attention could have reflected general task disengagement. If so, fluctuations of sustained attention would predict worse spatial attention during low moments of sustained attention. To explore these possibilities, we investigated whether we could decode the cued quadrant (Figure 5C). During inaccurately remembered trials, we reliably decoded the cued position (mean accuracy = 31.82%, 95% CIs [29.99%, 33.98%]; n = 30; one-tailed p < .001, chance = 25%; t = 0–1500 msec; Figure 5D). We also reliably decoded the cued position during accurately remembered trials (mean accuracy = 31.99%, 95% CIs [30.17%, 34.90%]; n = 30; one-tailed p < .001, chance = 25%; t = 0–1500 msec; Figure 5D). Critically, in both trials, we decoded the cued location, consistent with the robust influence of spatial attention on behavior. However, there was no reliable difference in spatial attention decoding between accurate and inaccurate trials (two-tailed p = .76). These decoding results suggest that the ability to predict long-term memory differences during prestimulus windows is not driven by differences in spatial attention.

Overall recognition memory was above chance for all probed items (A′ = 0.87; 0.85–0.89; one-tailed p < .001 vs. chance = 0.5). Recognized items exhibited a lower absolute response error in the working memory phase (d_recog = 14.30, 12.31–17.35; d_unrecog = 16.48, 14.16–19.28; one-tailed p < .001). We observed an effect of spatial attention on subsequent item recognition, a greater proportion of items that were later recognized had been cued (q_recog = 0.76, 0.75–0.78; q_unrecog = 0.73, 0.71–0.74; one-tailed p < .001). We replicated our finding that items that were later recognized exhibited better working memory response errors for cued items (d_recog = 8.73, 7.59–10.43; d_unrecog = 10.13, 8.40–12.76; one-tailed p = .01), but not for uncued items (d_recog = 32.01, 25.95–40.75; d_unrecog = 33.74, 27.70–41.24; one-tailed p = .20) or unprobed items (h_recog = 0.16, 0.13–0.19; h_unrecog = 0.15, 0.12–0.18; one-tailed p = .15). Though item recognition results are largely consistent with results obtained from continuous report, these findings also suggest that continuous report may provide a more sensitive assay of how spatial and sustained attentional factors influence long-term memory.

Experiment 2 replicated and extended the behavioral findings from Experiment 1, with concurrent eye tracking to ensure spatial attention was maintained covertly. We demonstrated that prestimulus multivariate EEG patterns predicted later memory. These EEG decoding results are consistent with a multivariate EEG representation of sustained attentional state that preceded stimulus onset and complement the behavioral evidence for trial-to-trial fluctuations in behavior.

Furthermore, these behavioral and neural findings supported the composite model of attention and long-term memory, as fluctuations of sustained attention were distinct from spatial attention. These results also rule out the interpretation that trial-by-trial fluctuations of sustained attention reflect moments during which participants completely disengaged from the task: EEG activity showed that participants maintained covert spatial attention at the cued position, even when the cued stimulus was later inaccurately remembered. Thus, multivariate analyses of EEG data suggest that fluctuations in sustained attention can be distinguished from the waxing and waning of spatial attention, and do not reflect episodes of global disengagement with the task. Future studies could explore how prestimulus attentional states, including both sustained attention and spatial attention, influence stimulus encoding, maintenance, and reinstatement. One possibility is that attentional states (sustained and spatial attention) determine a cascade of subsequent processes, including working memory, that are all reflected in long-term memory.

Thus far, we have focused primarily on recall of the spatial position of the memoranda. Experiment 3 examined whether distinct attentional subcomponents would also influence long-term memory for other features that were not shared with the spatial cue, namely, color. We hypothesized that sustained and spatial attention would distinctly predict long-term memory for color.

In Experiment 3a and 3b, a combined 52 adults participated for University of Chicago course credit or $20 payment ($10/hr). In Experiment 3a, 25 adults (15 female, mean age = 23.0 years) participated. In Experiment 3b, 27 adults (20 female, mean age = 20.1 years) participated. Two participants were excluded from Experiment 3 (one from 3a and one from 3b) because of errors during data collection, resulting in a final sample size of 50.

Apparatus used is the same as in Experiment 1b.

A subset of the real-world object images from Experiments 1–2 selected based on the relative uniqueness of the shape outlines was manipulated to be a one-dimensional color mask. The color of each image was sampled randomly from a 360° Hue Saturation Value space, with saturation and value of 1, and remapped to red, green, blue values for presentation in PsychoPy.

In the memory probes, participants reported the color memory for each item instead of its spatial location (Figure 6A). The item, colored dark gray, appeared at the center of the screen, surrounded by a color wheel. The color wheel was randomly oriented for each trial, and this random orientation was held consistent between working memory and long-term memory tests. Based on piloting, we reduced the working memory array to two items, which were separated by a minimum distance of 40° in color space. In the long-term memory phase, participants made source memory judgments only for the probed images from the working memory phase, both cued or uncued. In Experiment 3a, participants completed 24 blocks of 16 trials, and participants completed, on average, 368 of the maximum 384 trials (95.83%), ranging from 272 (70.83%) to 384 (100%). In Experiment 3b, we eliminated the recognition memory judgments and reduced the block length. In Experiment 3b, participants completed 48 blocks of eight trials and all participants completed all 384 trials.

Behavioral analysis and statistics are the same as Experiments 1–2. All results were consistent for both Experiment 3a and 3b, and therefore we report the results as Experiment 3. Results for each study separately are available on-line.

The goal of this experiment was to replicate the findings from Experiments 1 and 2 and extend them to the color dimension. We operationalized long-term memory as absolute response error from the original color (d_ltm = 67.90°, 95% CIs [63.44, 71.58]). For each participant, we sorted and binned trials according to their long-term color memory performance (Figure 6B). We observed that long-term color memory bins reflected trial-by-trial fluctuations of sustained attention, via a positive correlation between long-term memory and working memory (r = .18, .16–.21; one-tailed p < .001). We quantified this relationship by relating long-term memory (bin number, 1–8) to working memory (m = 2.77, 2.38–3.26; one-tailed p < .001; Figure 6C).

We next examined whether long-term color memory also reflected distinct influences of spatial and sustained attention. To examine the influence of spatial attention, we observed that long-term color memory was enhanced by cues (d_cued = 66.75°, 62.37–70.41; d_uncued = 71.36°, 66.39–75.50; one-tailed p < .001). Furthermore, the proportion of cued items within each bin declined across bins (m = −0.008, −0.011 to −0.004; one-tailed p < .001; Figure 6D).

To examine the influence of sustained attention, we repeated the binning analysis within each cueing condition (Figure 6C). We observed that the long-term memory bin was positively related to working memory response error for cued items and uncued items (r_cued = .17, 0.14–0.20; r_uncued = .20, .16–.24; one-tailed ps < .001). We further quantified this relationship as a slope across long-term memory bins (m_cued = 2.25, 1.91–2.69, m_uncued = 3.23, 2.51–4.00; one-tailed ps < .001). In summary, behavioral evidence confirms that both spatial attention and sustained attention are critical in understanding long-term color memory.

We only collected long-term recognition memory ratings in Experiment 3a. Overall recognition memory sensitivity was well above chance for all probed items (A′ = 0.78, 0.74–0.82; one-tailed p < .001 vs. chance = 0.5). We also observed lower working memory response error for recognized versus unrecognized items (d_recog = 23.52, 19.95–29.51; d_unrecog = 29.51, 25.67–34.64; one-tailed p < .001). We did not observe a reliable effect of spatial attention: There was no difference in the proportion of cued items for recognized versus unrecognized trials (q_recog = 0.76, 0.75–0.78; q_unrecog = 0.75, 0.74–0.75; one-tailed p = .08). However, trial-by-trial fluctuations of sustained attention influenced item recognition memory. Working memory absolute response error was lower for items that were later recognized, for cued items (d_recog = 19.50, 16.66–24.47; d_unrecog = 23.69, 20.29–28.18; one-tailed p < .001) and uncued items (d_recog = 37.76, 29.59–51.29; d_unrecog = 46.14, 38.02–58.34; one-tailed p < .001).

These findings verified the influence of attention for long-term memory of a nonspatial feature. Both sustained and spatial attention distinctly related to long-term color memory. These results extend the findings from Experiment 1–2, as the spatial location was a feature shared by the cue and item. These results reaffirm that sustained attention fluctuations are a broad and general influence for memories that can be clearly dissociated from the effects of spatial attention.

The goal of this study was primarily to explore attention and long-term memory within individuals. However, we also examined between-subjects variations in long-term memory performance and the underlying attentional factors that drove those differences. We collapsed across data from all experiments to boost the number of participants (n = 127). We were interested in exploring the degree to which individual differences in long-term memory reflected individual differences in sustained or spatial attention.

We collapsed across data from all participants in Experiments 1–3 (1a, 1b, 2, 3a, 3b) to maximize the number of participants. We included all participants from Experiment 1 (n = 47), Experiment 2 (n = 30), and Experiment 3 (n = 50).

We examined measures of sustained and spatial attention from the binning analysis of Experiments 1–3. For each participant, we used our measure of the influence of sustained attention from the previous studies, the slope that related long-term memory to working memory absolute response error. This operationalization of sustained attention highlights the key influence of trial-by-trial fluctuations of performance on later memory. To eliminate any influence of spatial cues, we calculated the slope separately for cued and uncued items and averaged across conditions for a single measure of sustained attention.

We also used our measure of the influence of spatial attention from the previous studies, the slope that related long-term memory to the proportion of cued items within each bin. So that sustained and spatial attention had the same sign, we modified this value to be the slope that related long-term memory to the proportion of uncued items within each bin (1− proportion of cued items per bin). This operationalization of spatial attention highlights the key influence of spatial cues on later memory.

We examined how measures of sustained and spatial attention related to average long-term memory absolute response error. We implemented two complementary analyses, linear regression and nonparametric correlations. We ran regression diagnostics to ensure linearity and homoscedasticity. We also ensured our results were robust after detecting and excluding any outliers. To examine the reliability of these measures of sustained attention, we replicated all of our results after subselecting trials and after z-scoring within each of the five experiments (Experiment 1a, 1b, 2, 3a, and 3b).

Descriptive statistics are reported as the mean and 95% CI of the bootstrapped distribution. Correlations and partial correlations were computed using the nonparametric Spearman’s rank-order correlation function. Linear models were fit in R, and we report the adjusted r² values. F statistics evaluated different model fits. For the downsampling analyses, we report the 95% CIs across downsampling iterations.

We obtained a measure of the influence of sustained and spatial attention on performance from Experiments 1 to 3. We were interested in whether relying on either sustained or spatial attention could be effective strategies for long-term memory. As such, we quantified measures of the strength of the influence of sustained and spatial attention on long-term memory behavior for each participant. We also quantified a measure of overall long-term memory performance for each participant as average long-term memory response error.

Our measure of the influence of sustained attention was the slope that related trial-by-trial fluctuations of working memory and long-term memory performance. A more positive slope reflects a stronger influence of sustained attention on memory. We were interested in whether a stronger influence of sustained attention correlated with overall long-term memory performance. There was a reliably negative relationship between the influence of sustained attention on long-term memory and average long-term memory response error (β_sust = −2.68, −3.90 to −1.46; r² = .12; p < .001; Figure 7A). We also verified a relationship by correlating the influence of sustained attention on long-term memory and average long-term memory response error (r = −.38, −.52 to −.22; p < .001). That is, a stronger influence of sustained attention correlates with better average long-term memory performance.

Our measure of the influence of spatial attention was the slope that related the proportion of items that had been uncued within each long-term memory bin. A more positive slope reflects a stronger influence of spatial attention on long-term memory. There was reliably negative relationship between the influence of spatial attention and average long-term memory response error (β_spatial = −310, −459 to −161, r² = .11, p < .001; Figure 7B). We also verified a relationship by correlating the influence of spatial attention on long-term memory and average long-term memory response error (r = −.28, −0.43 to −0.11; p = .002). That is, a stronger influence of spatial attention correlates with better average long-term memory performance.

The critical question is whether these two signals, sustained and spatial attention, reflect separate influences on long-term memory. If so, we would predict that they are uncorrelated with each other and explain a unique portion of the variance. We found that individual differences in spatial attention and sustained attention were uncorrelated with each other (r = .02, −.15 to .20; p = .81; Figure 7C). In addition, they explained unique variance in individual differences in long-term memory: The partial correlation between sustained attention and long-term memory (r = −.38, −0.52 to −0.23; p < .001) and the partial correlation between spatial attention and long-term memory (r = −.29, −.44 to −.12; p < .001) were both reliable. The model using both signals as predictors explained more variance in individual differences in long-term memory (adjusted r² = .23) than a model that just included sustained attention, F(1, 125) = 17.63, p < .001, or spatial attention, F(1, 125) = 19.70, p < .001. In summary, individual differences for sustained attention and spatial attention distinctly influence long-term memory.

We examined the influence of sustained and spatial attention on long-term memory performance across participants in the three experiments. These findings revealed that sustained and spatial attention predict distinct variance in long-term memory performance.

Over three experiments, we found that spatial and sustained attention had robust and distinct effects on long-term memory. Behaviorally, long-term memory was superior when spatial attention was oriented toward the item. Neurally, we decoded spatial attentional states from multivariate analyses of EEG data before stimulus presentation. We also observed robust behavioral influences of trial-by-trial fluctuations of sustained attention, regardless of whether items were spatially attended or not. In addition, trial-by-trial fluctuations of sustained attention were correlated between multiple items from the same display. We successfully decoded evidence for these sustained attentional states, as we predicted future memory performance from multivariate analyses of EEG data before stimulus presentation. EEG measures of sustained attention were distinct from spatial attention, as they generalized across spatial positions, and the spatially attended location was equally well decoded during lapses of sustained attention. Moreover, this empirical pattern rules out a general task engagement explanation of the fluctuations associated with sustained attention. Finally, individual differences analyses revealed that either spatial attention or sustained attention were effective strategies for overall long-term memory performance. In summary, memory failures can be attributed to failures of distinct spatial or sustained attentional processes, as shown by these behavioral results, neural analyses, and individual differences. Thus, our work provides key evidence that begins to fractionate the monolithic relationship between attention and long-term memory.

This work provides critical support for a composite model of attention and long-term memory. That is, distinct attentional subcomponents can explain memory failures. We can conceptually illustrate the consequences of distinct attentional processes with an addendum to the traditional spotlight analogy: If spatial attention reflects the location of the spotlight on the stage, sustained attention corresponds to the number of audience members who are awake. Critical to this analogy is the idea that sustained and spatial attention can each vary independently. That is, even when attention is oriented correctly, it could be that sustained attention was deployed unsuccessfully. Furthermore, either the spotlight or the audience’s wakefulness can contribute to later memory of the performance, as suggested by the composite model of attention and long-term memory. Notably, the composite model does not require complete independence, and there may be situations where these distinct processes fluctuate in unison. Conversely, the unified model would suggest that sustained attention and spatial attention have completely overlapping influences (e.g., spotlight location and brightness), which we did not observe.

This study sheds light on how two distinct attentional processes impact encoding into long-term memory. These findings align with prior attention research that has shown spatial and sustained attention reflect distinct cognitive processes, both within and between individuals (Dowd & Golomb, 2019; Robison & Brewer, 2019; Poole & Kane, 2009; Fan et al., 2005). Our study makes the key contribution of extending the dissociable nature of sustained and spatial attention to the domain of long-term memory. In particular, this work advocates for a nuanced approach of the relationship between multifaceted attentional states and long-term memory. Traditionally, the relationship between attention and long-term memory has been investigated by manipulating top–down attention (Aly & Turk-Browne, 2017). Here, we argue that it is important to distinguish between multiple forms of attention, and specifically consider fluctuations of sustained attention, to understand how attention shapes memory.

This study also motivates further investigation of the links between sustained attention and memory. In particular, recent work has begun to explore the role of trial-by-trial fluctuations of sustained attention as a gateway for working memory and long-term memory (Hakim et al., 2020; deBettencourt et al., 2018, 2019). Sustained attention itself has represented a growing area within recent attention research (deBettencourt et al., 2015, 2018, 2019; Rosenberg et al., 2016; Esterman, Rosenberg, & Noonan, 2014; Esterman et al., 2013). However, memory studies more rarely consider the role of sustained attention for memory formation (cf. Adam & deBettencourt, 2019). In part, sustained attention has been traditionally measured via specific tasks, especially the sustained attention to response task. This study extends our conceptualization of sustained attention from the sustained attention to response task or highly similar tasks, to a broader definition of trial-to-trial fluctuations of task performance. This research characterizes sustained attention as a sustained cognitive state, detectable pretrial, that impacts encoding for multiple items from the display into working memory and eventually long-term memory. Future studies could examine how the sustained attentional states in this study relate to those described in traditional sustained attention tasks.

This composite model of attention and memory reveals how distinct attentional components influence long-term memory. A key goal going forward will be to explore how these distinct attentional processes influence memory systems (Aly & Turk-Browne, 2016; LaRocque et al., 2013). It also raises the possibility that previous demonstrations of prestimulus signals that predict memory may reflect trial-by-trial fluctuations of sustained and/or spatial attention (Günseli & Aly, 2020; Weidemann & Kahana, 2020; Adam, Mance, Fukuda, & Vogel, 2015; Gruber & Otten, 2010; Guderian et al., 2009; Otten, Quayle, Akram, Ditewig, & Rugg, 2006). By carefully considering the role of distinct attentional processes, specifically spatial and sustained attention, this approach may reveal insight into the intricate and complex relationship between attention and memory.

Finally, this work provides important suggestions for strategies to improve or worsen memory. Cueing attention enhances long-term memories at the attended location at the expense of memories elsewhere. In addition, harnessing sustained attention fluctuations may be another key for altering memory performance. Our findings suggest that advantageous moments of sustained attention enhance memories for items distributed across spatial locations. Thus, targeting optimally attentive states or optimally attentive moments, or even improving general sustained attention abilities (deBettencourt et al., 2015), may be a key strategy for broadly improving memory.

We thank A. Gale for assistance with data collection for Experiment 2, M. Bolouri for assistance with data collection for Experiment 3, and K. C. S. Adam, N. Hakim, and A. Tompary for comments on earlier versions of the paper.

Reprint requests should be sent to Megan T. deBettencourt, Institute for Mind and Biology, University of Chicago, 940 E 57th St., Chicago, IL, or via e-mail: [email protected].

Megan T. deBettencourt: Conceptualization; Formal analysis; Funding acquisition; Investigation; Methodology; Software; Visualization; Writing—Original draft; Writing—Review & editing. Stephanie D. Williams: Investigation; Methodology; Writing—Review & editing. Edward K. Vogel: Conceptualization; Funding acquisition; Resources; Supervision; Writing—Review & editing. Edward Awh: Conceptualization; Funding acquisition; Resources; Supervision; Writing—Review & editing.

This research was supported by National Institutes of Health (https://dx.doi.org/10.13039/100000002), grant number: F32MH115597, the Office of Naval Research (https://dx.doi.org/10.13039/100000006), grant number: N00014-12-1-0972, and National Institute of Mental Health (https://dx.doi.org/10.13039/100000025), grant number: R01MH087214 (M. T. dB.).

A retrospective analysis of the citations in every article published in this journal from 2010 to 2020 has revealed a persistent pattern of gender imbalance: Although the proportions of authorship teams (categorized by estimated gender identification of first author/last author) publishing in the Journal of Cognitive Neuroscience (JoCN) during this period were M(an)/M = .408, W(oman)/M = .335, M/W = .108, and W/W = .149, the comparable proportions for the articles that these authorship teams cited were M/M = .579, W/M = .243, M/W = .102, and W/W = .076 (Fulvio et al., JoCN, 33:1, pp. 3–7). Consequently, JoCN encourages all authors to consider gender balance explicitly when selecting which articles to cite and gives them the opportunity to report their article's gender citation balance. The authors of this article report its proportions of citations by gender category to be as follows: M/M = .42, W/M = .36, M/W = .11, and W/W = .11.