Abstract
Traditionally, neuroscience and psychology have studied the human brain during periods of “online” attention to the environment, while participants actively engage in processing sensory stimuli. However, emerging evidence shows that the waking brain also intermittently enters an “offline” state, during which sensory processing is inhibited and our attention shifts inward. In fact, humans may spend up to half of their waking hours offline [Wamsley, E. J., & Summer, T. Spontaneous entry into an “offline” state during wakefulness: A mechanism of memory consolidation? Journal of Cognitive Neuroscience, 32, 1714–1734, 2020; Killingsworth, M. A., & Gilbert, D. T. A wandering mind is an unhappy mind. Science, 330, 932, 2010]. The function of alternating between online and offline forms of wakefulness remains unknown. We hypothesized that rapidly switching between online and offline states enables the brain to alternate between the competing demands of encoding new information and consolidating already-encoded information. A total of 46 participants (34 female) trained on a memory task just before a 30-min retention interval, during which they completed a simple attention task while undergoing simultaneous high-density EEG and pupillometry recording. We used a data-driven method to parse this retention interval into a sequence of discrete online and offline states, with a 5-sec temporal resolution. We found evidence for three distinct states, one of which was an offline state with features well-suited to support memory consolidation, including increased EEG slow oscillation power, reduced attention to the external environment, and increased pupil diameter (a proxy for increased norepinephrine). Participants who spent more time in this offline state following encoding showed improved memory at delayed test. These observations are consistent with the hypothesis that even brief, seconds-long entry into an offline state may support the early stages of memory consolidation.
INTRODUCTION
The brain transitions between a series of profoundly different states across the course of a 24-hr day, cycling between rapid eye movement sleep, non-rapid eye movement sleep, and wakefulness—each with their own distinct neurobiology, phenomenology, and function. Now, it is becoming increasingly clear that wakefulness itself is far from homogenous (Wamsley & Summer, 2020; Poulet & Crochet, 2019; Vidaurre et al., 2018; Wang et al., 2018; Reimer et al., 2014). New evidence from neuroscience and psychology suggests that even during periods of unchanging sensory input, the waking brain spontaneously and rapidly cycles between two or more functional states, including at least one state characterized by attention to the external environment (here termed an “online” state), and at least one state characterized by a decoupling from external sensory input and a shift toward internally oriented attention (here termed an “offline” state; see Table 1). Unlike the long-timescale alternations between sleep and wake states, movement between online and offline states during wake may occur on a timescale of seconds, or perhaps even milliseconds (Higgins et al., 2020; Wamsley & Summer, 2020; Poulet & Crochet, 2019; Vidaurre et al., 2018; Reimer et al., 2016; McGinley, Vinck, et al., 2015; Smallwood & Schooler, 2015). Here, we test the hypothesis that even these ultra-short bouts of offline time could promote the reactivation and consolidation of recently formed memories.
Hypothesized Characteristics of Online and Offline Waking States
Dimension . | Online State . | Offline State(s) . |
---|---|---|
Subjective attentional focus | Present sensory environment | Task-unrelated thought and imagery |
Behavioral performance | ↑ task performance, reflecting increased attention to the present sensory environment | ↓ task performance, reflecting decreased attention to the present sensory environment |
NE neuromodulation | ↑ NE, reflected in transient pupil dilation | ↓ NE, reflected in decreased pupil diameter |
EEG power | ↓ Alpha and slow oscillation | ↑ Alpha and slow oscillation |
Memory processes | ↑ Encoding | ↓ Encoding |
↓ Consolidation | ↑ Consolidation |
Dimension . | Online State . | Offline State(s) . |
---|---|---|
Subjective attentional focus | Present sensory environment | Task-unrelated thought and imagery |
Behavioral performance | ↑ task performance, reflecting increased attention to the present sensory environment | ↓ task performance, reflecting decreased attention to the present sensory environment |
NE neuromodulation | ↑ NE, reflected in transient pupil dilation | ↓ NE, reflected in decreased pupil diameter |
EEG power | ↓ Alpha and slow oscillation | ↑ Alpha and slow oscillation |
Memory processes | ↑ Encoding | ↓ Encoding |
↓ Consolidation | ↑ Consolidation |
We expected to find evidence of at least two statistically discriminable states of wake—an “online” state optimized for encoding information in the present sensory environment, and one or more “offline” states optimized for consolidation of previously encoded information. NE = neuroepinephrine.
Evidence that wakefulness is composed of a series of transitions between distinct states comes from multiple independent literatures. In psychology, research on mind wandering uses subjective report to describe how participants spontaneously alternate between attention to external stimuli and attention to internally generated thought, feeling, and imagery (Smallwood & Schooler, 2015; Barron, Riby, Greer, & Smallwood, 2011; Schooler et al., 2011). During periods of offline mind wandering, participants are less responsive to external stimuli, indicating that the brain is partially decoupled from its sensory inputs while focusing on internal thought and imagery (Barron et al., 2011; Schooler et al., 2011; Smallwood, Beach, Schooler, & Handy, 2008). In rodent models, there is analogous evidence of intermittent decoupling from the external sensory environment. During waking exploration, rats' cortical neurons show desynchronized firing patterns (a “desynchronized” state). However, during pauses in exploration, cortical neurons begin to show slow, coordinated fluctuations in firing (a “synchronized” state), during which processing of the external sensory environment is reduced (Lee, Kheradpezhouh, Diamond, & Arabzadeh, 2020; Beaman, Eagleman, & Dragoi, 2017), norepinephrine (NE) is transiently decreased (McCormick, Nestvogel, & He, 2020; Poulet & Crochet, 2019; Reimer et al., 2014, 2016), and sharp-wave ripple bursts indicative of memory replay increase (McGinley, David, & McCormick, 2015; McGinley, Vinck, et al., 2015). Furthermore, the application of hidden Markov modeling to magnetoencephalography data suggests that at rest, humans move between different resting-state networks in a nonrandom fashion. For example, one study reported that participants spontaneously transitioned between states resembling the default-mode network—presumed to reflect internally oriented processing—and states associated with external sensory processing (Vidaurre et al., 2018). Although employing widely divergent methods, these literatures converge on a central concept: that the waking brain exhibits spontaneous state-like fluctuations on a timescale of seconds (Reimer et al., 2014; Smallwood et al., 2008) or milliseconds (Vidaurre et al., 2018), with at least one state being associated with processing external stimuli (an “online” brain state) and one with disengagement from the sensory environment and a focus on internally focused processing (an “offline” brain state).
Recent work from our own laboratory has reached similar conclusions. We created a data-driven model of the movement between online and offline states during a 30-min sustained attention to response task (SART) by simultaneously measuring high-density EEG, pupillometry, and RTs, while intermittently prompting participants to report on their subjective focus of attention (Wamsley & Summer, 2020). Using a machine-learning approach, we reported that a sequence of SART trials could be described as a series of transitions between discrete online and offline states. During online trials, participants showed less alpha and slow oscillation power, faster RTs, and increased attentional focus on the ongoing task (Wamsley & Summer, 2020). During offline trials, participants showed large increases in alpha and slow oscillation EEG power, concomitant with slowed RTs and decreased subjective attention to the ongoing task. While there was a positive correlation between time spent offline and the retention of just-learned declarative information, this relationship did not reach statistical significance (r = .31, p = .19).
Still, there remains strong reason to hypothesize that brief, seconds-long entry into an offline state could benefit memory. Certainly, we know that longer periods of offline time during sleep and resting wake support consolidation. When participants sleep or rest with their eyes closed following learning, memory retention is improved in comparison to control groups who perform an active task (Paller, Creery, & Schechtman, 2021; Wang et al., 2021; Wamsley, 2019; Craig & Dewar, 2018; Stickgold & Walker, 2013; Cohen, Pascual-Leone, Press, & Robertson, 2005). These observations have not typically been considered to apply to short, seconds-long timescales—studies normally examine a full night of sleep, a 60- to 90-min nap, or at the very least, a 15- to 30-min period of waking rest.
Yet brief, seconds-long entry into an offline state could similarly benefit memory retention, allowing organisms to quickly switch between a neurobiological milieu optimized for encoding, and one that is optimized for consolidation. This is important because the encoding of new information and its subsequent consolidation are thought to rely on incompatible neural dynamics (Hasselmo & McGaughy, 2004; Buzsáki, 1998; Chrobak & Buzsáki, 1996; McClelland, McNaughton, & O'Reilly, 1995). For example, the successful encoding of new experiences into memory is associated with attention to the external environment (Feng, D'Mello, & Graesser, 2013; Risko, Anderson, Sarwal, Engelhardt, & Kingstone, 2012; Chun & Turk-Browne, 2007; Troyer & Craik, 2000), the presence of relatively desynchronized cortical EEG patterns, information flow into the hippocampus via entorhinal cortex, and high levels of NE (Hoffing & Seitz, 2015; McIntyre, McGaugh, & Williams, 2012). In contrast, brain states that facilitate consolidation (e.g., sleep and resting wake) are characterized by a reduction in externally oriented sensory processing, a profound increase in low-frequency EEG synchronization, the reactivation of prior memory via hippocampal sharp-wave ripples flowing out of entorhinal cortex, and a reduction in tonic levels of NE, punctuated by transient bursts of high NE activity (Eschenko, Magri, Panzeri, & Sara, 2012; Eschenko & Sara, 2008; Aston-Jones & Bloom, 1981).
Extended periods of offline time, such as sleep and extended rest, occur rarely during a busy day. If the initial stages of consolidation in the first few hours after learning must occur offline, a rapid alternation between states could allow organisms to quickly enter a transient state optimized for consolidation. In line with this hypothesis, recent observations suggest that, in humans, memory is reactivated during postlearning rest selectively during the expression of resting-state networks thought to promote inwardly oriented attention and inhibit sensory processing (Higgins et al., 2020). Likewise, in rodents, sharp-wave ripple bursts associated with memory reactivation are preferentially expressed during the offline “synchronized” cortical state (McGinley, David, et al., 2015; McGinley, Vinck, et al., 2015).
Here, we continued to pursue the question of whether seconds-long bouts of offline time promote memory consolidation in human participants. Participants encoded a declarative memory task just before a 30-min retention interval. Following our prior work, we parsed this 30-min interval into a series of “online” and “offline” states in a data-driven manner (Wamsley & Summer, 2020). As described below, we arrived at a three-state model, including one online and two offline states. Our central hypothesis was that the retention of just-learned information would be superior when participants spent more time in a spontaneously occurring offline brain state.
METHODS
Overview of the Approach
After listening to a short story, participants completed a 30-min SART, during which EEG, pupil diameter, and RTs were recorded. On 24 of 324 SART trials, participants also self-reported the current contents of their subjective experience (“thought probe trials”). Following our prior work, we parsed this 30-min interval into a series of “online” and “offline” states (Wamsley & Summer, 2020). In brief, we accomplished this by developing a naive Bayes classifier to label each 5-sec SART trial by state, on the basis of EEG spectral power, RTs, pupil diameter (as a proxy for NE levels, see Joshi & Gold, 2020; Murphy, O'Connell, O'Sullivan, Robertson, & Balsters, 2014), and participants' assessment of their current attentional focus.
Participants
A total of 65 participants enrolled in the study. Selection of sample size was driven by our goal of detecting an association between offline time and memory retention of r = .31. The original enrollment target was n = 80, but data collection was cut short at the onset of COVID-19–related restrictions in 2020. Participants were required to be full-time students between the ages of 18–29 years, to be native English speakers, and to report that they had never been diagnosed with a sleep or attention-deficit disorder. Participants were asked to keep a regular sleep schedule for the three nights before the study and to refrain from consuming caffeine after 10 a.m. on the day of the study. Following exclusions (see below), n = 46 participants were included in the final analysis (mean age = 19.5 years ± 1.2 SD; 34 women/12 men). This research was approved by Furman University's institutional review board. All participants signed informed consent and were compensated with either payment or course credit.
Procedure
Upon arrival at the laboratory, participants completed initial questionnaires including demographics forms, the Epworth Sleepiness Scale (Johns, 1991), the daydream frequency subscale of the Imaginal Processes Inventory (a measure of trait daydream frequency; Singer & Antrobus, 1972), the Mindfulness Attention and Awareness Scale (a measure of trait mind wandering propensity; Brown & Ryan, 2003), and a retrospective sleep log.
Participants were then prepared for EEG and pupillometry recording. For the majority of participants, 64 EEG electrode locations were recorded using a high-density cap following the 10–10 system of electrode placement. Nine initial participants were recorded with only nine electrode sites (F3, Fz, F4, C3, Cz, C4, O1, Oz, and O2). Because only a single electrode is required for the clustering and classification analyses (Wamsley & Summer, 2020), we initially decided that that this study did not require high-density EEG. However, we subsequently realized that this was a poor decision, because a larger number of electrodes is important for a more robust artifact-rejection pipeline, and thus we increased the number of electrodes. These nine participants were excluded from the EEG source localization analyses described below, which also require a larger number of electrodes. EEG data were acquired at 500 Hz using Brain Products' Brain Amp amplifiers. Impedance was kept under 10 kΩ. Participants were equipped with a PupilLabs head-mounted infrared eyetracking system for the purpose of acquiring pupillometry data (Kassner, Patera, & Bulling, 2014). Participants then completed measures of state sleepiness, including the Stanford Sleepiness Scale (Johns, 1991) and visual analog scales assessing perceived ability to concentrate and how refreshed participants felt.
Participants then trained on the short story recall task described below, just before beginning a ≈ 30-min SART (see Figure 1 and below). During the SART, EEG and pupillometry data were continuously recorded. Immediately following the completion of the SART, participants were again tested on their memory for the short story recall task.
Experimental paradigm. During the SART, participants responded to successive numeric stimuli with a button press, but were instructed to withhold response to the target digit (“3”). Right: Experience was intermittently sampled using a forced-choice thought probe prompting participants to categorize their immediately preceding experience as either related or unrelated to the experimental stimuli, and either externally or internally directed. Participants could also respond that they did not recall having any experience.
Experimental paradigm. During the SART, participants responded to successive numeric stimuli with a button press, but were instructed to withhold response to the target digit (“3”). Right: Experience was intermittently sampled using a forced-choice thought probe prompting participants to categorize their immediately preceding experience as either related or unrelated to the experimental stimuli, and either externally or internally directed. Participants could also respond that they did not recall having any experience.
Tasks
SART
The SART is a simple attention task designed to facilitate mind wandering while also measuring fluctuations in RT (Stawarczyk, Majerus, Maquet, & D'Argembeau, 2011; Christoff, Gordon, Smallwood, Smith, & Schooler, 2009). Participants were serially presented with the digits 1–9 on the computer monitor and were instructed to press the spacebar as quickly as possible as each digit appeared, but to refrain from responding to the digit “3” (the “target”). Each digit was on-screen for 450 msec, with a 5-sec SOA. This relatively long SOA was necessary to be certain that evoked pupil and EEG responses returned to baseline in advance of the next stimulus, allowing epochs of tonic prestimulus activity to be selected for analysis, uncontaminated by evoked responses.
Stimulus sequences were randomly generated in blocks of 9, 12, 15, or 18 stimuli, such that each block contained at least one but no more than three targets, and targets were always separated by at least one nontarget. Across all blocks, 16.4% of stimuli were targets. The last trial in every block was a “probe trial,” in which the digit stimulus (always a nontarget) was followed by a forced-choice question asking participants to reflect on the content of their current subjective experience. As illustrated in Figure 1, participants classified their experience into one of five categories: (a) external focus on sensory aspects of the experimental stimuli (external task-related); (b) external focus on other sensory stimuli in the environment (external task-unrelated); (c) internal thoughts, feelings, or imagery about the experimental stimuli (internal task-related); (d) internal thoughts, feelings, or imagery unrelated to the current sensory environment (internal task-unrelated, e.g., daydreaming); or (e) unable to recall any experience (mind blank). Before completing the task, participants had been trained on how to classify their experience into these categories, practicing using a set of 10 examples and receiving feedback from the research assistant. There were 324 trials in total, including 24 probe trials. Participants were allowed a short break at the halfway point of the task, during which they could stretch and reposition themselves.
Short Story Task
The short story recall task was adapted from the Wechsler Memory Scale (Wechsler, 1987), following Dewar and colleagues (2012). Participants listened to a digital recording of a short story, approximately 30 sec long, and then freely recalled as much of this story as they could, as accurately as possible, by typing everything that they remembered into an electronic form. They were given as much time as needed to complete their responses. Following the 30-min SART, a delayed recall test was administered in which participants again were given as much time as needed to type everything they could remember about the short story. Participants were explicitly informed that the delayed test would take place. Free recall responses were scored by two raters blind to experimental condition, according to the methods described in the Wechsler Memory Scale Manual. All reports were scored by both raters, and the final score for each report was calculated as the average score of the two raters. Interrater reliability was r = .98.
Exit Questionnaire
Finally, participants completed an exit questionnaire about their subjective experiences during the SART. Participants were first asked to indicate, using a 5-point scale, the extent to which they had “thought about,” “imagined,” or “tried to remember” the verbal learning task while completing the SART. They were then asked to indicate the proportion of the SART interval they spent in one or more of 13 predefined mental categories: thinking about the short story from earlier, thinking about the past (something earlier today/yesterday to a week ago/past year or several years ago), imagining the future (remainder of the day/tomorrow to next week/next year or several years), thinking about the numbers task, mind was blank, counting the time, doing focused meditation, sleeping, and other. For purposes of analysis, these categories were collapsed into the superordinate groupings of (1) thinking about the past, (2) thinking about the future, (3) thinking about the SART, and (4) other. Lastly, participants provided an open-ended response to the question “Please describe your thoughts, feelings, or daydreams while performing the numbers task in as much detail as you can remember.”
Data Preprocessing
EEG Recordings
EEG analyses were performed with the Brainstorm (Tadel, Baillet, Mosher, Pantazis, & Leahy, 2011) and EEGLab (Delorme & Makeig, 2004) toolboxes for MATLAB. Before analysis, recordings were processed with a semiautomated artifact-rejection pipeline. In brief, recordings were first band pass filtered at 0.3–35 Hz and subjected to an initial artifact rejection pass using the clean_rawdata() plugin for EEGLab. This was followed by the identification and removal of artifactual independent components and then a second pass of clean_rawdata() in which any remaining periods of artifact were marked. Data were then average referenced and passed to Brainstorm, where automated artifact markers were reviewed and manually adjusted via visual inspection as needed.
For all artifact-free trials, EEG power spectral density (PSD) was calculated for the 4-sec window of prestimulus EEG ending 200 msec before stimulus presentation, in five a priori frequency bands known to covary with vigilance and mind wandering (Braboszcz & Delorme, 2011): slow oscillation (0.3–1 Hz), delta (1–4 Hz), theta (4–7 Hz), alpha (8–12 Hz), and beta (13–35 Hz). For each trial, these values were then converted to relative power, defined as the proportion of total power across all frequencies that was accounted for by power within the particular frequency band. Finally, to ensure that PSD could be meaningfully combined across participants with differing overall EEG amplitudes, PSD values were separately z scored for each participant. The resulting values therefore reflect the extent to which trials contain proportionally more or less power in a particular frequency band, relative to other trials for that participant. Trials including excessive EEG artifact were excluded from all subsequent analysis steps (mean = 83.3, SD = 91.3 trials per participant were excluded for this reason). To avoid excessive redundancy in the features provided to the cluster analysis, only a single electrode (Fz) was passed to the clustering and classification procedures.
Pupil Diameter
Datapoints during which the pupil failed to be detected because of blinks or other artifact were deleted, as were extreme values (defined as points > 3 median absolute deviations from the median for that recording). Linear interpolation was then used to replace these missing datapoints. Data were then low-pass filtered at 10 Hz and z scored (separately for each participant). Finally, for each SART trial, mean pupil diameter was calculated for the 1-sec window ending 200 msec before SART stimulus presentation.
Some participants had large amounts of missing pupil data, because of failure of the tracker to maintain accurate pupil detection for the duration of the recording. Pupil data for individual trials was included in further analysis only when ≤ 20% of the datapoints for that trial had been interpolated. Four participants were excluded from further analysis because < 50% of their trials were usable according to this latter criterion.
Summary of Exclusions
Nineteen of 65 participants who enrolled in the study were excluded from analysis. This comprised six exclusions to technical problems during data collection, two for excessive EEG artifact, four for excessive pupil artifact (as defined above), four because of excessive sleepiness (Stanford Sleepiness Scale at training > 5 or mean total sleep time averaging < 5 hr for the three nights before the study), one because they had heard the short story before, and two because, following other exclusions, they had < 10 useable SART trials remaining. In addition, an error prevented accurate recording of RTs on the experience probe trials for the first n = 20 participants. Experience probe trials are therefore excluded from analyses for these participants. Following all exclusions, the final sample consisted of 8979 trials contributed by 46 participants (mean = 195, SD = 71 analyzed trials per participant). As a result of the data-cleaning criteria described above, these 46 participants had pupil data missing from an average of 61.4 ± 48.6 SD trials, EEG data missing from an average of 5.6 ± 2.9 SD trials, and RT data missing from an average of 2.9 ± 6.1 SD trials. All of these exclusion decisions were made before beginning the hypothesis-testing phase of the data analysis.
Clustering and Classification Procedures
Identification of Waking States via Cluster Analysis
Clustering and classification analyses were carried out using Weka 3.6 (Hall et al., 2009). Only nontarget trials with correct responses were considered. To define waking states in a data-driven manner, an expectation maximization (EM) cluster analysis was applied to all probe trials with valid RT data (mean = 20.8, SD = 3.2 trials per participant). Input features included EEG spectral power at Fz (preprocessed and z scored as described above), RT to SART stimuli (RTs > 1000 msec excluded as extreme values), pupil diameter (preprocessed and z scored as described above), and participants' forced-choice responses to the experience sampling probe. For all measures, datapoints more than 4 SDs above or below the mean were removed just before EM clustering.
Three different EM clustering models were evaluated, describing the data using two, three, or four clusters. To evaluate clustering, we examined three distance-based metrics (including the Davies-Bouldin, silhouette, and Calinski-Harabasz indices). These indicated that optimal cluster separation was obtained with three clusters. We also noticed that in the four-state model, the fourth state relied heavily on separating trials with missing pupil data into their own cluster. This did not make theoretical sense to us, because we presume these data to be missing at random (because of recording artifact causing > 20% interpolation on select trials). Therefore, we decided to proceed with subsequent analyses considering only this three-state model. For the remainder of the article, we refer to the clusters in this model as the “online,” “offline1,” and “offline2” states.
Classification of Nonprobe Trials into Online and Offline States
Because mind wandering is reduced when experience is sampled too frequently (Seli, Carriere, Levene, & Smilek, 2013), probes of conscious experience were administered at only 24 time points per participant, and the above cluster analysis included only these trials. Yet the full 30-min retention interval contains hundreds of 5-sec trials with EEG, pupillometry, and RT data. Because of our interest in examining seconds-level fluctuations in waking state, a primary analytic goal was to be able to classify all trials into EM-defined online and offline states, even when experience sampling data were not present on that trial. As a next step, we thus trained a naive Bayes classifier to determine the EM cluster assignment (online vs. offline) of each probe trial based on the EEG, pupil, and RT data alone (selection of optimal classification method was empirically determined in Wamsley and Summer (2020)). A 10-fold cross validation was used to estimate classifier accuracy, with trials being evenly divided into 10 subsets (“folds”), 9 used as training data and 1 as testing data on each of 10 sequential runs. A mean of 97.1% accuracy was achieved in determining cluster assignment using EEG, pupil, and RT data alone.
Applying the Classifier to Assign State Labels to All Trials
Having developed a classifier able to accurately determine waking state cluster based on EEG, pupil, and RT data, we then moved forward in applying this classifier to label all trials as belonging to the “online,” “offline1,” or “offline2” state, allowing us to define these states across the full length of the recording with a 5-sec temporal resolution (Figure 2).
Example raw data. A sequence of five SART trials from a single participant. A naive Bayes classifier was trained to label each 5-sec trial as belonging to the “online,” “offline1,” or “offline2” state, on the basis of the EEG (top trace, Fz), pupil diameter (bottom trace), and SART RTs. Dotted lines indicate stimulus onset on each of five trials. Pupil diameter is expressed as a z score. RT = reaction time to the SART stimulus on that trial.
Example raw data. A sequence of five SART trials from a single participant. A naive Bayes classifier was trained to label each 5-sec trial as belonging to the “online,” “offline1,” or “offline2” state, on the basis of the EEG (top trace, Fz), pupil diameter (bottom trace), and SART RTs. Dotted lines indicate stimulus onset on each of five trials. Pupil diameter is expressed as a z score. RT = reaction time to the SART stimulus on that trial.
To quantify the extent to which individual participants entered each waking state, for each subject, we calculated state probability as the mean probability of being in the online/offline1/offline2 state during the SART (posterior probability that a trial belongs to the state, as determined by the naive Bayes classifier and averaged across all trials for that participant). This metric was used to assess the association between offline time and memory retention for the short story recall task.
Calculation of Run Length and State Transition Probabilities
For each state, “bout length” was the mean number of trials that participants remained in a state before switching to a different state. As described above, participants often had missing trials because of low pupil detection quality and/or EEG artifact. To avoid the influence of missing data discontinuities on this calculation, individual runs were assessed only within continuous trial sequences with no missing data, and not across points of missing-trial discontinuity. Permutation tests assessed whether between-states differences in run length were greater than expected to occur by chance in a randomly ordered sequence of states. To accomplish this, run lengths for each state were calculated in 10,000 random permutations of the sequence of state labeling across trials. Missing datapoints occurred in groups in the observed data (because of blinks, movements, etc.), but at random in the permuted data. Because of this, mean run lengths were longer in the observed, relative to the permuted data. To account for this, run lengths were z scored before permutation testing.
We additionally examined the matrix of transition probabilities between states across trials. For each possible type of state transition (online → online, online → offline1, online → offline2, etc.), we calculated the degree to which the transition occurred more frequently than expected by chance. Above-chance transition probability was defined as the difference between the observed transition probability (number of times the transition occurred / total number of transitions) and the mean probability of that same transition within n = 10,000 randomly generated permutations of the state labelling sequence.
EEG Topography of Waking States
Following state classification, we described the EEG during online, offline1, and offline2 trials in more detail, examining between-states differences in spectral power across all electrodes and frequencies. These between-states EEG comparisons were conducted both in sensor space (Figure 4) and in source space (Figure 5). Source localization utilized the ICBM152 template cortical surface reconstruction, as distributed with Brainstorm (Tadel et al., 2011). An unconstrained-orientation minimum norm estimate was calculated using a realistic three-layer Boundary Element Method head model. For these analyses, PSD calculations were conducted in source space, following the same methods as described above. Comparisons between states were then conducted using paired-samples t tests, with the unit of analysis being an individual participant. The topographical maps in Figure 5 display t values only for the cortical regions in which comparisons remained statistically significant following the Benjamini–Hochberg correction for multiple comparisons (controlled for multiple comparisons across both voxel and frequency).
Statistical Analyses
Statistical analyses were conducted in R (R Core Team, 2019), with those for EEG data conducted in Brainstorm (Tadel et al., 2011). We utilized random-intercept mixed effect models to describe the features of online and offline trials, with trials grouped by participant. Where outcome variables were categorical, mixed effect logistic regression models were implemented using the generalized linear model function glmer in the lme4 package (Bates, Mächler, Bolker, & Walker, 2015). Here, statistical significance was assessed using Wald chi-square tests. Where outcome variables were continuous, linear mixed effect models were conducted using the lme4 and lmerTest packages for R (Kuznetsova, Brockhoff, & Christensen, 2017). ANOVA and pairwise test statistics derived from these models used Satterthwaite's method of estimating degrees of freedom. Pairwise comparisons between states were then conducted on the estimated marginal means using the emmeans package in R (Lenth, 2020).
As described above, analyses of the observed temporal features of the states utilized permutation tests to assess whether these differed from the characteristics of 10,000 randomly generated permutations of the state labelling sequence.
Pearson's correlations were used to test for associations between subject-level variables, including for associations between waking state probability and memory retention. Wherever large numbers of exploratory tests were conducted, false discovery rate was controlled using the Benjamini–Hochberg method (Benjamini & Hochberg, 1995). Comparison of subject-mean EEG spectral power between states at individual electrode sites was conducted using paired-samples t tests, controlling false discovery across n = 62 electrodes using the Benjamini–Hochberg method.
RESULTS
Subjective Experience during the SART
During the SART, thought probe responses most frequently indicated internal task-unrelated thought (33.3% of probes), followed by internal task-related (26.2%), external task-related (19.6%), external task-unrelated (14.6%), and mind blank (6.3%) responses. On the retrospective exit questionnaire, participants indicated they spent the greatest proportion of the time thinking about the SART (32.2% ± 27.4% of time), followed by thinking about the future (13.7% ± 11.4%), thinking about the past (10.9% ± 8.4%), “other” categories of thought (10.6% ± 11.2%), and thinking about the short story (3.3% ± 4.2%).
Classification of Trials into States of Wakefulness
Cluster and Classification Analyses
Clustering and classification analyses considered only nontarget trials yielding a correct response. States of wakefulness were initially defined by applying EM cluster analysis to thought probe trial EEG, RT, pupil, and subjective experience data (n = 541 trials; see Methods section). This revealed three distinct clusters of thought probe trials, which we will refer to as the “online” state (50.8% of trials), “offline1” state (32.7% of trials), and “offline2” state (16.5% of trials). In the Discussion section, we elaborate on the reasons why the features of these states motivated us to give them these names. A moderately positive silhouette index of .22 indicated that trials tended to be relatively closer in multidimensional state space to points in their own cluster, and farther away from points in the other clusters. To categorize the remaining trials (those not including thought probes) as online or offline, we developed a naive Bayes classifier. This classifier was 97.1% accurate in correctly labeling probe trials by their EM-defined cluster, using EEG, RT, and pupil diameter alone (assessed with stratified 10-fold cross-validation; see Methods section). Developed and tested using probe trial data, the classifier was then applied to all trials, including n = 8438 trials without thought probes.
Online and Offline State Characteristics
On average, participants were classified as spending 60.0% ± 4.5% of the 30-min SART interval in the online state, 27.8% ± 5.1% in the offline1 state, and 16.3% ± 3.6% in the offline2 state (mean percentage across participants). Every participant had at least some trials classified into each of the three states. Raw data across a series of five trials are illustrated in Figure 2.
Pupil diameter and RT by state.
Pupil diameter varied by state, F(2, 6122) = 25.11, p < .0001, being larger during offline2 trials, relative to offline1 (p < .0001) or online trials (p < .0001), as well as marginally larger during offline1 than online trials (p = .051; Figure 3A). RTs also varied by state, F(2, 8800) = 7.41, p = .0006, being slower during the online state, relative to both offline states (offline1 vs. online: p = .003; offlne2 vs. online: p = .002; offline1 vs. offline 2: p = .06; Figure 3B).
Classification of waking states. Applying the naive Bayes classifier to label all n = 8979 trials yielded three states, which we term the “online,” “offline1,” and “offline2” states. (A) Pupil diameter by state. (B) SART RTs by state. (C) Within-state proportion of experience probes in each response category. (D) EEG PSD by state in each of five frequency bands. (E) Scatter plot showing the separation between states in alpha (8–12 Hz) and slow oscillation (0.3–1 Hz) PSD, two of the features that most strongly distinguish between with states. Each point represents a single trial. All error bars = ±95% CI.
Classification of waking states. Applying the naive Bayes classifier to label all n = 8979 trials yielded three states, which we term the “online,” “offline1,” and “offline2” states. (A) Pupil diameter by state. (B) SART RTs by state. (C) Within-state proportion of experience probes in each response category. (D) EEG PSD by state in each of five frequency bands. (E) Scatter plot showing the separation between states in alpha (8–12 Hz) and slow oscillation (0.3–1 Hz) PSD, two of the features that most strongly distinguish between with states. Each point represents a single trial. All error bars = ±95% CI.
Subjective experience by state.
Participants reported less attention to the ongoing task during offline2 trials (external task-related thought probe responses), relative to both online (p = .02) and offline1 (p = .03) trials. Internal task-unrelated thought was numerically but nonsignificantly greater during offline, as compared with online trials (offline1 vs. online: p = .15; offline2 vs. online: p = .22). No other thought probe response categories differed significantly between states (Figure 3C).
EEG PSD by state.
As shown in Figure 3D–E, slow oscillation (0.3–1 Hz; F(2, 8968) = 4729.9, p < .0001), delta (1–4 Hz; F(2, 8972) = 1540.7, p < .0001), theta (4–7 Hz; F(2, 8953) = 791.79, p < .0001), alpha (8–12 Hz; F(2, 8944) = 7285.1, p < .0001), and beta (13–35 Hz; F(2, 8960) = 474.55, p < .0001) power all varied significantly by state.
Offline2 was characterized by high levels of slow oscillation power, in comparison to both online (p < .0001) and offline1 (p < .0001) trials. Slow oscillation power was also lower during offline1 than online trials (p < .0001). In contrast, offline1 was most prominently characterized by high levels of alpha, compared with both offline2 (p < .0001) and online (p < .0001). Alpha was significantly lower during offline2 than online trials (p < .0001). Meanwhile, online trials had the highest levels of both delta and theta activity. Online trials had significantly greater delta power than both offline1 (p < .0001) and offline2 (p < .0001) trials, with offline1 and offline2 being statistically equivalent to each other (p = .28). Online trials also had the highest amount of theta power, with theta being significantly greater than during either offline1 (p < .0001) or offline2 (p < .0001) trials. For theta, power was also greater during offline1 than during offline2 (p < .0001). Finally, beta power was significantly greater during online trials, relative to both offline1 (p = .005) and offline2 (p < .0001) trials. Beta was also lower during offline2 than offline1 trials (p < .0001).
Spatial distribution of between-states differences in EEG spectral power.
Although initial classification relied on EEG activity from a single electrode (Fz), we subsequently described the spatiotemporal features of the EEG in more detail, examining spectral power across all electrode sites and frequencies. Figure 4 displays the mean power spectrum for each state, averaged across electrodes. The spatial distribution of between-states differences in EEG spectral power was then examined in source space (Figure 5; see Methods section).
Mean EEG power spectrum differences between states in sensor space. Log PSD in online, offline1, and offline2 states, averaged across all electrodes. The large increase in alpha power in offline1 as well as the increase in slow oscillation power in offline2 are both visually apparent.
Mean EEG power spectrum differences between states in sensor space. Log PSD in online, offline1, and offline2 states, averaged across all electrodes. The large increase in alpha power in offline1 as well as the increase in slow oscillation power in offline2 are both visually apparent.
Topography of EEG power spectrum differences between states in source space. Pairwise comparison of participant mean EEG spectral power, computed in source space (see Methods section). Colormaps represent t values from paired-samples t tests, masked to exclude regions where comparisons were nonsignificant following Benjamin–Hochberg FDR correction.
Topography of EEG power spectrum differences between states in source space. Pairwise comparison of participant mean EEG spectral power, computed in source space (see Methods section). Colormaps represent t values from paired-samples t tests, masked to exclude regions where comparisons were nonsignificant following Benjamin–Hochberg FDR correction.
Relative to online trials, offline1 was most clearly associated with a global elevation in alpha power. Offline1 also showed reduced slow, delta, and theta power in frontal, temporal, and anterior parietal cortices. Meanwhile, theta and beta power were increased in posterior regions (Figure 5).
In comparison to online trials, offline2 was most prominently characterized by a spatially extensive increase in slow oscillation power, strongest in superior frontal regions. Significant increases in alpha power were spatially restricted to a region surrounding the L angular gyrus, as well as regions of R superior temporal and R inferior parietal lobe, including the supramarginal gyrus. Frontal theta power was significantly reduced relative to the online state. There were also significant reductions in delta and beta power in small regions of the anterior frontal lobe (Figure 5).
Temporal features of the transition between states.
As predicted, the probability of being in the online state decreased with increasing time on task (r269 = −.22, p = .0002; Figure 6A), whereas the offline1 state became increasingly prevalent over time (r269 = .25, p = .00003; Figure 6B). In contrast, offline2 probability did not change across the duration of the SART (p = .51).
Temporal features of online and offline states. (A) The probability of being online decreased significantly across trials. Each point represents an individual trial. Shading = ±95% CI. (B) Probability of the offline1 state increased significantly across trials. Each point represents an individual trial. Shading = ±95% CI. (C) Mean run length for each state was defined as the mean number of trials that participants remain in a particular state before transitioning to a new state (z scored). Run length differences between states did not deviate significantly from what permutation tests established would be expected in a random sequence of states with the same overall occupancy rates. (D) Matrix of state transition probabilities. The y axis represents the state transitioned from, and the x axis represents the state transitioned to. For example, the probability of transitioning from offline2 → online is listed in the lower left square. For each of the three states, permutation tests established that the probability of remaining in that same state, rather than transitioning to a new state, was significantly greater than would be expected in a random sequence.
Temporal features of online and offline states. (A) The probability of being online decreased significantly across trials. Each point represents an individual trial. Shading = ±95% CI. (B) Probability of the offline1 state increased significantly across trials. Each point represents an individual trial. Shading = ±95% CI. (C) Mean run length for each state was defined as the mean number of trials that participants remain in a particular state before transitioning to a new state (z scored). Run length differences between states did not deviate significantly from what permutation tests established would be expected in a random sequence of states with the same overall occupancy rates. (D) Matrix of state transition probabilities. The y axis represents the state transitioned from, and the x axis represents the state transitioned to. For example, the probability of transitioning from offline2 → online is listed in the lower left square. For each of the three states, permutation tests established that the probability of remaining in that same state, rather than transitioning to a new state, was significantly greater than would be expected in a random sequence.
We next used two methods to assess whether the pattern of transitions between states showed evidence of being nonrandom. First, we assessed the mean length of time that participants tended to remain in each state, before transitioning to a new state (run length). Mean run length was greater for the online state, relative to either offline1 or offline2 (trials spent online before switching states 1.66 ± 1.08; offline1 1.28 ± .62, offline2 1.14 ± .40; Figure 6C). In part, this is the consequence of the overall larger number of online trials, because even a randomly ordered sequence of states would show longer run lengths for more frequently occurring states. To test whether the observed run length differences between states were greater than expected by chance, we compared the observed mean differences to those derived from n = 10,000 randomly generated permutations of the state labelling sequence. The observed difference between offline1 and offlne2 run length was marginally larger than expected to occur by chance (p = .076). No other state differences in run length were statistically significant (online vs. offline1: p = .63; online vs. offline2: p = .18).
Second, we examined the matrix of transition probabilities between states (Figure 6D). For each type of transition, we calculated the degree to which the observed transition probability was greater than chance (see Methods section). For each of the three states, participants tended to remain in the same state on the next trial, rather than transition to a new state, at greater than chance levels (Figure 6D).
The Offline2 State Predicts Memory Retention
Waking State Probability as a Predictor of Memory Retention
As shown in Figure 7A, there was a significant positive association between probability of being in the offline2 state and subsequent improvement on the story recall task (r44 = .29, p = .047). This association was not significant for either the offline1 (r44 = −.11, p = .49) or online states (r44 = −.12, p = .42), and was near-significantly larger for offline2 than for either the offline1 or online states (Fisher's z tests: z = 1.87, p = .06 vs. offline1 and z = 1.95, p = .05 for online).
The offline2 state predicts and memory retention. Association between offline2 probability and memory retention. Shading = ±95% CI.
The offline2 state predicts and memory retention. Association between offline2 probability and memory retention. Shading = ±95% CI.
Importantly, offline2 was unrelated to recall of the short story at baseline, demonstrating that this state is selectively predictive of the extent to which memory changes after the SART retention interval (r44 = −.1, p = .49). Similarly, neither of the other two states were associated with baseline recall (offline1: r44 = .13, p = .37; online: r44 = −.08, p = .62).
Individual Component Features of the Waking States Do Not Predict Memory Retention
The definition of waking states was very strongly associated with particular measures (e.g., alpha and slow oscillation EEG power). Because of this, one might ask whether these individual features would alone be sufficient to predict memory, without needing to invoke the multimodal “states” proposed here. To assess this, we examined the association between memory change across the SART and each of the component features used to define online and offline states (Table 2). The only feature predicting story memory was the proportion of external task-unrelated thought probe responses (r24 = −.49, p = .01 uncorrected). This association did not survive Benjamini–Hochberg correction for multiple comparisons.
Retention Interval Component Features as Predictors of Memory Performance
Measure . | Correlation with% Change in Story Recall . | ||
---|---|---|---|
r . | 95% CI . | p . | |
Thought probe responses | |||
External task-related thought | .15 | [−0.25, 0.51] | .47 |
External task-unrelated thought | −.49 | [−0.74, −0.12] | .01* |
Internal task-related thought | .14 | [−0.26, 0.50] | .49 |
Internal task-unrelated thought | .03 | [−0.36, 0.41] | .89 |
Mind blank | .16 | [−0.24, 0.52] | .43 |
EEG frequency bands | |||
Beta (13–35 Hz) | .00 | [−0.29, 0.29] | .99 |
Alpha (8–12 Hz) | .10 | [−0.20, 0.38] | .50 |
Theta (4–7 Hz) | −.14 | [−0.41, 0.16] | .37 |
Delta (1–4 Hz) | −.14 | [−0.41, 0.16] | .37 |
Slow oscillation (0.3–1 Hz) | −.07 | [−0.36, 0.22] | .63 |
SART RT | −.27 | [−0.52, 0.02] | .07 |
Pupil diameter | .15 | [−0.15, 0.42] | .32 |
Measure . | Correlation with% Change in Story Recall . | ||
---|---|---|---|
r . | 95% CI . | p . | |
Thought probe responses | |||
External task-related thought | .15 | [−0.25, 0.51] | .47 |
External task-unrelated thought | −.49 | [−0.74, −0.12] | .01* |
Internal task-related thought | .14 | [−0.26, 0.50] | .49 |
Internal task-unrelated thought | .03 | [−0.36, 0.41] | .89 |
Mind blank | .16 | [−0.24, 0.52] | .43 |
EEG frequency bands | |||
Beta (13–35 Hz) | .00 | [−0.29, 0.29] | .99 |
Alpha (8–12 Hz) | .10 | [−0.20, 0.38] | .50 |
Theta (4–7 Hz) | −.14 | [−0.41, 0.16] | .37 |
Delta (1–4 Hz) | −.14 | [−0.41, 0.16] | .37 |
Slow oscillation (0.3–1 Hz) | −.07 | [−0.36, 0.22] | .63 |
SART RT | −.27 | [−0.52, 0.02] | .07 |
Pupil diameter | .15 | [−0.15, 0.42] | .32 |
Pearson's correlations, along with the 95% confidence interval on r.
Statistically significant at p < .05.
Bivariate Associations between Component Features
Bivariate associations between EEG, pupil, RT, and subjective experience during the SART were assessed using Pearson's correlations on a per-subject basis. An inverse correlation between task-unrelated internal thought (i.e., daydreaming) and task-related external thought (i.e., thinking about the SART) remained statistically significant following Benjamini–Hochberg correction for multiple comparisons (r24 = −.65, p = .005; Figure 8). Intercorrelations between power in select EEG frequencies also remained significant, but no other associations survived correction for multiple comparisons (Figure 8).
Subjectwise bivariate associations. Matrix of Pearson's coefficients (r), between subject means of the listed variables. Aside from associations between EEG frequency bands, following Benjamin–Hochberg correction for multiple comparisons, only the negative correlation between task-unrelated internally oriented thought (i.e., daydreaming) and task-related externally oriented thought (i.e., thinking about the SART) remained significant. Related Int = task-related internal thought; Related Ext = task-related external thought; Unrelated Int = task-unrelated internal thought; Unrelated Ext = task-unrelated external thought.
Subjectwise bivariate associations. Matrix of Pearson's coefficients (r), between subject means of the listed variables. Aside from associations between EEG frequency bands, following Benjamin–Hochberg correction for multiple comparisons, only the negative correlation between task-unrelated internally oriented thought (i.e., daydreaming) and task-related externally oriented thought (i.e., thinking about the SART) remained significant. Related Int = task-related internal thought; Related Ext = task-related external thought; Unrelated Int = task-unrelated internal thought; Unrelated Ext = task-unrelated external thought.
Waking States Not Associated with Other Measures of Subjective Experience or SART Accuracy
Waking state probabilities were not associated with the daydream frequency scale or the Mindful Attention Awareness Scale (all uncorrected p values ≥ .18), and were not associated with the proportion of time spent in any exit questionnaire thought category (all uncorrected p values ≥ .09; Table 3). Waking state probabilities were also unrelated to SART accuracy for both target trials (all uncorrected p values ≥ .36) and nontarget trials (all uncorrected p values ≥ .87).
Observed Characteristics of Online and Offline Waking States
Dimension . | Waking State . | ||
---|---|---|---|
Online . | Offline 1 . | Offline 2 . | |
Subjective Attentional focus | ↑ Task focus | ↑ Task focus | ↓ Task focus |
Behavioral Performance | Slower RT, may indicate more deliberate attention | Faster RT, may indicate inattention | Faster RT, may indicate inattention |
Pupil diameter (proxy for NE) | ↓ Pupil diameter (↓ NE) | ↔ Pupil diameter (↔ NE) | ↑ Pupil diameter (↑ NE) |
EEG power | ↑ Delta/theta | ↑↑ Alpha | ↑↑ Slow Osc. |
↓ Alpha and slow Osc. | |||
Memory | No association with memory | No association with memory | ↑ Memory retention |
Dimension . | Waking State . | ||
---|---|---|---|
Online . | Offline 1 . | Offline 2 . | |
Subjective Attentional focus | ↑ Task focus | ↑ Task focus | ↓ Task focus |
Behavioral Performance | Slower RT, may indicate more deliberate attention | Faster RT, may indicate inattention | Faster RT, may indicate inattention |
Pupil diameter (proxy for NE) | ↓ Pupil diameter (↓ NE) | ↔ Pupil diameter (↔ NE) | ↑ Pupil diameter (↑ NE) |
EEG power | ↑ Delta/theta | ↑↑ Alpha | ↑↑ Slow Osc. |
↓ Alpha and slow Osc. | |||
Memory | No association with memory | No association with memory | ↑ Memory retention |
Simplified summary of the observed waking states, emphasizing key features that were relatively greater than (↑), less than (↓), or similar to (↔) those observed in other states.
Omitting Thought Probe Data Yields Different Clusters
As outlined above, our a priori approach was to define online/offline states based on a two-step process in which clusters are first defined based only on thought probe trials, with those labels then leveraged to create a classifier that labels all trials by state, without use of subjective experience information. In an exploratory analysis, we ran an additional EM cluster analysis on the full set of trials, to test whether the same results could be achieved with a simpler one-step process of merely clustering based on pupil, RT, and EEG data, ignoring the thought probe responses. In total, only 81.7% of the state labels remained the same in this new clustering (Cohen's kappa = .71). The EEG profile of the resulting states differed in that the reclustered online state had higher beta power and the reclustered offline1 state had higher delta power, relative to our two-step method. Furthermore, with this new clustering, neither pupil diameter nor RT differed between states. This suggests that including the subjective experience data in defining the clusters via our two-step processes adds substantial information to our model of waking states.
DISCUSSION
After learning a short story, the amount of time spent in one of two data-defined offline states predicted memory retention. This state (offline2) has features that render it ideal for supporting the reactivation and consolidation of memory, including transient increases in EEG slow oscillation power and pupil diameter. Even in the absence of overt rest, intermittently entering this state throughout our daily activities could promote the early stages of memory consolidation, enabling us to quickly transition between a neurobiological milieu optimized for memory encoding and one optimized for consolidation.
Defining the States of Wakefulness
Several other lines of research have proposed a similar cyclic alternation between online and offline waking states (Poulet & Crochet, 2019; Vidaurre et al., 2018; Wang et al., 2018; Smallwood & Schooler, 2015; Reimer et al., 2014). Yet despite their conceptual overlap, these literatures have not yet arrived at a consensus method of describing the states of wakefulness. As few as two or as many as 12 discrete states have been proposed, described on timescales stretching from milliseconds to minutes (Wamsley & Summer, 2020; Zanesco, Denkova, Witkin, & Jha, 2020; Poulet & Crochet, 2019; Vidaurre et al., 2018; Wang et al., 2018; Reimer et al., 2014).
Our approach offers some advantages that may be useful in moving toward a universal definition of online and offline states. First, we use low-cost, noninvasive measures that are easily applied in humans. Second, the temporal resolution of our technique is not limited by the frequency with which participants can report on their experience. Third, this work joins only a handful of other investigations in defining waking states in an entirely data-driven manner (Vidaurre et al., 2018; Wang et al., 2018), rather than based on theoretical preconceptions of what the states should look like.
Yet the three data-driven states described here are coherent with the previous literature, in line with current theory on brain states and memory consolidation, and partially confirm our a priori expectations. We labeled two states as “offline” because they were characterized by reduced attention to the environment and by increased EEG power in the alpha and slow oscillation ranges, as observed in our earlier work (Wamsley & Summer, 2020). Still, we note that although the states themselves are data-driven, the labels we give them are a theoretically motivated interpretation of the meaning of these states.
The Online State
Relative to offline trials, the online state was characterized by slowed RTs, a constricted pupil, and decreased EEG power in the slow oscillation and alpha frequency bands. At the level of subjective experience, this state was characterized by increased attention to the ongoing task (the SART, in comparison to the offline2 state) and near-significantly reduced levels of task-unrelated internal thought (e.g., daydreaming).
Slowed RTs and pupil constriction during online attention to the environment are both consistent with prior literature on the correlates of task-focus during the SART. Increased RTs on nontarget trials likely reflect deliberative monitoring for targets, an attention-demanding process that prevents errors of commission (Polychroni, Hedman, & Terhune, 2020). As in the current study, prior literature examining pupil diameter during the SART has reported a decreased baseline pupil size during on-task trials (Smallwood et al., 2011).
We strongly expected offline trials to be characterized by increased alpha and slow oscillation power, and online trials to be characterized by increased delta and theta power (Wamsley & Summer, 2020), and this was indeed confirmed in the current study. Heightened delta and theta during online trials could reflect task-related processing, as oscillations in these frequency bands increase during performance of go/no-go tasks that, like the SART, require monitoring for a cue signaling whether a participant should respond on each trial (Harmony, 2013; Harmony, Alba, Marroquín, & González-Frankenberger, 2009).
Finally, increased subjective attention to the SART while online is also consistent with the notion that this state represents a time of heighted attention to external stimuli. However, interestingly, external task-related thought was equally frequent during offline1 trials. Metacognitive awareness of mind wandering is imperfect (Schooler et al., 2011), and given that both offline states showed speeded SART RTs, this may indicate that participants were aware of reduced attention to the task during offline2 trials, but were unaware that their attention had lapsed on offline1 trials.
The Offline2 State
Participants who spent more time in offline2 after encoding showed better memory retention following the SART. Strengthening our confidence that this is a true association, we note that this observation was in line with our a priori hypotheses, and confirms a nonsignificant trend reported in a prior study from our laboratory (Wamsley & Summer, 2020).
The features of offline2 are well-suited to promote memory consolidation. First, offline2 was associated with a large increase in slow oscillation EEG power. In numerous human and animal studies, slow oscillations have been associated with cortical plasticity and memory consolidation during sleep (Timofeev & Chauvette, 2017; Wei, Krishnan, & Bazhenov, 2016; Marshall, Helgadóttir, Mölle, & Born, 2006), and rest (Brokaw et al., 2016). Second, while both offline states showed increased pupil diameter relative to online trials, this effect was most pronounced for offline2. Pupil diameter tracks transient increases in NE neuromodulation (Joshi, Li, Kalwani, & Gold, 2016; Murphy et al., 2014), and it has long been known that posttraining increases in amygdala NE promote the consolidation of memory (McGaugh & Roozendaal, 2009). β-adrenergic receptor activation also promotes plasticity by facilitating long-term potentiation in the hippocampus (Ul Haq et al., 2016; Gelinas & Nguyen, 2005) and by promoting the occurrence of sharp-wave ripples (Ul Haq et al., 2016), which in turn are strongly associated with memory reactivation (Tang & Jadhav, 2018; Kudrimoti, Barnes, & McNaughton, 1999). Finally, we note that offline2 was associated with a localized increase in alpha power in a small parietal region surrounding the left angular gyrus. Although its precise role in memory remains unclear, this area of inferior parietal lobe is commonly activated during episodic memory retrieval tasks (Wagner, Shannon, Kahn, & Buckner, 2005). Alpha may not always indicate the simple inhibition of a brain region (Klimesch, 2012). Speculatively, an increase in alpha power here could represent a modulation of activity during offline memory retrieval. Indeed, angular gyrus is one region where item-specific pattern reinstatement has been observed in recent fMRI studies of episodic memory retrieval (Xiao et al., 2017). Together, these features of offline2 suggest that it could represent a transient neurobiological milieu well-suited to promote memory reactivation and consolidation.
The Offline1 State
Offline1 was most clearly characterized by a prominent increase in alpha power. Alpha oscillations have long been hypothesized to promote inhibition in task-irrelevant cortical networks (Klimesch, 2012), and here may have represented inhibition in regions devoted to sensory processing. In accordance with this hypothesis, task-unrelated internal thoughts were numerically greater during offline1, compared with online trials. Although this effect did not reach statistical significance, in numerous prior studies, increased alpha power has been associated with the subjective experience of mind wandering (Compton, Gearinger, & Wild, 2019; Jin, Borst, & van Vugt, 2019; Baldwin et al., 2017), as well as with the emergence of default mode network activity (Clancy et al., 2022; Mo, Liu, Huang, & Ding, 2013; Knyazev, Slobodskoj-Plusnin, Bocharov, & Pylkova, 2011). Thus, we speculate that offline1 may, like offline2, represent a form of temporary disengagement from task demands. That offline1 was not associated with memory retention suggests that not all periods of decreased task processing are equally well-positioned to support consolidation. If offline1 serves a cognitive function, this function may differ from that supported by offline2. For example, oscillations in the higher end of the alpha frequency range have been argued to support semantic memory access (Doppelmayr, Klimesch, Stadler, Pöllhuber, & Heine, 2002; Klimesch, 1997; Klimesch, Doppelmayr, Pachinger, & Ripper, 1997), which may not have been reflected in the memory performance metrics that we assessed here.
State Features that Did Not Support Our Hypotheses
First, reasoning that offline states should be characterized by lower arousal, we originally predicted that offline trials would be associated with reduced, rather than increased pupil size. However, whereas some investigations have indeed reported that task-unrelated thoughts during the SART are associated with reduced baseline pupil diameter (Smallwood et al., 2011), others have reported just the opposite, finding that mind wandering is associated with increased pupil diameter (Grandchamp, Braboszcz, & Delorme, 2014; Smallwood et al., 2012). Recent studies suggest that optimal task performance may instead occur at intermediate levels of pupil dilation, with some forms of off-task thought being associated with an increased pupil size, and others with smaller pupil size (El Haj & Moustafa, 2021; Unsworth & Robison, 2016; van den Brink, Murphy, & Nieuwenhuis, 2016). Thus, it may be oversimplistic to assume that off-task thought should always be accompanied by reduced pupil size. Second, in a 2020 article using a similar methodology, we observed that offline trials were associated with slower RTs (Wamsley & Summer, 2020). Here, we observed the reverse effect, which is more consistent with prior literature suggesting that poor performance on the SART is associated with speeded RTs on nontarget trials (Polychroni et al., 2020). The divergent results in the current study could be because of the fact that here we arrive at a three-state, rather than a two-state model.
Wakefulness is not a homogenous state. Here, we report that entering a particular form of offline wakefulness after learning is associated with improved memory retention. We suggest that spontaneous alternation between online and offline forms of wakefulness may help to explain how humans balance the competing demands of memory encoding and consolidation. Future studies should seek to further test this hypothesis, for example, by experimentally manipulating the amount of time spent offline following learning, or by asking whether the encoding of new information is indeed optimal during online moments.
Acknowledgments
This research was supported by Bursaries award 211/16 from the BIAL Foundation and NSF grant BCS-1849026. We thank Regan Schuetze and Tingtong Liu for their contributions to data collection. We thank Ted Summer for his contributions in developing earlier versions of this experimental paradigm. Brainstorm (Tadel et al., 2011) is documented and freely available for download online under the GNU general public license (https://neuroimage.usc.edu/brainstorm). EEGLab (Delorme & Makeig, 2004) is documented and freely available under the GNU general public license at https://sccn.ucsd.edu/eeglab/index.php.
Reprint requests should be sent to Erin J. Wamsley, Herman N. Hipp Associate Professor of Psychology, Furman University, Johns Hall 206K 864-294-3218, or via e-mail: [email protected].
Data Availability Statement
Supporting data and R analysis scripts have been made publicly available on Open Science Framework, at https://osf.io/ehdb4/.
Author Contributions
Erin J. Wamsley: Conceptualization; Formal analysis; Funding acquisition; Investigation; Methodology; Project administration; Resources; Software; Supervision; Writing—Original draft; Writing—Review & editing. Madison Arora: Conceptualization; Formal analysis; Investigation; Methodology; Writing—Review & editing. Hannah Gibson: Conceptualization; Investigation; Writing—Review & editing. Piper Powell: Investigation; Methodology; Software; Writing—Review & editing. Megan Collins: Conceptualization; Formal analysis; Investigation; Project administration; Supervision; Writing—Review & editing.
Funding Information
Fundação Bial (https://dx.doi.org/10.13039/501100005032), grant number: 211/16. National Science Foundation (https://dx.doi.org/10.13039/100000001), grant number: BCS-1849026.
Diversity in Citation Practices
Retrospective analysis of the citations in every article published in this journal from 2010 to 2021 reveals a persistent pattern of gender imbalance: Although the proportions of authorship teams (categorized by estimated gender identification of first author/last author) publishing in the Journal of Cognitive Neuroscience (JoCN) during this period were M(an)/M = .407, W(oman)/M = .32, M/W = .115, and W/W = .159, the comparable proportions for the articles that these authorship teams cited were M/M = .549, W/M = .257, M/W = .109, and W/W = .085 (Postle and Fulvio, JoCN, 34:1, pp. 1–3). Consequently, JoCN encourages all authors to consider gender balance explicitly when selecting which articles to cite and gives them the opportunity to report their article's gender citation balance.