Minor Changes Change Memories: Functional Magnetic Resonance Imaging and Behavioral Reflections of Episodic Prediction Errors

Episodic memories allow us to vividly remember events from our personal past (Tulving, 2002). These memories are not always exact but can be modified on the basis of new relevant information (Lee, Nader, & Schiller, 2017; Scully, Napper, & Hupbach, 2017; Nader, 2015; Nader & Einarsson, 2010). Such modifications are potentially fueled by mnemonic prediction errors (PEs), which occur when there is mismatch between what was expected based on memories and the true situational evidence (e.g., Siestrup et al., 2022; Sinclair & Barense, 2019). In that sense, PEs serve as learning signals to update internal predictive models, so that we can maintain valid predictions in the long run (Schubotz, 2015; Friston & Kiebel, 2009; Friston, 2005).

Memories include different kinds of details about experienced episodes that potentially differ in their relevance for our predictions. Although some perceptual and contextual details are not relevant to the overall storyline of an episode, there are central details that represent the episode's gist (Robin & Moscovitch, 2017). It has been suggested that both types of episode information, gist relevant and surface-level details, are encoded and support memory retrieval (e.g., Brainerd & Reyna, 2002). Although some argue that the gist and details of episodes are processed in two different traces (fuzzy trace theory; Brainerd & Reyna, 2002), a different view is that episodic memories can generally be assessed from different levels of specificity, which exist on a continuum (Greene & Naveh-Benjamin, 2021). The gist of an episode can be defined as “the central features of a particular episode (story line)” (Robin & Moscovitch, 2017). Noncentral details of an episode are much more likely to be forgotten (Sekeres et al., 2016) or influenced by misinformation, specifically if such misinformation is in line with the gist of the episode (Reyna, Corbin, Weldon, & Brainerd, 2016; Reyna & Lloyd, 1997). Furthermore, gist information is highly relevant for guiding judgment and decision-making (Corbin, Reyna, Weldon, & Brainerd, 2015). It has even been proposed that the episodic memory trace only represents the gist of an episode, whereas noncentral details are supplemented from semantic memory during retrieval (Cheng, Werning, & Suddendorf, 2016). Taken together, gist-relevant and surface-level details likely differ in their predictive relevance in the context of episodic memory. Mnemonic PEs based on the two types of information should therefore elicit different neural and behavioral responses.

The aim of the present study was to investigate how unexpected episodic modifications that either do or do not change the gist of an episode influence brain activity and memory. We previously demonstrated that subtle changes during episodic cueing lead to specific brain responses for qualitatively different types of modifications, either affecting episode content or structure (Siestrup et al., 2022). Furthermore, encounters with both types of modified episodes induced false memories in a post-fMRI memory test (Siestrup, Jainta, Cheng, & Schubotz, 2023; Jainta et al., 2022; Siestrup et al., 2022). In the present work, we focused on content modifications and specifically manipulated the impact of those on the episodes' gist.

In a first experimental session, participants encoded different episodes consisting of short toy stories from video material. On the next day, participants returned for an fMRI session during which they were presented originally encoded episodes or slightly modified versions thereof. Episode videos were modified by the exchange of a single object (Siestrup et al., 2022). Although some of these modifications were irrelevant for the episode's storyline (in the following termed surface modifications), others intentionally changed it (in the following termed gist modifications). This intended impact of the content modifications was validated in a behavioral pilot study. On a third day, participants completed a post-fMRI memory test to probe memory for originally encoded episodes. To this end, participants rewatched original as well as modified episode videos and had to decide if these had been encoded during the first session. Lastly, participants went through a rating task to assess their subjective evaluations of episode modifications.

We hypothesized that both types of modifications would trigger activation in specific brain regions we previously identified to respond to content modifications in our paradigm (Siestrup et al., 2022), including occipitotemporal cortex (OTC), fusiform gyrus (FG), (posterior) intraparietal sulcus (IPS), and ventrolateral pFC, specifically inferior frontal gyrus (IFG) and inferior frontal sulcus (IFS) in the right hemisphere. These brain regions are known to be relevant for the processing of (new) object information in actions (El-Sourani, Trempler, Wurm, Fink, & Schubotz, 2019; Lingnau & Downing, 2015; Wiggett & Downing, 2011; Reber, Gitelman, Parrish, & Mesulam, 2005; Grill-Spector, Kourtzi, & Kanwisher, 2001). Modifications that change the gist of the story were expected to elicit more pronounced activation in said areas because of their overall higher relevance for episode content and therefore enhanced processing at both encoding and retrieval.

In addition, we expected that there would be differences in brain activation between the two types of content modifications, for example, in the hippocampal complex. When a modification leads to inconsistency with the known episode storyline, there could be increased activation in areas relevant to novelty processing. In a previous study, we found that the hippocampus (HPC) responded more strongly to novel compared with familiar episodes in our paradigm (Jainta et al., 2022). Therefore, it is possible that gist modifications elicit stronger hippocampal responses than surface modifications. Accordingly, parahippocampal areas were found to be included in the processing of scene gist (Schubotz & Von Cramon, 2009; Oliva & Torralba, 2006; Epstein, 2005). However, it may be that surface modifications trigger stronger activation in HPC than do gist modifications, as it has been suggested that the HPC is more relevant to details in episodes and that gist representations become independent of the HPC, but rather after longer delays between encoding and retrieval (e.g., Winocur & Moscovitch, 2011). It has also been demonstrated that episode details are predominantly represented in posterior HPC, whereas the anterior HPC processes episode gist (Sekeres, Winocur, & Moscovitch, 2018; Sekeres, Winocur, Moscovitch, Anderson et al., 2018; Robin & Moscovitch, 2017; Moscovitch, Cabeza, Winocur, & Nadel, 2016; Poppenk, Evensmoen, Moscovitch, & Nadel, 2013). Therefore, it is likely that both types of modifications trigger hippocampal activation, but the localization of activity within HPC was expected to differ between the two modification types. Specifically, activation in anterior HPC was expected to be larger for gist than surface modified cues, and vice versa in posterior HPC. Aside from that, there is evidence that episode gist is mediated by ACC (Sekeres, Winocur, & Moscovitch, 2018) and OFC (Schubotz & Von Cramon, 2009), so activation in these brain areas might be higher for gist than for surface modifications.

Concerning memory performance in the post-fMRI memory test, we expected to replicate our general finding that mnemonic PEs during the fMRI session trigger false memories afterward, putatively because of internal model updating (Schiffer, Ahlheim, Ulrichs, & Schubotz, 2013; Schiffer, Ahlheim, Wurm, & Schubotz, 2012). Therefore, one hypothesis was that prediction violations lead to increased acceptance of modified episode videos as originally encoded, potentially accompanied by a decreased acceptance for unmodified videos (Siestrup et al., 2023, 2022; Jainta et al., 2022). As mentioned above, memories for noncentral details of episodes are usually less accurate. Therefore, we reasoned that memory accuracy for originally encoded episodes, as assessed by false alarm and hit rates and, more generally, the area under the curve (AUC; Brady, Robinson, Williams, & Wixted, 2022), might be generally lower for surface than for gist-modified episodes. In addition, prediction violation through surface and gist modifications might influence subsequent memory differently, in line with previous reports that surface-level detail is more likely to be influenced by misinformation (Reyna et al., 2016; Reyna & Lloyd, 1997). Thus, false memories might arise as a consequence of prediction violation because of surface, but not, or to a lesser extent, because of gist modifications.

Forty-two volunteers (28 women, 14 men) participated in the study. All were native German speakers and right-handed as assessed by the Edinburgh Handedness Inventory (Oldfield, 1971). Participants had (corrected-to-) normal vision, had intact color perception, and reported no history of neurological or psychiatric disorders. Data from three participants were excluded from analyses because of increased movement during the fMRI session (two participants; ca. 5-mm and 7-mm movement) and a benign anatomical abnormality of the brain that impeded normalization (one participant). Two participants terminated the study preliminarily for personal reasons. The final sample consisted of 37 participants (25 women, 12 men) between the age of 18 and 29 years (M_age = 23 years, SD = 3.17 years). Similar sample sizes have previously yielded stable results (Jainta et al., 2022; Siestrup et al., 2022). All participants gave written informed consent to participate in this study and were reimbursed with course credits or money. The study was conducted in accordance with the Declaration of Helsinki and approved by the Local Ethics Committee of the University of Münster.

The video stimuli were very similar to those used in our previous studies (Jainta et al., 2022; Siestrup et al., 2022; available upon request at https://www.uni-muenster.de/IVV5PSY/AvicomSrv/). We worked with 80 short videos (duration = 8.8–14.76 sec, M = 12.31 sec), which showed an actress manipulating PLAYMOBIL toys. Only the toy objects and the hands and arms of the actress were visible and filmed from above with a digital single-lens reflex camera (Nikon D5300). For filming, the actress wore a black pullover and black rubber gloves. Matte white paper served as a base. Objects that were needed for a particular story were positioned on a table and were only moved into view in the moment at which they appeared in the story. Stories included six to nine action steps (M = 7.60 steps) and four to 11 separable objects (M = 6.23 objects). The exact same object appeared in only one of the stories.

Video material was edited with Adobe Premiere Pro CC (Adobe Systems Software, Version 12.1.2). Final video stimuli had a frame of size 1920 × 1080 pixels and a frame rate of 25 frames per second. They always started with seven frames showing only background (white base) and ended after seven frames showing the final object constellation. Throughout the study, videos were presented using the stimulus presentation software Presentation (Version 20.3 02.25.19, NeuroBehavioral Systems) at a visual angle of approximately 7.3° × 13°.

There were 24 stories that existed in three different versions each. The first version was the original version, which was used for encoding. In addition, we created two modified version. In those modified versions, one single object was exchanged compared with the original. The new object was never just the same object in a different color; the new object diverged from the old one in color and shape. In one modified version, the object exchange did not affect the storyline (i.e., gist) of the episode. In the following, this type of modification will be referred to as surface modification. In the other modified version, the new object changed the storyline, that is, affected the gist of the episode (gist modification). We validated our a priori classification of surface and gist modifications in a behavioral pilot study and confirmed that all modifications could be identified by participants. Modifications were not introduced in the first two action steps so that the beginning of a video served as a cue for prediction. No modifications were introduced in the last two action steps either. The exact time point of the modification in each video was determined by identifying the video frame that diverged from the original version. For an example of an episode and its modified versions, see Figure 1.

Two additional videos were used for practice trials for the encoding, fMRI, and rating tasks. One of those existed in only one version, and one additionally had two modified alternatives. Four more stories were first introduced in the fMRI experiment. They existed in one version only and will be referred to as novels in the following.

The encoding session was conducted in a computer laboratory at the University of Münster and lasted, on average, 2 hr 9 min. For encoding, participants sat at the setup where the stimulus material had been filmed and wore black gloves like those seen in the video. Episodes were encoded by watching each video 5 times. Afterward, participants had to give a detailed description to the experimenter, which had to include all objects (including their color) and the correct order of actions in the story. If the participant made a mistake, the experimenter interrupted and corrected them, and the participant had to start the description anew, until one completely correct description was given. On average, participants needed 1.54 description attempts per story (SD = 0.22). The context and interpretation of the story were discussed and clarified, to ensure that all participants encoded the story the same way. Lastly, the participant was asked to summarize the story in a short sentence. Each participant encoded all 24 episodes in a randomized order. The session started with a short practice phase (two videos) to familiarize the participant with the task. At the end of the encoding session, participants briefly practiced the tasks they would do during the fMRI session.

The fMRI session was conducted on the day after the encoding. Participants went through two experimental parts, which are described in the following.

During the anatomical measurements, participants went through a so-called incomplete reminder to initiate memory destabilization, presumably facilitating later modification (Sinclair & Barense, 2018, 2019). Each video that had previously been encoded was presented, but interrupted during the step preceding the modification that would be presented during the fMRI experiment (e.g., while retracting the hand). Videos that would be shown in the original version during the fMRI experiment were shown in interrupted fashion as well. The time point of the interruption then matched the modified version that was later presented as an alternative in the memory test. It has been shown that allowing for complete retrieval following an incomplete cue can enhance memory (Antony, Ferreira, Norman, & Wimber, 2017). Therefore, participants had to solve a dummy task between incomplete video trials to prevent complete retrieval. This task was introduced to the participants as a “warm-up-task.” After each video, a number between 1 and 4 was presented for 300 msec and participants had to indicate via a button press whether the number was odd (index finger) or even (middle finger).

The fMRI experiment closely followed our previously published protocol (Jainta et al., 2022; Siestrup et al., 2022). During the fMRI session, participants were presented with videos reminiscent of the previously encoded episodes. Videos were either displayed in the original version as during encoding (eight videos) or in either the surface or gist modified version (eight videos each). Which stories belonged to which conditions was balanced over participants. As reported previously, all episodes were presented repeatedly to simulate natural circumstances that potentially benefit memory modification (Siestrup et al., 2022; Schiffer, Ahlheim et al., 2013; Schiffer et al., 2012). In addition, four novel stories were shown in the fMRI session for the first time.

The experiment consisted of six blocks. Each video was presented once per block. In addition, each block contained three null events during which only a fixation cross was presented (duration: 7–10 sec).

Participants were instructed to attentively watch the presented videos. They were told that after some videos, they were required to solve a quick task about the previously presented video. Although the participants were not aware of this, the task was merely included to ensure the participants' constant attention (Jainta et al., 2022; Siestrup et al., 2022; El-Sourani et al., 2019) and was not of interest for our actual research question. These task trials were characterized by the presentation of two verbs left and right of the fixation cross. One verb was closely associated with the video shown before; one was not at all associated with the video (as validated by three experimenters). Importantly, verbs did not describe any action actually depicted in the video, and all associations remained intact in the case of modifications. The participants' task was to press the left button (index finger) if the left verb matched the video, and to press the right button (middle finger) if the right verb matched the video. Whether the associated verb was presented on the left or right was pseudorandom and balanced between conditions for each participant. The verbs were presented for a maximum of 3 sec or until participants responded. Upon response delivery, participants received a 1-sec written feedback whether they answered correctly, incorrectly, or too late, in case no response was given. Each video was followed by a task trial twice during the experiment, resulting in a total number of 56 task trials in the experiment.

A fixation cross was presented between trials for 2 sec (1 sec after task trials) to serve as an interstimulus interval. In addition, a variable jitter of 0, .5, 1, or 1.5 sec of fixation was added before each trial to enhance the temporal resolution of the BOLD response (Figure 2). In total, the fMRI task had a duration of approximately 48 min. Transition probabilities were balanced between different trial types over the whole experiment.

The post-fMRI memory test was conducted on the day after the MRI session and followed a similar protocol as previously reported (Jainta et al., 2022; Siestrup et al., 2022). Participants were not informed that their memory for encoded episodes would be tested at any time during the experiment.

Participants were seated in front of a computer and instructed to remember the encoding session. Then, they were presented all stories in two different versions. When modified videos had been presented during the fMRI experiment, these versions were presented again in the memory test and, additionally, each original counterpart was presented. When original episodes had been presented during the fMRI experiment, they were presented again and, additionally, each story was displayed either with a surface or a gist modification (in half of the cases each). Videos were presented in a pseudorandomized order, so that half of the stories (of each experimental condition) were first presented in their original version followed by a modified version and vice versa. Novel videos were displayed twice in the same version. Versions of the same episode were not displayed in direct succession but with minimally two other videos in between. In total, the memory test comprised 56 video trials.

After each video, participants had to rate how confident they were that they had encoded exactly this episode, using a Likert scale from 1 to 6 (with 1 = 100% no, 2 = 80% no, 3 = 60% no, 4 = 60% yes, 5 = 80% yes, 6 = 100% yes). Participants were instructed to respond quickly and intuitively. The duration of the task was approximately 15 min.

The rating task was conducted after the post-fMRI memory test on the third day of the study. Each original video was presented, followed by the same modified version that had been presented in the fMRI and/or memory test. Participants had to press a button as soon as they noticed the change in the modified video. The video was played completely and not interrupted by the response. Participants were instructed to carefully watch the full video. They then were explicitly asked if they noticed a change or not and had to respond via a button press. This question was included to allow participants who missed to indicate the modification during the video to still rate the modification if they had seen it. If participants answered “no,” no further questions were displayed. If they had noticed the change, they were asked how unexpected they perceived the content modification on a Likert scale from 1 to 6 (with 1 = 0% unexpected, 2 = 20% unexpected, 3 = 40% unexpected, 4 = 60% unexpected, 5 = 80% unexpected, 6 = 100% unexpected; PE rating). Importantly, they were instructed to rate the unexpectedness in reference to the original story and to not consider that they might have noticed the modification previously during the experiment. Next, they were asked how much the change affected the story of the episode on a Likert scale from 1 to 6 (with 1 = 0% different, 2 = 20% different, 3 = 40% different, 4 = 60% different, 5 = 80% different, 6 = 100% different; story-change rating). Lastly, the participant had to describe the modification orally to the experimenter. The task was self-paced so that participants always started a new video themselves via a button press. The duration of this task was approximately 20 min.

Magnetic resonance imaging was conducted with a 3-Tesla Siemens Magnetom Prisma MR tomograph (Siemens) using a 20-channel head coil. Participants lay supine on the scanner bed. Movements of head, arms, and hands were minimized by fixation with form-fitting cushions. The participants' right index and middle finger were positioned on the two appropriate buttons on a response box. To attenuate scanner noise, participants were provided with earplugs and headphones. Visual stimuli were presented via a screen that participants saw through a mirror mounted on the head coil.

Before functional imaging, high-resolution T1-weighted anatomical images were obtained with a 3-D magnetization prepared rapid gradient echo sequence (scanning parameters: 192 slices, slice thickness = 1 mm, repetition time = 2130 msec, echo time = 2.28 msec, flip angle = 8°, field of view = 256 × 256 mm²). Functional images of the whole brain were acquired in interleaved order along the AC–PC plane using a gradient-echo EPI sequence to measure BOLD contrast (scanning parameters: 33 slices, slice thickness = 3 mm, repetition time = 2000 msec, echo time = 30 msec, flip angle = 90°, FoV = 192 × 192 mm²).

Imaging data were preprocessed with SPM12 (Wellcome Trust) implemented in MATLAB (Version R2020b, MathWorks Inc.). Preprocessing included slice time correction to the middle slice, movement correction and realignment to the mean image, co-registration of the individual structural scans to the mean functional image, normalization of functional and structural images into the standard MNI space (Montreal Neurological Institute, Montreal, QC, Canada) on the basis of segmentation parameters, and spatial smoothing using a Gaussian kernel of FWHM of 8 mm. A 128-sec high-pass temporal filter was applied.

fMRI data were analyzed using general linear models (GLMs) for serially autocorrelated observations (Worsley & Friston, 1995; Friston et al., 1994) implemented in SPM12. Regressors were convolved with the canonical hemodynamic response function. We used the smoothed individual normalized gray matter image (8-mm FWHM), which was thresholded at .2 using ImCalc in SPM12 to create a binary gray matter mask that was applied at the first level of analyses.

The first model (GLM1) included 15 regressors. These were original videos (original), videos containing a surface modification (surface), and videos containing a gist modification (gist), each comprising 48 trials. For modified videos, onsets of events were time-locked to the point in the video at which the modification occurred (time of modification). For original trials, a hypothetical time of modification was calculated (mean of times that corresponded to points of surface and gist modifications in the nonmodified video) to serve as a comparable onset. As reported previously (Siestrup et al., 2022), these conditions were modeled as events to investigate phasic effect of the prediction violation at the precise moment of its occurrence. A parametric modulator was added to each of those regressors to model the repeated presentation of each video (descending coding). Please note that all parametric modulators were mean centered (Mumford, Poline, & Poldrack, 2015). The 24 novel videos were modeled as events, and onsets were timed to the middle of each video. In addition, regressors for the 18 null events and 56 task trials were included, with onsets time-locked to their respective onsets. Null events were modeled as epochs, including the full presentation time (7–10 sec). Task trials were modeled as events. The six subject-specific rigid-body transformations obtained from realignment were included as regressors of no interest.

The parametric modulation described above might not optimally account for differences between the first presentation of a modification (potentially the most surprising one) and all subsequent ones, as it assumes a linear decrease in activation. For this reason, we calculated a second model (GLM2) as an additional control for the repeated presentation of the same modification. The difference between GLM1 and GLM2 was that original, surface, and gist trials were separately modeled for the first presentation (eight trials per condition) and the remaining five presentations (40 trails per condition). No parametric modulators were included. We then calculated the same contrasts as for GLM1 (see below), but only including the second to sixth presentations of each condition. Qualitatively, results from whole-brain as well as ROI analyses were the same as for GLM1, with a slight decrease in power because of the reduced number of trials.

To control for participants' individual variation in the perception of surface and gist modification, as assessed by the rating task, we computed a third model (GLM3). Trials for original and novel videos, null events, and tasks were modeled as described for GLM1. Modified videos were not separated into surface and gist modifications, but included in the model as one regressor to which we added a parametric modulator composed of the participants' story-change rating for each video obtained from the rating task (mean-centered). No further parametric modulators were added. Including movement parameters, GLM3 comprised 12 regressors. The parametric analysis of differences between surface and gist modifications gave rise to highly similar results as those we report for GLM1 with our a priori classification of modification types.

We calculated first-level t-contrasts for surface > original and gist > original as well as the direct contrasts surface > gist and gist > surface to analyze brain activity in response to the different modification types. In addition, we calculated the reverse contrasts, original > surface, and original > gist to further validate the specificity of modification responses. To investigate potential attenuation effects in brain responses because of repeated presentations of the same episodic modifications, we additionally computed the contrasts for the parametric modulators for surface and gist modifications against the implicit baseline. The corresponding results are reported in the appendix (Appendix A). Second-level group analyses were conducted with one-sample t tests across participants. Furthermore, we computed the first-level t-contrasts for each type of video versus novel videos (original > novel, surface > novel, gist > novel) and created the conjunction of these on the second level (Nichols, Brett, Andersson, Wager, & Poline, 2005) to replicate our previous findings of episodic retrieval in general (Jainta et al., 2022; Siestrup et al., 2022). We applied false discovery rate (FDR) correction with a threshold of p < .05 or higher (voxel level) to resulting t maps. For the conjunction contrast concerning episodic retrieval, we applied a more conservative threshold of p < .001 because due to the very strong activation, a threshold of p < .05 would not have been conclusive for relating activation to anatomical regions. For completeness of our analysis and with regard to our hypotheses, when no significant activation could be detected using this threshold, we applied a threshold of p < .001 (uncorrected). Generally, we only report clusters with a minimum number of 10 voxels. Brain activation was visualized with MRIcroGL (Version 1.2.20200331, McCausland Center for Brain Imaging, University of South Carolina).

With regard to our hypotheses, we performed ROI analyses to more specifically investigate which brain areas are involved in the processing of surface and gist modifications. We hypothesized that both, surface and gist modifications, activate brain regions we identified to be involved in the processing of content modifications in general in our paradigm. We used functional ROIs of these regions, namely, right IFS, bilateral OTC and IPS (Siestrup et al., 2022), and extracted mean contrast estimate values from the surface > original and gist > original contrasts using the MarsBar Toolbox (Brett, Anton, Valabregue, & Poline, 2002). Functional ROIs were created from the peak voxel coordinates we identified in our previous study (Siestrup et al., 2022; right IFS: x = 42, y = 8, z = 32; left OTC: x = −42, y = −58, z = −7; right OTC: x = 51, y = −52, z = −10; left IPS: x = −27, y = −61, z = 50; right IPS: x = 33, y = −67, z = 35), which served as central points for spheres with a radius of 6 mm.

Furthermore, we investigated the influence of surface and gist modifications on activation in anterior HPC and posterior HPC. For creating hippocampal ROIs, we used the automated anatomical labeling atlas (Tzourio-Mazoyer et al., 2002) from the Wake Forest University Pickatlas toolbox (Maldjian, Laurienti, Kraft, & Burdette, 2003) in SPM12 to extract maps of the left and right HPC. To divide the HPC into anterior and posterior sections, we used an anterior–posterior border at y = −21 (Poppenk et al., 2013). To avoid contamination between the two sections, a 2-mm coronal gap was introduced from this border in anterior and posterior directions (Guo et al., 2020; Li, Li, Wang, Li, & Li, 2018). Contrast estimates were extracted for surface > original and gist > original from anterior HPC and posterior HPC, aggregated over both hemispheres. We conducted an additional explorative ROI analysis using a more fine-grained hippocampal parcellation. Results are reported in Appendix B.

Contrast estimates for surface > original and gist > original were subjected to one-tailed, one-sample t tests to check for significant activation within ROIs. Furthermore, we used two-tailed paired t test to compare contrast estimates from surface > original and gist > original in the functional ROIs (right IFS, OTC, IPS). For hippocampal ROIs, we conducted a 2 × 2 repeated-measures analysis of variance (rmANOVA) with the factors ROI (anterior HPC, posterior HPC) and Modification (surface, gist). Pairwise comparisons between surface > original and gist > original in hippocampal ROIs were conducted using one-tailed t tests, according to our hypotheses. Briefly, we expected that in anterior HPC, activation would be larger for gist > original compared with surface > original, whereas the opposite was expected for posterior HPC (e.g., Poppenk et al., 2013). p Values obtained from one-sample and pairwise t tests were Bonferroni-corrected for multiple comparisons within each ROI/analysis.

The analysis of behavioral data was conducted with RStudio (R Core Team, 2020; Version 1.3.1073).

To test the hypothesis that repeated presentations of modified videos during the fMRI decrease memory accuracy in the post-fMRI memory test, we computed the AUC (Brady et al., 2022), as well as false alarm and hit rates (Ratings 1, 2 and 3 aggregated as “rejection,” and 4, 5, and 6 aggregated as “acceptance”). In addition, we considered RTs from the post-fMRI memory test, as longer RTs in memory tasks are believed to be indicative of increased retrieval difficulty because of higher cognitive processing demands (Larsen & Plunkett, 1987; Noppeney & Price, 2004). For the analysis of AUC, hit rates, false alarm rates, and RTs for modified videos in the post-fMRI memory test (modified_MT), a 2 × 2 within-subject factorial design with the factors Modification_fMRI (yes, no) and Version_MT (surface, gist) was applied. For analyzing RTs for original videos in the post-fMRI memory test (original_MT), we applied a within-subject design with the factor Version_fMRI (original, surface, gist).

To again confirm our classification of surface and gist modifications, we analyzed participants' story-change ratings from the rating task. Furthermore, we considered the RTs to notice changes in modified videos. The same 2 × 2 within-subject factorial design as described above was applied. In individual cases, participants' story-change ratings were inconsistent with our a priori classification. This means that participants sometimes chose higher story-change ratings (Ratings 4, 5, 6) when we had previously rated the change to be less impactful (corresponding to Ratings 1, 2, 3) or vice versa. On average, this applied to 1.70 out of 24 videos per participant. To address this individual variation, we repeated all behavioral analyses with individually rearranged surface and gist modification conditions, which consistently gave rise to the same results we report on the basis of our a priori classification.

Furthermore, we analyzed the behavioral data from the cover task in the fMRI session by calculating the error rate and mean RT according to the within-subject factor Version_fMRI (original, surface, gist, novel) per participant. In only 0.77% of all question trials, no response was given so that these trials were not further considered in the analysis.

For all RT analyses, only trials with correct answers were considered. Furthermore, extreme outliers in RT (as defined as values above Quartile 3 + 3 * interquartile range or lower than Quartile 1–3 * interquartile range) were removed from each participant's data. RTs were averaged over all trials of the same factorial combination for each participant. For choosing statistical tests, data were inspected for normal distribution using the Shapiro–Wilk test and checked for extreme outliers. When data were normally distributed or could be transformed to fit normal distribution (RTs from memory test and rating task; logarithmic transformation) and included no extreme outliers, we used conventional rmANOVA. When the prerequisites for parametric analysis were not met, we applied nonparametric rmANOVA based on aligned rank-transformed data (package ARTool; Wobbrock, Findlater, Gergle, & Higgins, 2011). Post hoc pairwise comparisons were conducted with paired t tests or Wilcoxon signed-ranks tests (one-tailed with respect to the hypotheses, always two-tailed for explorative analysis of fMRI task).

We report mean values and standard errors of the mean. A significance level of α = .05 was applied, and we report Bonferroni-corrected p values for pairwise comparisons (Bonferroni, 1936). If the assumption of sphericity was violated (Mauchly's test of sphericity), we report Greenhouse–Geisser-corrected degrees of freedom and p values.

We conducted a behavioral pilot study in an independent sample with 18 participants (M_age = 23.839 years, SD = 4.730, age range = 18–32 years, 15 women, three men) to validate the suitability of our stimulus material and procedure. Participants encoded episodes the same way as participants of the fMRI study did. On the next day, participants returned to complete a memory test and a rating task. Both were conducted as described for the main study.

In total, memorability of all videos was comparable, as we did not detect any extreme outliers in AUC per video (M = .950 ± .010). As expected, participants showed a reduced memory performance in the surface modification condition compared with the gist modification condition (M_surface = .907 ± .013; M_gist = .995 ± .003; Z = −3.88, p < .001 [one-tailed]) in the memory test. Although we expected that all modifications, no matter if surface or gist modification, could be visually identified by the participants, it might take longer to become subjectively aware of surface modifications because of the smaller impact on the storyline. Indeed, participants did not miss significantly more surface than gist modifications (M_surface = 0.278 ± 0.135, M_gist = 0.0 ± 0.0; Z = −1.70, p = .089 [two-tailed]). Over the whole pilot study, five out of 24 surface modifications were missed only once each; gist modifications were never missed. When analyzing the time it took participants to notice the modifications in the rating task, we did not detect a significant difference between conditions (M_surface = 907.682 ± 75.694 msec, M_gist = 822.913 ± 64.316 msec; t(17) = −1.44, p = .084 [one-tailed]). The rating for the change of storyline was significantly higher for gist than for surface modifications (M_surface = 1.304 ± 0.059, M_gist = 4.894 ± 0.110; t(17) = 38.88, p < .001 [one-tailed]), as expected. Therefore, we validated that surface and gist modifications were perceived according to our a priori categorization.

During the fMRI session, participants solved a cover task during which two verbs were presented after some videos and participants had to indicate which verb was more closely associated with that video. A nonparametric rmANOVA on error rates with the factor Version_fMRI (original, surface, gist, novel) did not reveal significant differences, F(3, 108) = 0.22, p = .883, ηp² = .01. For all factor levels, error rates were generally low (M_original = .061 ± .009; M_surface = .056 ± .011; M_gist = .058 ± .009; M_novel = .100 ± .024). Version_fMRI had a significant effect on RTs in the fMRI task, F(3, 108) = 7.89, p < .001, ηp² = .18. Post hoc paired t tests revealed that participants took significantly longer to respond in the task after novel videos than after any other version, original versus novel: t(36) = −4.08, p = .001; surface versus novel: t(36) = −3.35, p = .012; gist versus novel: t(36) = −3.48, p = .008; M_original = 1267.910 ± 35.721 msec; M_surface = 1275.592 ± 32.560 msec; M_gist = 1277.184 ± 33.558 msec; M_novel = 1328.929 ± 35.486 msec (all tests two-tailed), whereas other pairwise comparisons did not reach significance, t(36) > −0.80, p = 1 (all tests two-tailed).

To investigate which brain regions respond to surface or gist modification in episodic cueing, we calculated the contrasts gist > original and surface > original. To better understand differences between different types of episodes, we also computed the reverse contrasts, original > gist and original > surface.

In contrast to original episodes, gist-modified episodes elicited activation in bilateral IPS, inferior frontal junction (IFJ), IFS (BA 45), and FG. In addition, we found significant activation in right angular sulcus, inferior temporal sulcus, (posterior) middle temporal gyrus (pMTG), and posterior superior temporal sulcus (pSTS; Table 1, Figure 3A). The reverse contrast (original > gist) revealed reduced activation of several brain regions in gist-modified episodes. These were bilateral pregenual ACC (pACC), right superior frontal sulcus (SFS), and left midcingulate cortex (MCC; Table 1, Figure 3C). Interestingly, this deactivation pattern was reminiscent of the one we previously reported for novel episodes compared with reactivated ones (Jainta et al., 2022; Siestrup et al., 2022).

For surface-modified episodes compared with original ones, we did not detect significant activation after correction for multiple comparisons. For completeness, we present results without correction for multiple comparisons at p < .001. Such subthreshold activation (uncorrected, p < .001) was detected in hypothesized brain regions, which were the right IPS and an inferior frontal area (IFJ), as well as left FG. In addition, activation clusters were located in the right medial superior frontal gyrus (SFG; BA 8), caudate nucleus, and anterior insula (Table 1, Figure 3B). The reverse contrast (original > surface) did not yield significant activation with correction for multiple comparisons. Subthreshold activation clusters were found in bilateral pSTS, as well as in left postcentral sulcus, posterior MTG, and middle occipital gyrus (Table 1).

In addition, we calculated the direct contrasts between both modification types (gist > surface, surface > gist). Compared with surface modifications, gist modifications elicited significantly higher activation in bilateral IFJ/IFS (BA 44, 45), pMTG, pSTS, and FG. In addition, activity was increased in left IPS and right precuneus and postcentral sulcus (Table 2, Figure 4A). Surface modifications yielded higher brain activation than gist modifications in bilateral pACC, MCC, and left cuneus, as well as right SFS (Table 2, Figure 4B).

Aside from our main research question, we aimed to replicate our previous findings concerning the neural effects of episodic recall compared with novel episodes. To this end, we calculated the conjunction of the three contrasts original > novel, surface > novel, and gist > novel. Because of the very strong activation we found for this contrast, we used a more conservative threshold of p < .001 (FDR) to allow for a better attribution of activation to anatomical regions. In fact, we detected activation in previously identified brain regions. These were bilateral pACC, lingual gyrus, cuneus, precuneus, posterior cingulate cortex (PCC), angular gyrus (AG), middle frontal gyrus, and left insula (Jainta et al., 2022; Siestrup et al., 2022). In addition, we found activation in bilateral SFS and MTG, as well as left STS and putamen (Table 3, Figure 5).

To more specifically investigate which brain areas are in involved in the processing of surface and gist modifications, we performed ROI analyses. We used functional ROIs (right IFS, OTC, IPS) of regions that responded to content modification, that is, object exchanges, in our previous study (Siestrup et al., 2022) to test our hypotheses that (1) both types of modifications elicit activation in these areas and (2) that activation in these regions might be stronger for gist than for surface modifications. Further ROIs were the anterior and posterior HPC to test whether gist and surface modifications differ regarding long-axis hippocampal activation. Specifically, we expected that in anterior HPC, activation would be larger for gist > original compared with surface > original, whereas the opposite was expected for posterior HPC (e.g., Poppenk et al., 2013). Mean contrast estimates were extracted from surface > original and gist > original contrasts.

In right IFS, there was significant activation for both, surface and gist modifications in contrast to originals, surface: M = 0.403 ± 0.133, t(36) = 3.03, p = .005 (one-tailed); gist: M = 0.829 ± 0.148, t(36) = 5.62, p < .001 (one-tailed). A paired t test revealed that activation was greater for gist than for surface modifications, t(36) = −3.27, p = .002 (two-tailed). In the OTC ROI, gist modifications showed significant activation, M = 0.383 ± 0.080, t(36) = 4.79, p < .001 (one-tailed), whereas there was a nonsignificant trend for activation in surface modifications, M = 0.147 ± 0.078, t(36) = 1.90, p = .066 (one-tailed). Gist modifications elicited significantly higher activation than surface modifications, t(36) = −2.35, p = .025 (two-tailed). Likewise, gist modifications led to significant activation in the IPS ROI, M = 0.430 ± 0.132, t(36) = 3.25, p = .002 (one-tailed), whereas this was not the case for surface modifications, M = 0.138 ± 0.111, t(36) = 1.25, p = .219 (one-tailed). The difference between the two modification types was significant, t(36) = −2.30, p = .028 (two-tailed).

In anterior and posterior HPC, neither type of modification elicited significant activation, anterior HPC: surface: M = 0.026 ± 0.033, t(36) = 0.78, p = .876 (one-tailed); gist: M = 0.033 ± 0.044, t(36) = 0.75, p = .915 (one-tailed); posterior HPC: M = 0.066 ± 0.036, t(36) = 1.84, p = .147 (one-tailed); M = −0.021 ± 0.041, t(36) = −0.51, p = 1 (one-tailed). A rmANOVA with the factors ROI (anterior HPC, posterior HPC) and Modification (surface, gist) revealed no significant effects of ROI, F(1, 36) = 0.07, p = .794, ηp² = .00, or Modification, F(1, 36) = 0.93, p = .342, ηp² = .03, but a significant interaction of both factors, F(1, 36) = 10.18, p = .003, ηp² = .22. Pairwise t tests revealed that, as expected, activation was significantly stronger for surface modifications than for gist modifications, t(36) = 2.04, p = .049 (one-tailed), in posterior HPC. However, activation for modification types did not differ in anterior HPC, t(36) = −0.15, p = .884 (one-tailed; Figure 6).

In the post-fMRI memory test, participants were presented modified episodes from the fMRI session and, additionally, their originally encoded counterparts. Episodes that had appeared in the original version during scanning were now presented in the original and, additionally, one modified version. After each video, participants had to rate how confident they were that they had originally encoded exactly this episode.

We calculated a nonparametric rmANOVA with the factors Modification_fMRI (yes, no) and Version_MT (surface, gist) to analyze AUC values as a measure of memory performance in the post-fMRI memory test. There was a significant effect of Modification_fMRI, F(1, 36) = 18.94, p < .001, ηp² = .34, as lower AUC values were achieved when participants had previously encountered modified episodes during the fMRI session (M_yes = .906 ± .010; M_no = .936 ± .010). In addition, there was a significant effect of Version_MT, F(1, 36) = 239.97, p < .001, ηp² = .87, driven by higher AUC values for gist than for surface-modified versions (M_surface = .853 ± .016; M_gist = .988 ± .004). Furthermore, the interaction between Modification_fMRI and Version_MT was significant, F(1, 36) = 12.35, p = .001, ηp² = .26. Post hoc Wilcoxon tests revealed that AUC values were only reduced significantly by previous experience with modified versions for surface (z = −2.97, p = .006 [one-tailed]) but not for gist modifications (z = −1.00, p = .634 [one-tailed]; Figure 7A).

Notably, AUC scores obtained from the pilot study and the main study when only originals were considered from the fMRI session, that is, when participants did not have any previous experience with modified episodes before the memory test, were highly similar (pilot study: M_surface = .907 ± .013; M_gist = .995 ± .003; main study: M_no-surface = .882 ± .019; M_no-gist = .990 ± .006).

To more specifically investigate the influence of modified episodic cueing on memory, we additionally analyzed the false alarm and hit rates. Using a nonparametric rmANOVA, we found a significant effect of the factor Modification_fMRI on false alarm rates, F(1, 36) = 41.59, p < .001, ηp² = .54, which were higher when modified episodes had been experienced already during the fMRI session (M_yes = .188 ± .017; M_no = .118 ± .018). In addition, there was a significant influence of Version_MT, F(1, 36) = 473.63, p < .001, ηp² = .93, as participants had a higher tendency to falsely accept surface-modified versions compared with gist-modified versions as originals (M_surface = .291 ± .028; M_gist = .015 ± .006). In addition, there was a significant interaction between both factors, F(1, 36) = 18.16, p < .001, ηp² = .34. Post hoc Wilcoxon tests showed that there was only an increase in false alarms for surface modifications (z = −3.29, p = .003 [one-tailed]), but not for gist modifications (z = −1.47, p = .284 [one-tailed]) when participants had previously seen modified versions during scanning (Figure 7B). Hit rates were close to ceiling for all experimental conditions (M_yes-surface = .929 ± .020; M_yes-gist = .949 ± .014; M_no-surface = .932 ± .023; M_no-gist = .966 ± .014) and were neither significantly affected by the factor Modification_fMRI, F(1, 36) = 1.84, p = .184, ηp² = .05; Version_MT, F(1, 36) = 0.02, p = .899, ηp² = .00; nor an interaction of both, F(1, 36) = 0.23, p = .631, ηp² = .01 (Figure 7C).

For modified_MT videos, there was a significant effect of Modification_fMRI on RTs, F(1, 36) = 4.43, p = .042, ηp² = .11, as participants responded faster when they had not experienced the modification during the fMRI session (M_yes = 471.383 ± 23.250 msec; M_no = 439.958 ± 20.392 msec). In addition, there was a nonsignificant trend for an effect of Version_MT, F(1, 36) = 3.57, p = .067, ηp² = .09, driven by the participants' tendency to respond faster for gist than for surface-modified videos (M_surface = 475.620 ± 25.157 msec; M_gist = 435.721 ± 17.603 msec). There was no significant interaction, F(1, 36) = 2.00, p = .166, ηp² = .05 (Figure 7D). Concerning RTs for original_MT videos, we found no significant effect of the factor Version_fMRI, F(1.59, 57.23) = 0.25, p = .727, ηp² = .01 (Figure 7E).

For novel videos, we found very low false alarm rates (M = .020 ± .017) and a mean RT of 527.041 ± 27.941 msec in the post-fMRI memory test.

In the rating task, participants had to rate how much the storyline of modified episodes deviated from the originally encoded version (from 1 = 0% different to 6 = 100% different). A nonparametric rmANOVA with the factors Modification_fMRI (yes, no) and Version_MT (surface, gist) confirmed a significant influence of Version_MT on story-change ratings in the post-fMRI rating task, F(1, 36) = 347.29, p < .001, ηp² = .91. As expected, surface-modified versions received lower ratings than gist-modified versions (M_surface = 1.363 ± 0.058; M_gist = 4.897 ± 0.093), once again confirming our classification of modifications. Interestingly, there was also a significant effect of Modification_fMRI, F(1, 36) = 4.87, p = .034, ηp² = .12, as modified videos that had already been presented during the fMRI session received lower story-change ratings than those that had been presented in the original version before (M_yes = 3.083 ± 0.066; M_no = 3.177 ± 0.069). There was no significant interaction of both factors, F(1, 36) = 1.791, p = .189, ηp² = .05.

A rmANOVA on RTs in the rating task, that is, how long it took participants to notice the change in modified videos, revealed a significant effect of Version_MT, F(1, 36) = 9.72, p = .004, ηp² = .21. Participants took longer to notice the modification in surface-modified versions than in gist-modified versions (M_surface = 882.181 ± 42.644 msec; M_gist = 798.305 ± 32.695 msec). Furthermore, there was a nonsignificant trend for an effect of Modification_fMRI, F(1, 36) = 3.94, p = .055, ηp² = .10, as participants tended to notice the modification sooner when they had already seen the same modified version during the fMRI session (M_yes = 793.114 ± 26.710 msec; M_no = 887.373 ± 54.061 msec). No significant interaction was found, F(1, 36) = 0.88, p = .356, ηp² = .02.

In the present study, we examined the impact of PEs on episodic memories. To this end, we used modified cues to induce PEs of different episodic relevance, altering the episodic gist or only surface properties. Surface modifications led to significant activation in right IFS. In contrast, modifications that also changed the gist of the episode led to overall more pronounced activation in the same area and additional activity in OTC, IPS, and pMTG. Posterior HPC responded to surface changes more strongly than to gist changes. Interestingly, there was a significant reduction of cingulate activity for gist modifications as compared with original or surface-modified cueing. The day after the fMRI session, we found that a history with surface-modified, but not with gist-modified episodic cues increased the erroneous acceptance of modified stories as originals. Our findings demonstrate that the relevance of mnemonic PEs for the storyline of episodes is crucial for brain responses and influences on memory.

We expected that both types of episodic modifications, characterized by the exchange of only one object, elicit brain activation in areas that we had previously identified for content-based PEs with intermediate episodic relevance (Siestrup et al., 2022). Indeed, gist modifications elicited brain activation in those hypothesized regions, namely, OTC, including FG, posterior IPS, and IFS. This was revealed by whole-brain and ROI analyses. For surface modifications, we found subthreshold whole-brain activation in right IFS, left FG, and right posterior IPS. The ROI analysis revealed significant activation in the right IFS, as well as a trend for activation in OTC for surface modifications. As previously discussed (Siestrup et al., 2022), these brain areas are involved in the processing of object information in actions (El-Sourani et al., 2019; Lingnau & Downing, 2015; Wiggett & Downing, 2011; Reber et al., 2005; Grill-Spector et al., 2001). Furthermore, lateral OTC, which was primarily activated by gist modifications, codes for conceptual action representations (Wurm & Caramazza, 2022). Activation in right IFS, OTC, and IPS ROIs was significantly greater for gist than for surface modifications, as expected because of the gist modifications' comparably high relevance for episode content. In summary, we confirmed our general hypotheses concerning activation typical for PEs elicited by content modifications in an episodic cueing setting.

In addition, we found subthreshold activation clusters for surface modifications in the caudate nucleus, anterior insula, and medial BA 8. These three areas, along with TPJ and IPS, were found to be co-activated when predictions about complex sequences of movements were violated (Schiffer & Schubotz, 2011). Although the activations found in the present study did not remain significant after correction for multiple comparisons, the similarity to this previously identified network is striking. Caudate nucleus is known to respond to breaches of expectation (Schiffer et al., 2012; Schiffer & Schubotz, 2011; Delgado, Li, Schiller, & Phelps, 2008; Haruno & Kawato, 2006), whereas BA 8 and anterior insula activations were shown to reflect unexpectedness and uncertainty (Schiffer, Krause, & Schubotz, 2013; Sarinopoulos et al., 2010; Zaretsky, Mendelsohn, Mintz, & Hendler, 2010; Volz, Schubotz, & Von Cramon, 2003, 2005). In summary, the high degree of similarity between surface-modified episodes and originals might have contributed to increased uncertainty, whereas gist-modified episodes were more clearly distinguished from original ones.

As hypothesized, we found significant qualitative differences in brain activation between gist and surface modifications. For gist compared with surface modifications (and original episodes), there was increased activation in pSTS and pMTG. This finding is especially interesting as we have previously observed these areas to be activated specifically for structural episodic modifications, that is, the exchange of adjacent action steps, but not for the exchange of an object with intermediate episodic relevance (Siestrup et al., 2022), which will be further discussed below. Posterior temporal cortex, as part of the OTC (Wurm & Caramazza, 2022), is crucial for conceptual processing (Martin, 2007) and is thought to represent conceptual object properties (Fairhall & Caramazza, 2013). In the context of such concepts, especially left pMTG is suggested to be important for retrieval of semantic information and the comprehension and recognition of actions (Davey et al., 2016), together with other network components that were identified for gist modifications, namely, STS, IPS, and lateral OTC (James, VanDerKlok, Stevenson, & James, 2011). Davey and colleagues (2016) found that pMTG was functionally connected with IFG and IPS, and proposed that pMTG, together with these regions, is involved in adjusting semantic retrieval to different contexts. Interestingly, pMTG showed increased activation when goal-based expectations were violated (Jastorff, Clavagnier, Gergely, & Orban, 2011), which can also be transferred to prediction violations in the present study. Furthermore, pMTG in conjunction with IFG has been suggested to have a role in the resolution of interfering episodic representations via semantic elaboration (Han, O'Connor, Eslick, & Dobbins, 2012). In that sense, pMTG could subserve the integration of semantic and episodic information, which might be particularly required in the case of gist modifications.

Furthermore, gist modifications yielded higher brain activation in precuneus compared with surface modifications. This area is involved in episodic memory retrieval (Cavanna & Trimble, 2006). Increased activity for gist modifications here might reflect a strong dissociation between the self-experienced episode and the presented version of the episode. Interestingly, we previously identified this area for structure but not for content modifications with intermediate episodic relevance (Siestrup et al., 2022). Aside from its role in the sequential organization of memories, precuneus has been shown to be involved in their contextual reconstruction (Foudil, Kwok, & Macaluso, 2020). Therefore, activation in this area for gist modifications is a hint that the exchange of a gist-relevant object seems to be meaningful not only for episode content but also for structural and contextual (re)evaluations of that memory, specifically with reference to our previous work (Siestrup et al., 2022).

In summary, we saw that gist modifications elicited activation we identified to be specific for content, but also structure-based PEs in episodic memory in our previous work (Siestrup et al., 2022). Schiffer, Ahlheim, and colleagues (2013) showed that adaptation of internal models following PEs occurred in exactly those areas that responded to gist modifications in the present study. In their experiment, episodes were modified by introducing a breach-point from which the action unfolded with new content and structural aspects. Thus, brain responses to Schiffer's gist modifications reflected more than mere object information processing, as the modified aspect was relevant for the integrity of an episode as a whole. Similarly, in the present study, integration of new episodic gist likely requires revision of object information and structure and context of events within an episode as the new storyline unfolds.

We had hypothesized to find increased ACC activation for gist modifications because it has been reported that more gist-like memories depend strongly on ACC (e.g., Sekeres, Winocur, Moscovitch, Anderson et al., 2018). Surprisingly, we observed exactly the opposite: Compared with original and surface-modified episodes, pACC activation, as well as MCC activation, was reduced in gist-modified episodes. Bonasia and colleagues (2018) found a similar response pattern in their study. Participants encoded episodes with a gist that was either congruent or incongruent with previous experiences. With increasing congruence, enhanced encoding-related activation in pACC, PCC, superior temporal gyrus, and AG was observed. This is consistent with our findings for gist-modified episodes and also for novel episodes we presented. For the latter, activation was decreased compared with previously experienced episodes in all four areas mentioned by the researchers. The authors argued that ACC and PCC were likely engaged because of their role during the activation of prior knowledge and semantic processing (Bonasia et al., 2018), thus linking new experience to prior ones. It has been suggested that ACC is responsible for gist processing by providing a template that allows for prediction and interpretation of events (Sekeres, Winocur, & Moscovitch, 2018). In the case of gist modifications in the present study, the gist of the encoded episode was no longer intact, so predictions could potentially no longer rely on the established template. In addition, it is known that ACC is involved in conflict processing (Vassena, Holroyd, & Alexander, 2017; Botvinick, Cohen, & Carter, 2004). In gist-modified episodes, conflict arising because of the mismatch with encoded episodes might be rapidly resolved, as the new gist defines a new episode. This also matches our behavioral findings in the post-fMRI memory test as well as the fact that brain responses to gist modifications strongly decreased over repeated encounters with the same modified episode. Similarly, Webb, Turney, and Dennis (2016) reported that retrieval of non-gist details was characterized by higher (dorsal) ACC activation than retrieval of gist details in a recognition memory task and argued that this finding was because of elevated cognitive efforts and conflict monitoring processes, as mediated by this brain area. In our previous study, pACC activation in response to episodic modifications was increased in the condition that was characterized by strong memory modification effects in the post-fMRI memory test and pACC activation increased over time for later false memories (Siestrup et al., 2023). In the present study, pACC activation was reduced in gist-modified episodes, which did not trigger memory modification in the post-fMRI memory test. Therefore, we suggest that pACC might be involved in memory modification in response to mnemonic PEs. In summary, the relevance of PE within the episode is probably responsible for the impact on memory: When the overall storyline remains intact, a memory might be more likely to be (slightly) modified through PE, whereas a new, distinct episode is established when PE is gist relevant. This idea is similar to a computational approach of memory modification formulated by Gershman, Monfils, Norman, and Niv (2017). They proposed that memories are modified when old and new information are inferred to share the same latent cause, whereas new memories are formed when sensory input is ascribed to a new latent cause.

Surface modifications elicited higher levels of activation in posterior HPC than gist modifications. This is consistent with the suggestion that detailed memory representations depend more on the posterior HPC than general ones do (Sekeres, Winocur, & Moscovitch, 2018; Sekeres, Winocur, Moscovitch, Anderson et al., 2018; Moscovitch et al., 2016; Poppenk et al., 2013). Accordingly, surface modifications likely elicited increased processing of more fine-grained information within modified episodes (Robin & Moscovitch, 2017; Moscovitch et al., 2016). Posterior HPC activation has been linked to episodic retrieval (e.g., Poppenk et al., 2013), which lends further support to our suggestion that surface modifications are processed with close reference to existing memories, whereas gist modifications are rather interpreted as new episodes because of the clear drift in content compared with the original. However, we could not confirm that gist modifications, compared with surface modifications, lead to increased activation in anterior HPC, neither on the whole brain level nor on the basis of ROI analyses. It can be speculated that this null finding might be attributed to the signal-to-noise ratio in this region (Brunec et al., 2018), which might have complicated the detection of condition differences.

Despite the significant interaction between HPC ROI and type of modification, the overall BOLD response in the HPC to episodic PE was unexpectedly low (did not significantly differ from random). This is surprising, because several previous studies showed that HPC responds to mnemonic PEs (e.g., Bein, Duncan, & Davachi, 2020; Long, Lee, & Kuhl, 2016; Duncan, Ketz, Inati, & Davachi, 2012). It is not clear why we could not replicate these findings, but it is possible that univariate analysis approaches cannot fully capture the complexity of hippocampal responses to mnemonic PEs (e.g., Sinclair, Manalili, Brunec, Adcock, & Barense, 2021) or that hippocampal mismatch responses were strongest on the first trial with a modified cue and decreased afterward. Furthermore, it has been shown that the magnitude of hippocampal responses to PE is positively related to the number of changes in episodes (Duncan et al., 2012). Our findings provide the first evidence that the HPC is sensitive to not only quantitative differences in PE strength but also qualitative ones, as implemented through surface and gist modifications. Furthermore, future work could focus on the influence of surface and gist modification on representations in posterior and anterior HPC, respectively, to shed more light on processes of pattern separation and integration in episodic memory (Brunec, Robin, Olsen, Moscovitch, & Barense, 2020).

We replicated our finding that the repeated experience of modified episodes during the fMRI session influences memory performance in a post-fMRI memory test. Importantly, in the current study, we found this effect only for surface- but not for gist-modified episodes. Specifically, previous encounters with surface-modified videos increased the participants' tendency to erroneously accept modified versions as truly encoded, as reflected by reduced AUC values and increased false alarm rates. Hit rates were not influenced and remained at a high level even after the repeated exposure to modified episodes, similar to what we showed before (Siestrup et al., 2023, 2022; Jainta et al., 2022). This finding is in line with reports that the inclusion of new information into an old memory is not necessarily accompanied by a deterioration of the original memory (Sinclair & Barense, 2018; St. Jacques, Olm, & Schacter, 2013). Together, the pattern of results in the post-fMRI memory test suggests that memory modification occurred because of the encoding of an additional memory trace that interfered with the original one, but not a replacement of original memory content (Siestrup et al., 2023). We found the memory modification effect only for surface but not for gist modifications, which was expected because of their differential relevance for the episodes. In line with that, participants were also faster to detect gist than surface modification in the rating task, that is, when explicitly asked to rate how much the storyline of modified episodes deviated from the originally encoded version. It has previously been demonstrated that the experience of episode modifications can also improve memory, but only when such modifications are detected and remembered by participants (Wahlheim & Zacks, 2019). Furthermore, it is known that distinct items, that is, those that violate the current context, are remembered better and that such distinctiveness reduced false memories (Sommer & Sander, 2022; Hunt, 2013). The fact that gist modifications effectively constituted new episodes likely contributed to distinctiveness between original and modified versions, explaining the lack of confusion between them and, ultimately, the low rate of false alarms. This distinctiveness is probably also relevant for another finding from the post-fMRI memory test: Memory performance was generally worse for episodes with a surface modification compared with those with a gist modification, irrespective of any previous experience with modified videos during scanning. This was evidenced by reduced AUC values and increased false alarm rates in the surface modification condition. Therefore, it was more difficult for participants to correctly differentiate modified from original versions when the modification was not gist relevant. This observation is in agreement with reports that noncentral details are less likely to be remembered (Sekeres et al., 2016) and can be easily confused by misinformation (Reyna et al., 2016; Reyna & Lloyd, 1997).

In addition, when modified episodes had been presented in the scanner, participants were slower to correctly judge the same as not part of the original episode repertoire in the post-fMRI memory test. Longer RTs in memory tasks can be indicative of higher cognitive processing demands during retrieval (Noppeney & Price, 2004; Larsen & Plunkett, 1987). Potentially, it became more difficult for participants to correctly reject modified versions because of the encoding of alternative episode versions.

In a previous study, we found an increase in activity in the episodic memory network for episodes that later lead to false memories (Siestrup et al., 2023). In the present study, it was not possibly to model brain responses as a function of subsequent behavioral outcomes in the post-fMRI test. This was because the total number of false alarms was too low, and, as expected, they were correlated with the surface type of cue manipulation. Thus, low statistical power and expected confound prevented us from testing a replication of our earlier findings.

Overall, behavioral evidence we collected so far suggests that based on mnemonic PEs, alternative episode representations are encoded, which then stand in conflict to the originally encoded ones. In the case of surface-level PEs, these alternative representations interfered, leading to false memories, whereas in the case of gist level PEs, the two alternatives were clearly separated.

Our results indicate that the relevance of mnemonic PEs within episodes shapes neural responses and memory performance. Although both gist and surface modifications of episodic cues were characterized by the exchange of only one object within an episode, their implications were highly specific: Surface-level PE increased uncertainty and conflict between alternative episode representations, whereas gist-level PE induced the encoding of a new, distinct episode not conflicting with the original one. An important implication of our findings is that not all types of mnemonic PE influence memory in the same way, which needs to be considered in future research.

To investigate whether brain responses to episodic modification attenuate with repeated presentation of the same modified episodes, we additionally investigated the parametric contrasts for surface and gist modifications. For gist modifications, we found significant attenuation in several areas that responded to modifications (Table A1). These included bilateral IPS, IFS/IFJ, FG, and inferior temporal regions. Furthermore, attenuation was found in bilateral parahippocampal gyrus (PHG), retrosplenial cortex, and right IFG (Table A1). For surface modifications, we did not detect significant attenuation effects with correction for multiple comparisons. For completeness, we investigated the parametric contrast at a subthreshold level (uncorrected, p < .001) and found attenuation in the right IPS and retrosplenial cortex and bilateral FG and PHG (Table A1).

In addition to our ROI analysis in anterior and posterior HPC, we conducted an explorative analysis using a more fine-grained parcellation of HPC. To this end, anatomical ROIs were created from probabilistic maps from the Julich-Brain Cytoarchitectonic Atlas (Amunts, Mohlberg, Bludau, & Zilles, 2020). We used maps of hippocampal subfields CA1 (Palomero-Gallagher, Kedo, Mohlberg, Zilles, & Amunts, 2020a, 2020b), CA2 (Palomero-Gallagher et al., 2020b; Palomero-Gallagher, Kedo, Mohlberg, Zilles, & Amunts, 2020c), and CA3 (Palomero-Gallagher et al., 2020b; Palomero-Gallagher, Kedo, Mohlberg, Zilles, & Amunts, 2020d). A threshold of 0.3 was implemented in ImCalc in SPM12, and maps were divided into anterior and posterior portions as described above. Contrast estimates were extracted as described above from surface > original and gist > original contrasts. A nonparametric rmANOVA (chosen because of missing normality and extreme outliers) with the factors Modification (surface, gist), ROI (CA1, CA2, CA3), and Location (anterior, posterior) revealed no significant main effects, Modification: F(1, 36) = 2.04, p = .162, ηp² = .05; ROI: F(2, 72) = 1.04, p = .357, ηp² = .03; Location: F(1, 36) = 1.82, p = .186, ηp² = .05. There was a nonsignificant trend for an interaction of Modification and ROI, F(2, 72) = 2.48, p = .091, ηp² = .06. Furthermore, we found a significant interaction of Modification and Location, F(1, 36) = 4.68, p = .037, ηp² = .12. Paired Wilcoxon tests according to our hypotheses revealed that in posterior locations, surface modifications elicited significantly more activation than gist modifications (z = −2.56, p = .02 [one-tailed]; M_{surface-posterior} = 0.077 ± 0.035; M_{gist-posterior} = −0.018 ± 0.045). There was no difference between modification types in anterior locations (z = −0.53, p = 1 [one-tailed]; M_{surface-anterior} = −0.006 ± 0.043; M_{gist-anterior} = −0.002 ± 0.055). There were no further interaction effects (Figure B1). Please note that no further HPC subfields (dentate gyrus, subiculum) were analyzed because of signal loss in the anterior portion of these ROIs.

The authors thank Monika Mertens, Alea Bexten, and Lana Steuernagel for their help during data collection. We thank Benjamin Jainta for helpful comments on an earlier draft of this article and members of Research Unit FOR 2812 for valuable discussions.

Reprint requests should be sent to Sophie Siestrup, University of Münster, Fliednerstraße 21, 48149 Münster, Germany, or via e-mail: s.siestrup@uni-muenster.de.

The data sets generated for this study are available on request to the corresponding author.

Sophie Siestrup: Conceptualization; Formal analysis; Investigation; Methodology; Visualization; Writing—Original draft; Writing—Review & editing. Ricarda I. Schubotz: Conceptualization; Funding acquisition; Methodology; Resources; Supervision; Writing—Original draft; Writing—Review & editing.

This work was funded by the German Research Foundation (Deutsche Forschungsgemeinschaft; https://dx.doi.org/10.13039/501100001659), grant number: 419037023. The funder had no role in study design, data collection, analysis and interpretation, decision to publish, or writing of the report.

Retrospective analysis of the citations in every article published in this journal from 2010 to 2021 reveals a persistent pattern of gender imbalance: Although the proportions of authorship teams (categorized by estimated gender identification of first author/last author) publishing in the Journal of Cognitive Neuroscience (JoCN) during this period were M(an)/M = .407, W(oman)/M = .32, M/W = .115, and W/W = .159, the comparable proportions for the articles that these authorship teams cited were M/M = .549, W/M = .257, M/W = .109, and W/W = .085 (Postle and Fulvio, JoCN, 34:1, pp. 1–3). Consequently, JoCN encourages all authors to consider gender balance explicitly when selecting which articles to cite and gives them the opportunity to report their article's gender citation balance.