Abstract

Disrupting the configural context, or relative organization and orientation of paired stimuli, between encoding and retrieval negatively impacts memory. Using univariate and multivariate fMRI analyses, we examined the effect of retaining and manipulating the configural context on neural mechanisms supporting associative retrieval. Behavioral results showed participants had significantly higher hit rates for recollecting pairs in a contextually congruent, versus incongruent, configuration. In addition, contextual congruency between memory phases was a critical determinant to characterizing both the magnitude and patterns of neural activation within visual and parietal cortices. Regions within visual cortices also exhibited higher correlations between patterns of activity at encoding and retrieval when configural context was congruent across memory phases than incongruent. Collectively, these findings shed light on how manipulating configural context between encoding and retrieval affects associative recognition, with changes in the configural context leading to reductions in information transfer and increases in task difficulty.

INTRODUCTION

With respect to memory, context has often been defined as background information that is irrelevant, incidental, or peripheral to the cognitive task being performed (Hayes, Nadel, & Ryan, 2007; Murnane, Phelps, & Malmberg, 1999). Researchers have characterized context in many different ways, including the background color behind an item (Mori & Graf, 1996; Dulsky, 1935), word font style, item color, item orientation (Murnane & Phelps, 1993, 1994, 1995; Graf & Ryan, 1990), and item location on a screen (Murnane & Phelps, 1995). Decades of memory research have shown that item retrieval is more difficult when contextual information shifts such that the congruency between study and test becomes disrupted (Gruppuso, Lindsay, & Masson, 2007; Hayes et al., 2007; McKenzie & Tiberghien, 2004; Macken, 2002; Murnane et al., 1999; Smith, 1979, 1982, 1988; Eich, 1985; Nixon & Kanak, 1981; Mandler, 1980; Godden & Baddeley, 1975; DaPolito, Barker, & Wiant, 1972; Thomson, 1972; Thomson & Tulving, 1970). For example, researchers have found a decrease in memory for the item itself when contextual details such as those described previously (e.g., background color behind an item, item color, location of information on a screen) change between encoding and retrieval phases, compared to when contextual information is congruent between the two memory phases (Smith, 1988; Eich, 1985; Godden & Baddeley, 1975). This disruption of context is also associated with slower RTs (Criss, 2010; Ratcliff & McKoon, 2008), even when the items are accurately remembered (Overman, McCormick-Huhn, Dennis, Salerno, & Giglio, 2018).

A critical aspect of item retrieval difficulty in the presence of a changed context is the fact that the item is not encoded and stored in isolation. Rather, it is often incidentally bound to the context irrespective of any instructions to do so. For example, Nakashima and Yokosawa (2011) found that, when participants were instructed to encode an item placed in the foreground of a scene, memory was poorer for foreground items when the background was removed at the time of test compared to when the background was reinstated at test, suggesting that scene was incidentally encoded alongside the item. Hayes, Ryan, Schnyer, and Nadel (2004) and Hayes et al. (2007) found a similar decrement in memory for item retrieval, also showing that item memory associated with a change in background color at the time of test. Item retrieval absent of the original context in this study was associated with more right-lateralized activity in the parahippocampal cortex (PHC), compared to bilateral activity in the same region when the background remained consistent between encoding and retrieval (Hayes et al., 2004, 2007). The authors interpret this increase in PHC activation in the absence of scenes as evidence that the scenes are being retrieved during item recognition, indicating that they were implicitly bound to the items during encoding (Epstein & Ward, 2010; Hayes et al., 2007; Goh et al., 2004). Thus, past research suggests that reinstatement of encoding background, or context, facilitates item retrieval, and when context is removed, individuals may attempt to retrieve it to support item retrieval.

It has been posited that context reinstatement improves item retrieval because context cues facilitate recollection, a more deliberate and detailed retrieval process than familiarity (Mandler, 1980; DaPolito et al., 1972). Mandler's (1980) classic “butcher on the bus” example illustrates this principle by noting a feeling of familiarity may arise when an item, such as one's butcher, is unexpectedly encountered in an unusual context (the bus), but recollection of the individual's identity is hindered when the retrieval context is different than the encoding context. In another example, an empirical study of this phenomenon found that, for face–context pairs, switching contexts at test significantly reduced recollection of the face while leaving item-based familiarity intact (Gruppuso et al., 2007). This effect is seen across verbal (McKenzie & Tiberghien, 2004; Thomson, 1972) and visual (Tibon, Vakil, Goldstein, & Levy, 2012; Bar, 2004; Bar & Aminoff, 2003) stimuli. Thus, the absence of contextual information negatively impacts the ability of the retrieval cue to support successful recollection (Tiberghien & Cauzinille, 1979).

Associative memory shows similar declines in accuracy and increases in RT to that of item memory when contextual shifts occur between encoding and retrieval phases, such as when the location and relative positioning of items that comprise the pair are changed between encoding and retrieval (e.g., De Brigard, Langella, Stanley, Castel, & Giovanello, 2020; Overman et al., 2018; Giovanello, Schnyer, & Verfaellie, 2009). Whereas context can be defined as background when referring to item memory, in associative memory, context can refer to the way individual parts of the associative pair are organized with respect to one another. We label the relative organization and orientation of paired stimuli on a computer screen as configural context. When there is a disruption to configural context such that information is shifted from its original presentation format during encoding to a different format at retrieval, memory itself is disrupted (De Brigard et al., 2020; Overman et al., 2018; Siegel & Castel, 2018; Giovanello et al., 2009; Rhodes, Castel, & Jacoby, 2008; Naveh-Benjamin, Guez, Kilb, & Reedy, 2004). For example, in Overman et al. (2018), a change in the configural context (see Figure 1) of face–scene pairs from encoding to retrieval significantly reduced associative hit rates and led to longer RTs. This reduction in behavior occurred although the configural context of the face–scene pairs between encoding and retrieval was irrelevant to the memory decision regarding face–scene associations. Similar results have been observed when the ordering of word pairs or face pairs incurs a similar configural shift between encoding and retrieval (Giovanello et al., 2009; Rhodes et al., 2008). Like the item memory studies discussed above, these findings imply that, in associative memory tasks, a lack of encoding–retrieval congruency of the configural context can negatively impact memory across various stimuli types and complexities.

Figure 1. 

Depiction of task showing both encoding conditions (side-by-side and superimposed) and retrieval conditions as a function of configural context congruency. Encoding trials displayed the question “How welcoming is the face and scene?” and the four response choices below the face–scene pair on every trial. Retrieval trials displayed the text “Please identify the pairings that have been presented previously.” on every trial above the three responses choices: Remember, Know, and New.

Figure 1. 

Depiction of task showing both encoding conditions (side-by-side and superimposed) and retrieval conditions as a function of configural context congruency. Encoding trials displayed the question “How welcoming is the face and scene?” and the four response choices below the face–scene pair on every trial. Retrieval trials displayed the text “Please identify the pairings that have been presented previously.” on every trial above the three responses choices: Remember, Know, and New.

With respect to the neural correlates underlying contextual changes in associative memory, studies (Giovanello et al., 2009; Hayes et al., 2007; Pihlajamäki et al., 2004) have shown that shifts in the configural context from encoding to retrieval have an impact on activation within the medial temporal lobe (MTL), a region critical to processing associative memory (Achim, Bertrand, Montoya, Malla, & Lepage, 2007; Mayes, Montaldi, & Migo, 2007). For example, simple inversion of word pairs between encoding and retrieval (e.g., a switch from clock–river to river–clock) was found to lead to differences in MTL recruitment during retrieval. Specifically, reversed pairs were associated with greater anterior hippocampal activity, whereas activity for intact pairs was greater in posterior hippocampus (Giovanello et al., 2009). The authors interpreted this finding as suggesting that activity in the posterior hippocampus reflects processing of the exact reinstatement of a study episode, whereas activity in the anterior hippocampus reflects the flexible retrieval of learned associations. Although this study is important in demonstrating the relationship between neural activity at retrieval and behavioral accuracy when manipulating verbal associations, additional replication and expansion are warranted to investigate whether a similar manipulation using complex visual stimuli would result in a similar dissociation within the MTL as well as differences across the larger retrieval network.

Behavioral consequences of shifts in the configural context (e.g., reduced hit rates and increased RTs) suggest that such retrieval is a more difficult retrieval process. Difficulty with respect to memory retrieval has been associated with increased activity across the retrieval network, including increased activity in the medial and lateral pFC, lateral and medial MTL (hippocampus, PHC, and perirhinal cortex [PrC]), ventral parietal cortex, and posterior cingulate cortex (McDermott, Szpunar, & Christ, 2009; Spreng, Mar, & Kim, 2009; Cabeza & St Jacques, 2007; Svoboda, McKinnon, & Levine, 2006). Upregulation of these regions has been interpreted as a need to increase retrieval monitoring and evaluation processes associated with memory decisions of greater complexity. Similar increases in activity within these regions have been observed for successful recollection (Poppenk, McIntosh, Craik, & Moscovitch, 2010; Achim et al., 2007; Johnson & Rugg, 2007; Mayes et al., 2007; Vilberg & Rugg, 2007; Von Zerssen, Mecklinger, Opitz, & Von Cramon, 2001; McDermott, Jones, Petersen, Lageman, & Roediger, 2000; Schacter, Verfaellie, & Pradere, 1996), which itself is a difficult retrieval process that relies on a great deal of monitoring, decision-making, and evaluation (Philiastides, Ratcliff, & Sajda, 2006; Egner, Jamieson, & Gruzelier, 2005; Fellows & Farah, 2005; Ranganath & Paller, 2000).

The benefits of contextual congruency at retrieval are highlighted by the principles of transfer appropriate processing (TAP) and cortical reinstatement. On the basis of the idea that encoding and retrieval processes are interdependent, it has long been suggested that successful memory is associated with the recapitulation of cognitive processes between encoding and retrieval (Ritchey, Wing, LaBar, & Cabeza, 2013; Gisquet-Verrier & Riccio, 2012; Rugg, Johnson, Park, & Uncapher, 2008; Rasch & Born, 2007; Vaidya, Zhao, Desmond, & Gabrieli, 2002; Morris, Bransford, & Franks, 1977; Tulving & Thomson, 1973). In the model put forth by Norman and O'Reilly (2003), not only was encoding–retrieval overlap in activation critical to memory success, but so was the reactivation of encoded representations, reflected in the reinstatement of the pattern of cortical activity encoded at retrieval. The principles of this theory can be tested using encoding–retrieval similarity (ERS) analysis, which evaluates the neural pattern similarity between individual trials across encoding and retrieval (Ranganath & Ritchey, 2012; Rugg et al., 2008; Cabeza, Dolcos, Graham, & Nyberg, 2002; Nairne, 2002). For example, using ERS, Ritchey et al. (2013) found that overlap in neural patterns within middle occipital gyrus, middle temporal gyrus, and supramarginal gyrus was associated with memory success, suggesting that retrieval-related recapitulation of neural patterns elicited at the time of encoding benefits memory. The TAP theory suggests that this reinstatement should be greater when the configural context is reinstated at retrieval; however, it is unclear how manipulations of contextual congruency affect cortical reinstatement. Like ERS analysis, multivoxel pattern analysis (MVPA) has been able to examine neural processes at the level of cortical patterns of activation. With respect to context processing, MVPAs have been able to discriminate differences in stimulus orientation in visual processing regions (Harrison & Tong, 2009), memories of context locations in parahippocampal gyrus and posterior cingulate (Polyn, Natu, Cohen, & Norman, 2005), and perceptually distinct targets and lures in midline occipital cortex (Bowman, Chamberlain, & Dennis, 2019). The application of MVPA to manipulations of contextual configurations at retrieval is useful for elucidating how context congruency affects neural patterns of activation and, in turn, how these patterns support successful memory.

The current study aims to extend our original work with regard to the role of configural context in associative memory retrieval (Overman et al., 2018) and broadens that of Giovanello et al. (2009) to better understanding the neural signature of contextual reinstatement when manipulating the configural context of complex object pairs from encoding to retrieval. We first aim to replicate previous findings in both the item and associative memory fields, showing that disruptions of context have a detrimental effect on memory retrieval. With respect to the neural basis of associative retrieval across congruent and incongruent configural contexts, we hypothesize that, in line with Giovanello et al. (2009), congruent retrieval will be associated with greater activation in posterior hippocampus, because of the fact that this region has been shown to be involved with processing the exact reinstatement of a study episode irrespective of memory decision (Giovanello et al., 2009). By comparison, we predict viewing pairs in an incongruent configural context will exhibit increased activation in anterior hippocampus (Giovanello et al., 2009). On the basis of the position that incongruent retrieval is a more difficult memory process, we also posit that, compared to congruent retrieval, incongruent retrieval will be associated with increased activity in the anterior cingulate, right inferior frontal cortex, and visual processing regions (Gould, Brown, Owen, Ffytche, & Howard, 2003). If congruency at retrieval is facilitated by the fact that the original encoding context is recapitulated at retrieval, we predict that congruent and incongruent targets should be discriminable with respect to neural patterns representing both types of associations. Furthermore, we would predict greater ERS in neural patterns for congruent compared to incongruent associations across the retrieval network.

METHODS

Participants

Thirty-one right-handed adults were recruited from The Pennsylvania State University community and received $25 for their participation. Participants were screened for history of psychiatric and neurological illness, head injury, stroke, learning disability, medication that affects cognitive and physiological function, and substance abuse. On the day of the study, all participants provided written informed consent for a protocol approved by The Pennsylvania State University institutional review board. All participants were native English speakers with normal or corrected-to-normal vision. All participants were enrolled in college or postgraduate education. Three participants were removed from the study because of claustrophobia issues during scanning; one because of excess movement in the scanner; and one because of misunderstanding the paradigm. Thus, the reported results are based on data from 26 participants (19 women; age: M = 20.5 years, SD = 1.95 years).

Stimuli

Stimuli consisted of 170 color photographs of faces and 170 color photographs of scenes paired together. Face stimuli consisted of both male and females faces, each exhibiting a neutral expression, taken from the following online databases: the Color FERET Database (Phillips, Moon, Rizvi, & Rauss, 2000), adult face database from Dr. Denise Park's laboratory (Minear & Park, 2004), the AR Face Database (Martinez & Benavente, 1998), and the FRI CVL face database (Solina, Peer, Batagelj, Juvan, & Kovač, 2003). Scene stimuli consisted of outdoor and indoor scenes collected from an Internet image search. Using Adobe Photoshop CS2 Version 9.0.2 and Irfanview 4.0 (www.irfanview.com/), we edited face stimuli to a uniform size (320 × 240 pixels) and background (black), and scene stimuli were standardized to 576 × 432 pixels.

During encoding, half of the face–scene pairs were presented side by side; and the other half, with the face superimposed on top of the scene (see Figure 1). At retrieval, half of the pairs were present in a manner congruent with their original contextual configuration; and the other half, in the opposite configuration (i.e., incongruent contextual configuration). Each encoding and retrieval block consisted of 36 total pairs. Within the 36 pairs per retrieval block, 10 were lures (five of which were side-by-side configured and five of which were superimposed configured) in which the face was rearranged with a different scene than was originally presented at encoding. During encoding, pairs were presented for 4 sec and for 4 sec at retrieval. A jittered ISI (2–8 sec) separated the presentation of each image. Each encoding and retrieval block lasted 4 min 18 sec. A second version of this task was created to counterbalance the design such that the same stimuli were presented in the opposite configuration across versions (e.g., a face–scene pair presented side by side in Version A was presented as superimposed in Version B).

Procedure

The encoding phase of this experiment was reported in Dennis et al. (2019). Before scanning, all participants practiced several trials of encoding and retrieval. Participants were encouraged to ask questions during this time. The scanning session began with a structural scan (magnetization prepared rapid gradient echo) that took approximately 7 min. During this time, the participants were asked to remain as still as possible. After the structural scan, participants completed five encoding and five retrieval blocks presented in an alternating order. Instruction screens were presented before each block reiterating the verbal instructions participants received before entering the scanner. Presentation of all instruction screens was self-paced, meaning that the participants pressed “1” on the handheld button box to advance to the next screen when they read the instructions and were ready to begin the task. When the instruction slides appeared on the screen, the participant was asked to explain the instructions verbally before proceeding with the experiment to verify an accurate understanding of the task.

After advancing past the instruction slides in encoding, the participant was presented with a series of face and scene pairings displayed on the screen in either an item–item or an item–context configuration. Each pair was presented for 4 sec, during which time the participant responded to the question “How welcoming are the scene and face?” presented in text below each pair by utilizing a rating scale from 1 to 4 (1 = not at all, 4 = very) and making a keypress on their handheld button box. The instructions emphasized that the participants should choose the rating based on how welcoming the face and scene pairing was together to facilitate encoding of both the face and the scene, rather than only one or the other. This question also helped ensure that participants paid attention to the scene, even when it was configured behind the face in the item–context condition because we did not want the scene to be incidentally encoded while the face was intentionally encoded. Across versions, faces and scenes were counterbalanced for their inclusions in either an item–item or an item–context pair. No differences across versions were noted, and all analyses were collapsed across versions.

Each encoding block was followed by a retrieval block. Similar to encoding, each face–scene pair at retrieval was presented for 4 sec. During this time, participants were asked to respond to the question: “Please identify the pairings that have been presented previously.” Displayed below the question were the following choices: 1 = remember, 2 = know, and 3 = new. A remember–know–new design was chosen to isolate recollection-related activity, associated with “remember” responses, from that of familiarity, associated with “know” responses. This distinction has shown to be critical when assessing memory-related activity particularly within the MTL (Yonelinas et al., 2007; Yonelinas, Otten, Shaw, & Rugg, 2005; Yonelinas, 2002). Participants were instructed to make their memory judgments based on the co-occurrence of the face and scene and not to base their judgments on the configuration of the display. Specifically, participants were instructed to select “Remember” if they were able to retrieve details of the face and scene appearing together at study, select “Know” if they thought the face and scene appeared together previously but they could not remember specific details about the original appearance, and select “New” if they believe that the face–scene pair appeared together previously. Thus, whether a retrieval configuration was congruent or incongruent, the Remember/Know/New labels applied equally to both types of configurations, with a participant's response simply dependent on the vividness of their memory. Similar to encoding, all responses were made using the button box (only retrieval data are analyzed in the current analysis). With respect to retrieval configurations, half of the trials were congruent with their encoding configuration and half were incongruent (such that a side-by-side trial at encoding was presented as a superimposed trial at retrieval).

Image Acquisition

Structural and functional images were acquired using a Siemens 3-T scanner equipped with a 12-channel head coil, parallel to the AC–PC plane. Structural images were acquired with a 1650-msec repetition time, a 2.03-msec echo time, a 256-mm field of view, a 2562 matrix, 160 axial slices, and a 1.0-mm slice thickness for each participant. Echo-planar functional images were acquired using a descending acquisition, a 2500-msec repetition time, a 25-msec echo time, a 240-mm field of view, a 802 matrix, a 90° flip angle, and 42 axial slices with a 3.0-mm slice thickness resulting in 3.0-mm isotropic voxels.

Image Processing

For univariate analyses, raw anatomical and functional images were first skull stripped using the Brain Extraction Tool (Smith, 2002) in the FMRIB Software Library (FSL) Version 5.0.10 (www.fmrib.ox.ac.uk/fsl). FSL's MCFLIRT function (Jenkinson, Bannister, Brady, & Smith, 2002) was then applied for realignment and motion correction within each functional run. All volumes were aligned to the middle volume of the middle run of encoding. The realigned functional images were then processed by FSL's fMRI Expert Analysis Tool (Woolrich, Ripley, Brady, & Smith, 2001), where they were high-passed filtered and spatially smoothed at 6-mm FWHM. These data were then prewhitened to account for temporal autocorrelations within voxels. Finally, the structural data underwent nonlinear transformation into the standardized Montreal Neurological Institute (MNI) space by using the warping function in FSL's FNIRT (Andersson, Jenkinson, & Smith, 2010). For multivariate analyses, the raw data underwent the exact same steps as above, absent smoothing.

Behavioral Analyses

All behavioral analyses were conducted in RStudio (www.rstudio.com/; RStudio Team, 2018). Because this study aimed to examine pure associative memory, behavioral and neuroimaging analyses were focused on accurately recollected memory decisions. To test for congruency-related differences in memory, recollection values were entered into a paired-samples t test. RTs for configurally congruent and incongruent recollection were subjected to t tests to examine whether differences in visual presentation affect processing speed. Last, RTs were correlated against recollection rates to examine whether there is a speed–accuracy trade-off.

Univariate Analyses

At the first level, trial-related activity was modeled in SPM12 using two general linear models (GLMs) with a stick function corresponding to trial onset convolved with a canonical hemodynamic response function. To address our first aim, namely, to better understand the neural signature of contextual reinstatement when manipulating the configural presentation of complex object pairs from encoding to retrieval, two second-level random effects GLMs were created, and one-sample t tests were conducted to investigate contrasts of interest. The first model focused on two trial types of interest: (1) congruent targets, which were defined as any target trial presented in a congruent visual presentation from encoding, and (2) incongruent targets, which were defined as any target trial presented in an incongruent visual presentation from encoding. Configurally congruent and incongruent targets contained two regressors that were combined at the contrast stage: side-by-side congruent and superimposed congruent for the former, and side-by-side incongruent and superimposed incongruent for the latter. All lures were coded as a regressor of no interest.

The second model focused on four trial types of interest: (1) congruent recollection, which were defined as “Remembered” trials that were configurally congruent with encoding; (2) incongruent recollection, which were defined as “Remembered” trials that were configurally incongruent from encoding; (3) combined familiar, which were defined as targets in which participants accurately responded “Know”; and (4) combined miss, which were defined as targets that participants misidentified as “New.” Similar to the above model, congruent and incongruent recollection contained two regressors: side-by-side congruent and superimposed congruent for the former, and side-by-side incongruent and superimposed incongruent for the latter. With respect to Trial Types 3 and 4, low trial counts in the “Know” responses precluded us from modeling, and examining, familiarity within each visual condition. Thus, the decision was made to isolate recollection-related activity for each condition of interest and combine Know and New responses within each condition to create an “Other” regressor for contrasting recollection in obtaining recollection success effects. All other trial types, along with no response trials, were coded together as a regressor of no interest.

On the basis of our a priori hypotheses regarding the role of MTL subregions in processing congruent and incongruent trials, we investigated all univariate and multivariate effects within the bilateral hippocampus, PHC, and PrC. The hippocampus and PHC masks were derived from the AAL PickAtlas (Lancaster et al., 2000), and the bilateral PrC mask was taken from Holdstock, Hocking, Notley, Devlin, and Price (2009).

For all univariate contrasts, we employed Monte Carlo simulations as implemented by 3dClustSim in AFNI Version 16.0 (Cox & Hyde, 1997), to determine activation that was corrected for multiple comparisons at p < .05, using an uncorrected p threshold (p < .005). An additional simulation was run to determine a correction specific to the MTL.

Multivariate Pattern Classification and Analysis

To estimate neural activity associated with individual trials, an additional GLM was estimated in SPM12 defining one regressor for each trial at retrieval (170 in total). An additional six nuisance regressors were included in each run corresponding to motion. Whole-brain beta parameter maps were generated for each trial at retrieval for each participant. In a given parameter map, the value in each voxel represents the regression coefficient for that trial's regressor in a multiple regression containing all other trials in the run and the motion parameters. These beta parameter maps were next concatenated across runs and submitted to the CoSMoMVPA toolbox (Oosterhof, Connolly, & Haxby, 2016) for pattern classification analyses. Given our interest in determining which region in the associative retrieval network discriminated between congruent and incongruent targets, separate classification accuracies were computed in ROIs previously identified as critical to associative memory retrieval. These include separate predictions within the aforementioned regions in the MTL (bilateral hippocampus, PHC, and PrC), pFC, posterior parietal cortex, early visual cortex, and late visual cortex. Regions that comprised pFC were first identified from meta-analyses of memory, including inferior, medial, and middle frontal gyri (Maillet & Rajah, 2014; Kim, 2011). These regions were then identified in AAL PickAtlas using anatomically defined boundaries identified by the anatomical labeling to create a single ROI for pFC. Because of our previous work indicating that encoding configuration significantly influences patterns of neural activity in the inferior occipital cortex (IOC; Brodmann's area [BA] 17/18) and middle occipital cortex (MOC; BA 19; Dennis et al., 2019), these regions were also selected using the AAL PickAtlas. Finally, parts of the posterior parietal cortex, including BAs 5 and 7, were chosen based on previous work that showed consistent involvement of these regions in memory retrieval (Sestieri, Shulman, & Corbetta, 2017; Davis, Dennis, Daselaar, Fleck, & Cabeza, 2008; Wagner, Shannon, Kahn, & Buckner, 2005). Along with this, recent MVPA studies suggest that regions within the posterior parietal cortex, which includes the angular gyrus, BA 5, and BA 7 (Brodt et al., 2018; Sestieri et al., 2017), can represent content-specific episodic information during retrieval (Kuhl & Chun, 2014; Xue et al., 2013). We therefore created a posterior parietal mask using bilateral angular gyrus, BA 5, and BA 7 labels again using the AAL PickAtlas.

Classification analyses were computed for retrieval runs using a support vector machine classifier with a linear kernel using all voxels within an ROI (Mumford, Turner, Ashby, & Poldrack, 2012). Training and testing followed an n − 1 cross-validation procedure with four runs used as training data and one run used as testing data. Group-level results were generated from averaging across validation folds from all possible train-data/test-data permutations from the individual participant level. We first tested whether a classifier was significantly able to discriminate between all configurally congruent and incongruent targets above theoretical chance (two trial types; 50%) using a one-tailed one-sample t test for classification accuracy within each ROI. We also ran a separate analysis that tested whether a classifier was accurately able to discriminate above theoretical chance between targets presented across the four configural trial types (side-by-side congruent, side-by-side incongruent, superimposed congruent, and superimposed incongruent; 25%) using a one-tailed one-sample t test for classification accuracy within each ROI. Last, to assess whether classifier accuracy was related to behavioral performance, separate repeated-measures ANOVAs were computed for each ROI that contained significant classification with overall recollection rates as the dependent measure.

All significant findings were confirmed using permutation testing to correct for the occurrence of false positives (Etzel & Braver, 2013; Stelzer, Chen, & Turner, 2013; Gaonkar & Davatzikos, 2012; Kriegeskorte, Goebel, & Bandettini, 2006). Specifically, we ran a follow-up test that repeatedly randomized the retrieval labels and reran the classification analysis on the permuted data. This was done 1000 times for each significant region to produce a null distribution that simulates the potential accuracy scores that could be obtained if the retrieval manipulation had no effect (Dennis et al., 2019).

ERS Analysis

To examine whether shifts in the configural context had an effect on the similarity of target representations between encoding and retrieval, we directly compared neural patterns of activation between encoding and retrieval across configurally congruent and incongruent conditions. On the basis of our previous work indicating that encoding configuration (i.e., side-by-side, superimposed) significantly influences patterns of neural activity during encoding (Dennis et al., 2019), we separated all targets into four conditions of interest: side-by-side congruent targets, superimposed congruent targets, side-by-side incongruent targets, and superimposed incongruent targets. Activation for each individual trial for a given condition at encoding was correlated with every trial of the same type at retrieval (e.g., targets at encoding that were presented as a side-by-side configuration that were congruently re-presented as side-by-side at retrieval). This resulted in similarity scores, as operationalized by Pearson's r correlation values, for each trial. The correlations were then averaged within condition for each participant. Group-level results were generated from averaging within condition across all participants.

RESULTS

Behavioral

A paired-samples t test comparing the effect of retrieval presentation on recollection rates revealed that participants recollected significantly more configurally congruent pairs (Mrecollection = 68.53%, SDrecollection = 13.30%) than incongruent pairs (Mrecollection = 54.43%, SDrecollection = 15.76%), t(25) = 6.34, p < .001, 95% CI [0.10, 0.19]. The same pattern of results emerged for a t test comparing corrected recollection rates (recollection rate − false alarm rates), such that the corrected recollection rates were significantly higher for configurally congruent pairs (McorrRecollection = 61.54%, SDcorrRecollection = 13.63%) than incongruent pairs (McorrRecollection = 50.00%, SDcorrRecollection = 15.14%), t(25) = 5.25, p < .001, 95% CI [0.07, 0.16]. Another paired-samples t test revealed participants false alarmed to congruently presented pairs (Mrecollection = 18.20%, SDrecollection = 8.46%) more often than incongruently presented pairs (Mrecollection = 14.18%, SDrecollection = 7.91%), t(25) = 4.26, p < .001, 95% CI [0.02, 0.06]. RT also significantly differed for recollection between conditions, such that participants were faster at recollecting congruently presented trials (M = 1.69 sec, SD = 0.30 sec) than incongruently presented trials (M = 1.87 sec, SD = 0.37 sec), t(25) = −0.24, p < .001, 95% CI [−0.24, −0.12]. Last, higher overall recollection rate was significantly correlated with faster RT (r = −.36, p < .01).

For the sake of thoroughness, we also computed adjusted familiarity hit rate [pKnow/(1 − pRemember)] and adjusted familiarity false alarm rate [pKnow FA/(1 − pRemember FA)]. There was no significant difference in the identification of configurally congruent pairs as familiar (using the “know” response; MadjustedFamiliarity = 42.90%, SDadjustedFamiliarity = 15.18%) compared to configurally incongruent pairs (MadjustedFamiliarity = 44.77%, SDadjustedFamiliarity = 14.60%), t(25) = 1.60, p = .12, 95% CI [−0.01, 0.11]. Similarly, there was no significant difference in the false alarm rate of familiarity to configurally congruent pairs (MadjustedFamiliarityFA = 12.07%, SDadjustedFamiliarityFA = 8.36%) compared to familiarity for configurally incongruent pairs (MadjustedFamiliarityFA = 10.20%, SDadjustedFamiliarityFA = 8.31%), t(25) = 1.75, p = .09, 95% CI [0.00, 0.04] (see Table 1 for a depiction of all behavioral results).

Table 1. 
Behavioral Data
 CongruentIncongruent
Overall hit* 0.88 (0.06) 0.79 (0.09) 
Overall hit RT* 1.85 sec (0.32 sec) 2.07 sec (0.042 sec) 
  
Recollection 
 Hit* 0.69 (0.13) 0.54 (0.16) 
 False alarm* 0.07 (0.03) 0.04 (0.03) 
 Adjusted hit* 0.62 (0.14) 0.50 (0.15) 
 RT* 1.69 sec (0.30 sec) 1.87 sec (0.37 sec) 
  
Adjusted familiarity 
 Hit 0.60 (0.13) 0.55 (0.15) 
 False alarm 0.12 (0.08) 0.10 (0.08) 
 Adjusted hit 0.48 (0.06) 0.45 (0.07) 
 RT 2.57 sec (0.49 sec) 2.56 sec (0.50 sec) 
 CongruentIncongruent
Overall hit* 0.88 (0.06) 0.79 (0.09) 
Overall hit RT* 1.85 sec (0.32 sec) 2.07 sec (0.042 sec) 
  
Recollection 
 Hit* 0.69 (0.13) 0.54 (0.16) 
 False alarm* 0.07 (0.03) 0.04 (0.03) 
 Adjusted hit* 0.62 (0.14) 0.50 (0.15) 
 RT* 1.69 sec (0.30 sec) 1.87 sec (0.37 sec) 
  
Adjusted familiarity 
 Hit 0.60 (0.13) 0.55 (0.15) 
 False alarm 0.12 (0.08) 0.10 (0.08) 
 Adjusted hit 0.48 (0.06) 0.45 (0.07) 
 RT 2.57 sec (0.49 sec) 2.56 sec (0.50 sec) 

This table reports the mean (and standard deviation) hit rate, false alarm rate, adjusted hit rate, and RT broken down by recognition response and condition. Significant differences between configurally congruent and incongruent trials are noted by an asterisk (*p < .001). Adjusted recollection hit rate = recollection rate − false alarm (FA) rate. Adjusted familiarity hit rate = [pKnow/(1 − pRemember)]. Adjusted familiarity FA rates rate = [pKnow FA/(1 − pRemember FA)].

Univariate

Overall successful recollection exhibited neural activity within the typical associative memory network, including bilateral occipital cortex, bilateral superior parietal cortex, bilateral medial and superior pFC, left PHC, and right hippocampus. Although retrieval success was not a planned analysis, we conducted the analysis to confirm the success of our task design in achieving typical recollection-related activity throughout the associative retrieval network.

Examination of the effect of configural congruency found that congruent compared to incongruent targets exhibited significantly greater activation in right medial superior frontal gyrus, bilateral middle temporal gyrus, left angular gyrus, and left cuneus. Alternatively, incongruent compared to congruent targets exhibited significantly greater activation in bilateral middle frontal gyrus, right inferior frontal gyrus, right middle cingulate gyrus, bilateral posterior parietal cortex, left inferior temporal gyrus, and left cerebellum. Neither contrast exhibited significant differences in MTL activation.

When focusing on recollection-related activation, successful recollection in the congruent compared to incongruent condition exhibited significantly greater activity in the right medial superior frontal gyrus, left middle temporal gyrus, and left caudate nucleus. The reverse contrast showed that incongruent compared to congruent recollection elicited greater activity in bilateral posterior parietal cortex, bilateral inferior frontal gyrus, right middle frontal gyrus, right fusiform gyrus, right cerebellum, and bilateral inferior temporal gyrus. Within the MTL, congruent recollection exhibited significantly greater activity than incongruent recollection in the left PHC. There was no greater activity for incongruent recollection compared to congruent recollection in any MTL region. See Figure 2 and Table 2 for a depiction of all univariate results.

Figure 2. 

Whole-brain activity for (A) congruent and incongruent targets and (B) congruent and incongruent recollection at retrieval. Red = greater activity for congruent compared to incongruent; blue = greater activity for incongruent > congruent.

Figure 2. 

Whole-brain activity for (A) congruent and incongruent targets and (B) congruent and incongruent recollection at retrieval. Red = greater activity for congruent compared to incongruent; blue = greater activity for incongruent > congruent.

Table 2. 
Congruency Activation
 BAHMNI Coordinateskt BAHMNI Coordinateskt
xyzxyz
Congruent > Incongruent Incongruent > Congruent 
Recollection Recollection 
Caudate nucleus 11 −12 14 −14 267 5.92 Superior parietal gyrus 26 −66 50 3184 8.55 
Superior frontal gyrus, medial 10 54 514 4.47 Superior parietal gyrus −24 −66 42 2877 6.7 
Middle temporal gyrus 39 −54 −56 22 253 3.62 Inferior frontal gyrus, opercular part 44 −40 10 24 1860 6.34 
Parahippocampal gyrusa 36 −26 −26 −16 52 4.75 Middle frontal gyrus 32 48 783 5.93 
                Inferior frontal gyrus, triangular part 46 50 40 14 1404 5.48 
                Fusiform gyrus 19 32 −48 −6 294 5.24 
                Cerebellum   −10 −78 −36 580 4.94 
                Inferior temporal gyrus 37 52 −54 −8 319 4.59 
                Inferior temporal gyrus 37 −52 −64 −10 482 4.48 
  
Targets Targets 
Medial prefrontal gyrus 10 58 12 2021 5.27 Superior/inferior parietal lobule 7/39/40 26 −66 50 2800 6.83 
Middle temporal gyrus 38 −60 −28 556 5.05 Superior/inferior parietal lobule 7/39/40 −20 −66 50 2019 6.21 
Angular gyrus 39 −60 −54 36 1052 4.62 Middle frontal gyrus −26 48 452 5.99 
Middle cingulate gyrus 31 −22 40 281 4.55 Inferior temporal gyrus 37 48 −54 −12 515 5.72 
Middle temporal gyrus 22 62 −34 385 4.41 Inferior frontal gyrus, triangular part 45 44 24 14 1510 5.62 
Cuneus 18 −8 −92 24 1008 4.22 Cerebellum 18 −10 −80 −34 301 5.09 
                Middle frontal gyrus 26 12 48 734 4.95 
                Middle cingulate gyrus 12 18 38 438 4.57 
                Inferior temporal gyrus 37 −48 −60 −8 246 4.51 
 BAHMNI Coordinateskt BAHMNI Coordinateskt
xyzxyz
Congruent > Incongruent Incongruent > Congruent 
Recollection Recollection 
Caudate nucleus 11 −12 14 −14 267 5.92 Superior parietal gyrus 26 −66 50 3184 8.55 
Superior frontal gyrus, medial 10 54 514 4.47 Superior parietal gyrus −24 −66 42 2877 6.7 
Middle temporal gyrus 39 −54 −56 22 253 3.62 Inferior frontal gyrus, opercular part 44 −40 10 24 1860 6.34 
Parahippocampal gyrusa 36 −26 −26 −16 52 4.75 Middle frontal gyrus 32 48 783 5.93 
                Inferior frontal gyrus, triangular part 46 50 40 14 1404 5.48 
                Fusiform gyrus 19 32 −48 −6 294 5.24 
                Cerebellum   −10 −78 −36 580 4.94 
                Inferior temporal gyrus 37 52 −54 −8 319 4.59 
                Inferior temporal gyrus 37 −52 −64 −10 482 4.48 
  
Targets Targets 
Medial prefrontal gyrus 10 58 12 2021 5.27 Superior/inferior parietal lobule 7/39/40 26 −66 50 2800 6.83 
Middle temporal gyrus 38 −60 −28 556 5.05 Superior/inferior parietal lobule 7/39/40 −20 −66 50 2019 6.21 
Angular gyrus 39 −60 −54 36 1052 4.62 Middle frontal gyrus −26 48 452 5.99 
Middle cingulate gyrus 31 −22 40 281 4.55 Inferior temporal gyrus 37 48 −54 −12 515 5.72 
Middle temporal gyrus 22 62 −34 385 4.41 Inferior frontal gyrus, triangular part 45 44 24 14 1510 5.62 
Cuneus 18 −8 −92 24 1008 4.22 Cerebellum 18 −10 −80 −34 301 5.09 
                Middle frontal gyrus 26 12 48 734 4.95 
                Middle cingulate gyrus 12 18 38 438 4.57 
                Inferior temporal gyrus 37 −48 −60 −8 246 4.51 

In the table, x, y, and z represent peak MNI, and k indicates cluster extent. H = hemisphere (L = left; R = right); t = statistical t value.

a

Corrected at MTL threshold.

MVPA

We first examined whether there was an effect of configural congruency on classification of targets at retrieval, irrespective of the trial's encoding configuration (i.e., side-by-side or superimposed configuration). No region showed significant above-chance classification. Two regions had marginally significant accuracy (IOC; Maccuracy = 0.52, t(25) = 1.99, p = .057; posterior parietal cortex: Maccuracy = 0.53, t(25) = 2.05, p = .051).

On the basis of our previous work showing that the configural context at encoding is a critical factor in classifying neural patterns of activation (Dennis et al., 2019), we next examined the effect of configural congruency on classification considering the configural context at encoding. Specifically, we examined whether a classifier could accurately classify targets based on the four conditions of interest above theoretical chance (25%) in each ROI. The classifier was able to accurately predict retrieval condition above chance in the IOC (Maccuracy = 0.31, t(25) = 8.12, p < .001), MOC (Maccuracy = 0.36, t(25) = 10.86, p < .001), and posterior parietal cortex (Maccuracy = 0.29, t(25) = 4.03, p < .001). The classifier was marginally above chance in pFC (Maccuracy = 0.27, t(25) = 1.89, p = .07). It also fell significantly below chance in the PHC (Maccuracy = 0.23, t(25) = −2.10, p = .047). No other region reached significance. In none of the foregoing regions did classification accuracy correlate with recollection rates. All multivariate results reported above were confirmed through permutation testing. See Table 3 for a depiction of all MVPA results.

Table 3. 
Classifier Accuracy at Retrieval
 Classifier Accuracytp Value
Combined (chance = 0.50) 
 IOC 0.523 2.00 .057 
 MOC 0.515 1.31 .203 
 Posterior parietal cortex 0.529 2.05 .051 
 HC 0.512 1.09 .288 
 PHC 0.485 −1.70 .101 
 PrC 0.496 −0.46 .650 
 pFC 0.516 1.352 .189 
  
Separate (chance = 0.25) 
 IOC 0.309 8.12 .000 
 MOC 0.36 10.86 .000 
 Posterior parietal cortex 0.293 4.03 .000 
 HC 0.246 −0.61 .549 
 PHC 0.233 −2.10 .046 
 PrC 0.246 −0.52 .605 
 pFC 0.272 1.89 .071 
 Classifier Accuracytp Value
Combined (chance = 0.50) 
 IOC 0.523 2.00 .057 
 MOC 0.515 1.31 .203 
 Posterior parietal cortex 0.529 2.05 .051 
 HC 0.512 1.09 .288 
 PHC 0.485 −1.70 .101 
 PrC 0.496 −0.46 .650 
 pFC 0.516 1.352 .189 
  
Separate (chance = 0.25) 
 IOC 0.309 8.12 .000 
 MOC 0.36 10.86 .000 
 Posterior parietal cortex 0.293 4.03 .000 
 HC 0.246 −0.61 .549 
 PHC 0.233 −2.10 .046 
 PrC 0.246 −0.52 .605 
 pFC 0.272 1.89 .071 

Results of a one-sample t test for classifier accuracy. All df = 25. Combined = combined congruent vs. combined incongruent targets; Separate = congruent side-by-side vs. congruent superimposed vs. incongruent side-by-side vs. incongruent superimposed; posterior parietal cortex = angular gyrus and BA 5/7; HC = hippocampus.

ERS

We examined the effect of contextual congruency and configural context on neural pattern similarity for targets by computing Pearson's r correlation values for each target type (side-by-side congruent, superimposed congruent, side-by-side incongruent, and superimposed incongruent) between encoding and retrieval for each ROI. We then computed a repeated-measures ANOVA to determine whether pattern similarity within each ROI significantly differed based on target type. We also corrected for multiple comparisons across the four conditions within ROI using the Bonferroni method (significance at p < .0125). There was a significant effect of Target Type within the IOC, F(3, 23) = 10.07, p < .001, and the MOC, F(3, 23) = 28.07. p < .001 (Figure 3). Pattern similarity did not significantly differ between target type in any other region, nor did the ERS of any target type within the IOC or MOC significantly predict or correlate with recollection.

Figure 3. 

ERS for targets broken down by condition in the IOC and MOC.

Figure 3. 

ERS for targets broken down by condition in the IOC and MOC.

Paired-samples t tests were used to make post hoc comparisons between conditions in the IOC and MOC. Using the same corrected significance threshold as above, ERS within the IOC significantly differed for side-by-side congruent and side-by-side incongruent targets, t(26) = 3.97, p < .005, as well as superimposed congruent and side-by-side incongruent targets, t(26) = 4.02, p < .001. Within the MOC, ERS significantly differed for side-by-side congruent and side-by-side incongruent targets, t(26) = 7.88, p < .001; superimposed congruent and side-by-side incongruent targets, t(26) = 6.48, p < .001; side-by-side congruent and superimposed incongruent targets, t(26) = 5.24, p < .001; and superimposed congruent and superimposed incongruent targets, t(26) = 4.02, p < .001. See Table 4 for all pairwise comparisons and Figure 3 for a depiction of ERS results.

Table 4. 
Pairwise Comparisons for ERS Correlations
 tp Value
IOC 
 Side-by-side congruent − superimposed congruent −1.70 .102 
 Side-by-side congruent − side-by-side incongruent 3.97 .001 
 Side-by-side congruent − superimposed incongruent 2.07 .049 
 Superimposed congruent − side-by-side incongruent 4.02 .000 
 Superimposed congruent − superimposed incongruent 2.58 .016 
 Side-by-side incongruent − superimposed incongruent −2.50 .019 
  
MOC 
 Side-by-side congruent − superimposed congruent −1.71 .100 
 Side-by-side congruent − side-by-side incongruent 7.88 .000 
 Side-by-side congruent − superimposed incongruent 5.24 .000 
 Superimposed congruent − side-by-side incongruent 6.48 .000 
 Superimposed congruent − superimposed incongruent 4.54 .000 
 Side-by-side incongruent − superimposed incongruent −1.59 .123 
 tp Value
IOC 
 Side-by-side congruent − superimposed congruent −1.70 .102 
 Side-by-side congruent − side-by-side incongruent 3.97 .001 
 Side-by-side congruent − superimposed incongruent 2.07 .049 
 Superimposed congruent − side-by-side incongruent 4.02 .000 
 Superimposed congruent − superimposed incongruent 2.58 .016 
 Side-by-side incongruent − superimposed incongruent −2.50 .019 
  
MOC 
 Side-by-side congruent − superimposed congruent −1.71 .100 
 Side-by-side congruent − side-by-side incongruent 7.88 .000 
 Side-by-side congruent − superimposed incongruent 5.24 .000 
 Superimposed congruent − side-by-side incongruent 6.48 .000 
 Superimposed congruent − superimposed incongruent 4.54 .000 
 Side-by-side incongruent − superimposed incongruent −1.59 .123 

Pairwise comparisons for ERS correlations within regions that produced a main effect of configural context.

DISCUSSION

The focus of the current investigation was to examine the behavioral and neural differences associated with maintaining and manipulating configural context when remembering associative pairs. Behaviorally, participants recollected, and false alarmed, significantly more for associative pairs when the configural context was maintained between encoding and retrieval than when there was a disruption to the configural context. Remembering congruent pairs was also associated with faster RTs compared to remembering associates when the contextual congruency with encoding was disrupted. Furthermore, manipulations of contextual congruency affected neural activation across the retrieval network. Specifically, viewing contextual congruent targets at retrieval was associated with greater activation within the visuospatial processing network (Kravitz, Saleem, Baker, & Mishkin, 2011), including right medial superior frontal gyrus, bilateral middle temporal gyrus, left angular gyrus, and left cuneus, whereas incongruent targets compared to congruent targets exhibited significantly greater activation across a frontoparietal monitoring network (Iidaka, Matsumoto, Nogawa, Yamamoto, & Sadato, 2006), including bilateral middle frontal gyrus, right inferior frontal gyrus, right middle cingulate gyrus, and bilateral superior/inferior parietal lobule. When examining the effect of configural context within recollection, recollection associated with congruent compared to incongruent pairs exhibited significantly greater activity in the left PHC, right medial superior frontal gyrus, left middle temporal gyrus, and left caudate nucleus. The reverse contrast showed that recollection associated with incongruent compared to congruent pairs again elicited greater activity across the frontoparietal monitoring network, including bilateral posterior parietal cortex, bilateral inferior frontal gyrus, and right middle frontal gyrus. Finally, we also showed that both configural context and contextual congruency were critical determinants to characterizing patterns of neural activation within visual and parietal cortices. The aforementioned visual regions also had significantly different patterns of activity across encoding and retrieval for congruent and incongruent associative targets. Taken together, our results support our hypothesis that contextual congruency at retrieval affects both behavior and neural processing within associative memory.

Behaviorally, our results are consistent with the larger literature indicating that context is important in memory retrieval (Tibon et al., 2012; Gruppuso et al., 2007; Bar, 2004; Bar & Aminoff, 2003; Mandler, 1980; DaPolito et al., 1972). Specifically, participants exhibited lower recollection rates and slower RTs for associations that were presented in a configural context that was incongruent with respect to how the pair was learned at encoding compared to pairs in which the configural context was consistent with that of encoding. Replicating our previous work examining contextual congruency (Overman et al., 2018), this suggests there is increased difficulty involved with retrieval of associative information when contextual congruency is disrupted between study and test. Collectively, our results add to the long-standing literature highlighting the benefits of context reinstatement to memory (Gruppuso et al., 2007; Hayes et al., 2007; McKenzie & Tiberghien, 2004; Macken, 2002; Murnane et al., 1999; Smith, 1979, 1982, 1988; Eich, 1985; Nixon & Kanak, 1981; Mandler, 1980; Godden & Baddeley, 1975; DaPolito et al., 1972; Thomson, 1972; Thomson & Tulving, 1970) by also identifying the benefits of configural context to associative memory.

Of interest to our findings regarding the effect of configural congruency to memory is the notion that all individual aspects of the original associative information are recapitulated from encoding to retrieval. It is merely the orientation and configuration of the information that differs. To this point, we posit that a disruption to configural congruency parallels work in the domains of working memory and mental rotation (Anguera, Reuter-Lorenz, Willingham, & Seidler, 2010; Cepeda & Kramer, 1999). That is, mental rotation is a process by which one needs to mentally reorient a visual image until it matches a previously encountered state or target image (Podzebenko, Egan, & Watson, 2002; Corballis, 1997; Cohen et al., 1996; Shepard & Metzler, 1971). Just as mental rotation tasks require participants to mentally rotate a given object back to its original state, we posit that a similar mental reconfiguration is required during associative memory retrieval when responding to associative pairs that are configurally incongruent with that shown at encoding. Importantly, this process may be fundamentally different than the need to recall a missing context as is the case in past work investigating the effect of context changes to item retrieval (Bar, Aminoff, & Schacter, 2008; Hayes et al., 2004, 2007). The current study extends the definition and role of context in memory retrieval to consider not only the presence and absence of specific configural elements but also the orientation of those elements and the influence of orientation to retrieval success.

We extend these behavioral results to show that configural congruency also affects neural processing involved in associative retrieval. Although our overall associative memory success results are consistent with typical associative retrieval studies showing recollection-related activation in the hippocampus, parahippocampal gyrus, medial superior frontal gyrus, anterior and posterior cingulate gyrus, angular gyrus, cuneus, and cerebellum (Dennis & McCormick-Huhn, 2018; Wang et al., 2014; van Kesteren, Rijpkema, Ruiter, & Fernández, 2010; Takashima et al., 2007), we see that configural congruency modifies this activation. Extending our investigation of configural congruency to neural functioning, we found that the retrieval of associative pairs presented in an incongruent configural context was met with greater widespread activity across frontoparietal regions including bilateral superior and inferior parietal cortex, bilateral inferior frontal gyrus, right middle frontal gyrus, and right fusiform gyri compared to those retrieved within a congruent context. Within the general retrieval literature, increases in frontoparietal activity have been associated with increases in task difficulty (Gould et al., 2003) associated with a need for greater monitoring, decision-making, and evaluation (Achim & Lepage, 2005; Fellows & Farah, 2005; Rugg, Henson, & Robb, 2003; Rugg, Otten, & Henson, 2002; Ranganath & Paller, 2000). We posit that the difference in the configural context displayed at retrieval with that stored in memory from encoding induces a mismatch of representations that needs to be resolved through monitoring and evaluation processes before producing a memory response. Thus, consistent with the behavioral findings, the neuroimaging results also support the conclusion that the incongruent condition results in a greater cognitive load posed to individuals as they attempt to retrieve the associative link across individual items.

The foregoing heightened activation throughout the frontoparietal monitoring network is observed not only for general associative retrieval (across all targets) but also for successful recollection in the current study and suggests that this activity supports the successful recollection of incongruent associative pairs. In other words, if heightened activity was observed for all targets but was not prominent during recollection, it would suggest that the observed activity supports search and monitoring processes that are engaged during any retrieval attempt. The similar pattern of heightened activity during both general retrieval and successful recollection suggests that recollection is the driving force behind the differences. Noted above, we suggested a parallel between mental rotation tasks and incongruent associative retrieval, as individuals are likely attempting to mentally rearrange the pair at retrieval to match its original configural context when responding to configurally incongruent trials. Supporting this interpretation, mental rotation tasks are also associated with increased activation across the same frontal and parietal regions observed in the current study (Christophel, Cichy, Hebart, & Haynes, 2015; Gogos et al., 2010; Levin, Mohamed, & Platek, 2005; Diwadkar, Carpenter, & Just, 2000; see Zacks, Speer, Swallow, Braver, & Reynolds, 2007, for a review). Moreover, increased activation across the frontoparietal network is modulated as a function of the degree of rotation and has been shown to support successful behavior during mental rotation tasks (Gogos et al., 2010; Milivojevic, Hamm, & Corballis, 2009). Additional work investigating the role of mental rotation in incongruent associative retrieval is needed to fully understand the meaning underlying the observed frontoparietal activation.

Despite the absence of univariate differences in overall levels of neural activation within visual cortices, multivariate analyses that measure differences in patterns of neural activity identified differences across both configural context and contextual congruency at retrieval across occipital ROIs. Specifically, considering both configural context at encoding and contextual congruency at retrieval, patterns of neural activity were distinguishable in both visual cortex, specifically the IOC and MOC, and posterior parietal cortex. Thus, our findings suggest neural patterns at retrieval are discriminable only when separated by both the configural context in which they were originally encoded as well as whether that specific context was maintained at retrieval. Noted in our Methods section, our attempt to train a classifier to identify differences between incongruent and congruent retrieval configurations, irrespective of the configural context shown at encoding (mirroring the foregoing univariate analysis), resulted in no significant classification across our a priori ROIs. This is not altogether surprising, considering our classification results from encoding found that configural context influenced neural patterns of activation across the same visual regions (Dennis et al., 2019). The current finding is consistent with a larger literature that shows that the occipital cortex is sensitive to the physical and categorical properties of visual stimuli (Dennis et al., 2019; Harrison & Tong, 2009; Kapadia, Westheimer, & Gilbert, 2000; Kanwisher, McDermott, & Chun, 1997; Gibson, 1969) as well as mental imagery paradigms that find early visual cortices maintain information regarding the orientation of a visual image (Naselaris, Olman, Stansbury, Ugurbil, & Gallant, 2015; Albers, Kok, Toni, Dijkerman, & de Lange, 2013; Lee, Kravitz, & Baker, 2012).

The fact that the posterior parietal cortex also shows this same pattern of results is also consistent with a larger retrieval literature showing that the parietal cortex is responsible for processing and storing spatial information of visual stimuli (Ramanan & Bellana, 2019; Crowe, Averbeck, & Chafee, 2010; Kesner, 2009; Vilberg & Rugg, 2008; Cabeza et al., 2002). In considering neural activity within the parietal cortex that can also discriminate between retrieval conditions when accounting for configural context at encoding and subsequent contextual congruency, this complements past work that has shown that the angular gyrus can discriminate patterns of activity elicited by two or more study conditions (Kuhl & Chun, 2014; Johnson, McDuff, Rugg, & Norman, 2009). Specially, this past work has shown frontoparietal regions can discriminate between individual events within memory, part of which is because of the contextual reinstatement of the encoded episode.

Interestingly, whereas univariate contrasts showed greater activity across both frontal and parietal regions for incongruent configural contexts, multivariate analyses found discriminable neural patterns between all four retrieval conditions in parietal cortex, but not our frontal ROI. With respect to the role of each region in mental rotation tasks that have parallel demands to our retrieval task, it has been suggested that, whereas the parietal lobe mediates the specific rotation aspects of the task (Milivojevic et al., 2009; Zacks, 2008; Koshino, Carpenter, Keller, & Just, 2005; Podzebenko et al., 2002; Jordan, Heinze, Lutz, Kanowski, & Jäncke, 2001; Harris et al., 2000; Carpenter, Just, Keller, Eddy, & Thulborn, 1999; Alivisatos & Petrides, 1997; Cohen et al., 1996), frontal regions underlie monitoring and comparison between the rotated image and the stored representation of the target image. Following from this literature, the current results would suggest that, although the occipital and parietal cortices clearly represent differences in configural context and contextual congruency, the underlying evaluation processes engaged in by frontal cortices may be similar in nature across conditions (e.g., Christophel et al., 2015), albeit heightened for incongruent retrieval. Future work will be needed to further examine this dissociation with respect to contextual congruency in retrieval processes.

In addition to significant differences in neural activation patterns in IOC and MOC at retrieval, our ERS analysis showed that patterns of neural activity were found to be similar across encoding and retrieval in the aforementioned visual regions when considering configural context and configural reinstatement. These findings are indicative of greater recapitulation of visual processes and stored mental representation of associative pairs when viewing associative targets presented in the same configural context across both memory phases. Stronger recapitulation for congruent opposed to incongruent trials supports the TAP theory, which posits that (1) memories are represented in terms of the cognitive operations engaged by an event as it is initially processed and (2) successful memory retrieval occurs when those earlier operations are recapitulated (Rugg et al., 2008; Morris et al., 1977; Kolers, 1973). Accordingly, TAP suggests that the effectiveness of a retrieval cue depends on the similarity between the cognitive operations engaged by the cue and the cognitive operations that occurred during study (Roediger & Guynn, 1996; Roediger & Challis, 1989). Previous empirical applications of this principle have shown that recapitulation, by way of greater neural pattern similarity between encoding and retrieval, is associated with better memory (e.g., Ritchey et al., 2013; Xue et al., 2010). The current results extend this previous work to show that recapitulation of neural activation within occipital cortices also underlies associative retrieval and that these processes are benefited by maintaining not only the reoccurrence of all individual items but also the contextual congruency with respect to configuration as well. Interestingly, not only did we observe significantly greater correlations between encoding and retrieval activation patterns for congruent associations, but we saw significant anticorrelations in our ERS analysis for incongruent targets in the IOC and MOC. This suggests that, despite the fact that the information in the associative pairs was retained from study to test, altering the configural context of the pairs from encoding to retrieval leads to unique neural patterns that systematically differ across memory stages (Fox et al., 2005). Although disruptions of contextual congruency across memory phases clearly disrupt the recapitulation of neural patterns from encoding to retrieval, it is unclear whether greater deviation from encoding processes leads to greater detriments in behavior. Future work is needed to understand the behavioral consequences of such anticorrelations in neural activation.

Contrary to our predictions, we did not see any univariate or multivariate differences within the hippocampus. This runs counter to work by Giovanello et al. (2009), who demonstrated greater activation in posterior hippocampus for congruent compared to incongruent word pairs and greater activity in anterior hippocampus for the reverse contrast. This absence of hippocampal differences may be because of the fact that the previous study used words, whereas the present used more salient face–scene associative pairs. Furthermore, the incongruent condition in the previous study involved a reversal of the placement of the word pairs on the screen. As such, it could be that this switch induced a different overall interpretation by the participant of the word meanings, leading to more complex or varied meanings of the associative pair from encoding to retrieval. Although past work has also found that activity in the hippocampus at encoding can be reactivated at retrieval for content-specific information for perceptually associated word pairs (Prince, Daselaar, & Cabeza, 2005), our findings speak to the idea that contextual congruency may not impact hippocampal processing when the associative meaning is not disrupted, such as the case with objects or face–scene associations. In other words, the meaning-based mental representation of associative word pairs in Giovanello et al. (2009) may be more susceptible to disruption at retrieval by a change in configuration (e.g., word order) than the mental representation of a face–scene pair would be at retrieval. Accordingly, an adjustment to the relative location of the face and scene likely has little impact on their interdependent meaning. Future studies might examine the effect of disrupting meaning with associative pairs on neural processing supporting associative retrieval. Of note, if our threshold was lowered to that reported in the Giovanello study (p < .01 and five contiguous voxels), univariate differences would emerge. Thus, it may also be that the effect size of differences in contextual congruency along the long axis of the hippocampus is relatively small and only emerge at lower thresholds.

Future Directions

In light of the current findings, we suggest exploring several future directions with respect to elucidating the neural processes underlying the effect of configural context on associative retrieval. First, given the difficulties older adults face with associative memory (Naveh-Benjamin, 2000), especially when configural context is manipulated (Overman et al., 2018), examining the extent to which a change in configural context impacts neural processing underlying their memory deficit would be highly beneficial to understanding if older adults' errors arise from similar disruptions in TAP and ERS processing. In addition, it would be interesting to examine whether manipulating configural context in other ways affects neural processing, such as introducing a third retrieval condition in which the pair is presented in a novel configuration pattern that does not mirror the configuration of either of the encoding conditions. Last, because the current paradigm involved retrieving highly salient items, in which the associative meaning did not change as a function of the configural context. It may be interesting to explore whether the degree of stimuli complexity or disruptions to the associative meaning across encoding and retrieval impact differences in visual and MTL processing during associative retrieval.

Conclusions

The current results extend prior work showing configural context within association memory is a critical determinant of memory success. In doing so, we expand the definition of context to include the orientation and configuration of associated pairs. As well, the current study demonstrates that configural context as well as configural congruency between encoding and retrieval are critical factors in determining neural processes supporting associative memory retrieval. Mirroring work in the field of mental rotation, we showed that disruptions to the configural context between encoding and retrieval are met with increased neural activation across frontoparietal regions. We posit that this activation supports the need to reconfigure the associative pair to match the encoding configuration before making a memory decision. This increased processing is supported by behavioral findings of greater errors and increased RTs for incongruent compared to congruent associative retrieval. In addition, patterns of neural activation at retrieval were distinguishable in occipital and parietal cortex only when considering both configural context and contextual congruency. Finally, the ERS results support the behavioral findings and past work promoting TAP, in showing greater neural recapitulation of activation patterns across encoding and retrieval for configural reinstatement.

Acknowledgments

We thank Catherine Carpenter and Valeria Martinez Goodman for help with data collection and analyses as well as Jordan Chamberlain and Daniel Elbich for support in analyses and comments on an earlier version of the article. This work was supported by the National Institutes of Health under grant R15AG052903 awarded to A. A. O. and N. A. D. In addition, N. A. D. was also supported in part by the National Science Foundation under grant BCS1025709. Portions of the research in this article used the Color FERET (Facial Recognition Technology) database of facial images collected under the FERET program, sponsored by the Department of Defense Counterdrug Technology Development Program Office.

Reprint requests should be sent to Nancy A. Dennis, The Department of Psychology, The Pennsylvania State University, 450 Moore Building, University Park, PA 16802, or via e-mail: nad12@psu.edu.

REFERENCES

Achim
,
A. M.
,
Bertrand
,
M.-C.
,
Montoya
,
A.
,
Malla
,
A. K.
, &
Lepage
,
M.
(
2007
).
Medial temporal lobe activations during associative memory encoding for arbitrary and semantically related object pairs
.
Brain Research
,
1161
,
46
55
.
Achim
,
A. M.
, &
Lepage
,
M.
(
2005
).
Dorsolateral prefrontal cortex involvement in memory post-retrieval monitoring revealed in both item and associative recognition tests
.
Neuroimage
,
24
,
1113
1121
.
Albers
,
A. M.
,
Kok
,
P.
,
Toni
,
I.
,
Dijkerman
,
H. C.
, &
de Lange
,
F. P.
(
2013
).
Shared representations for working memory and mental imagery in early visual cortex
.
Current Biology
,
23
,
1427
1431
.
Alivisatos
,
B.
, &
Petrides
,
M.
(
1997
).
Functional activation of the human brain during mental rotation
.
Neuropsychologia
,
35
,
111
118
.
Andersson
,
J. L. R.
,
Jenkinson
,
M.
, &
Smith
,
S. M.
(
2010
).
Non-linear registration, a.k.a. spatial normalisation (FMRIB Technical Report No. TR07JA2)
.
Oxford, UK
:
FMRIB Centre
.
Anguera
,
J. A.
,
Reuter-Lorenz
,
P. A.
,
Willingham
,
D. T.
, &
Seidler
,
R. D.
(
2010
).
Contributions of spatial working memory to visuomotor learning
.
Journal of Cognitive Neuroscience
,
22
,
1917
1930
.
Bar
,
M.
(
2004
).
Visual objects in context
.
Nature Reviews Neuroscience
,
5
,
617
629
.
Bar
,
M.
, &
Aminoff
,
E.
(
2003
).
Cortical analysis of visual context
.
Neuron
,
38
,
347
358
.
Bar
,
M.
,
Aminoff
,
E.
, &
Schacter
,
D. L.
(
2008
).
Scenes unseen: The parahippocampal cortex intrinsically subserves contextual associations, not scenes or places per se
.
Journal of Neuroscience
,
28
,
8539
8544
.
Bowman
,
C. R.
,
Chamberlain
,
J. D.
, &
Dennis
,
N. A.
(
2019
).
Sensory representations supporting memory specificity: Age effects on behavioral and neural discriminability
.
Journal of Neuroscience
,
39
,
2265
2275
.
Brodt
,
S.
,
Gais
,
S.
,
Beck
,
J.
,
Erb
,
M.
,
Scheffler
,
K.
, &
Schönauer
,
M.
(
2018
).
Fast track to the neocortex: A memory engram in the posterior parietal cortex
.
Science
,
362
,
1045
1048
.
Cabeza
,
R.
,
Dolcos
,
F.
,
Graham
,
R.
, &
Nyberg
,
L.
(
2002
).
Similarities and differences in the neural correlates of episodic memory retrieval and working memory
.
Neuroimage
,
16
,
317
330
.
Cabeza
,
R.
, &
St Jacques
,
P.
(
2007
).
Functional neuroimaging of autobiographical memory
.
Trends in Cognitive Sciences
,
11
,
219
227
.
Carpenter
,
P. A.
,
Just
,
M. A.
,
Keller
,
T. A.
,
Eddy
,
W.
, &
Thulborn
,
K.
(
1999
).
Graded functional activation in the visuospatial system with the amount of task demand
.
Journal of Cognitive Neuroscience
,
11
,
9
24
.
Cepeda
,
N. J.
, &
Kramer
,
A. F.
(
1999
).
Strategic effects on object-based attentional selection
.
Acta Psychologica
,
103
,
1
19
.
Christophel
,
T. B.
,
Cichy
,
R. M.
,
Hebart
,
M. N.
, &
Haynes
,
J.-D.
(
2015
).
Parietal and early visual cortices encode working memory content across mental transformations
.
Neuroimage
,
106
,
198
206
.
Cohen
,
M. S.
,
Kosslyn
,
S. M.
,
Breiter
,
H. C.
,
DiGirolamo
,
G. J.
,
Thompson
,
W. L.
,
Anderson
,
A. K.
, et al
(
1996
).
Changes in cortical activity during mental rotation: A mapping study using functional MRI
.
Brain
,
119
,
89
100
.
Corballis
,
M. C.
(
1997
).
Mental rotation and the right hemisphere
.
Brain and Language
,
57
,
100
121
.
Cox
,
R. W.
, &
Hyde
,
J. S.
(
1997
).
Software tools for analysis and visualization of fMRI data
.
NMR in Biomedicine
,
10
,
171
178
.
Criss
,
A. H.
(
2010
).
Differentiation and response bias in episodic memory: Evidence from reaction time distributions
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
36
,
484
499
.
Crowe
,
D. A.
,
Averbeck
,
B. B.
, &
Chafee
,
M. V.
(
2010
).
Rapid sequences of population activity patterns dynamically encode task-critical spatial information in parietal cortex
.
Journal of Neuroscience
,
30
,
11640
11653
.
DaPolito
,
F.
,
Barker
,
D.
, &
Wiant
,
J.
(
1972
).
The effects of contextual changes on component recognition
.
American Journal of Psychology
,
85
,
431
440
.
Davis
,
S. W.
,
Dennis
,
N. A.
,
Daselaar
,
S. M.
,
Fleck
,
M. S.
, &
Cabeza
,
R.
(
2008
).
Qué PASA? The posterior–anterior shift in aging
.
Cerebral Cortex
,
18
,
1201
1209
.
De Brigard
,
F.
,
Langella
,
S.
,
Stanley
,
M. L.
,
Castel
,
A. D.
, &
Giovanello
,
K. S.
(
2020
).
Age-related differences in recognition in associative memory
.
Aging, Neuropsychology, and Cognition
,
27
,
289
301
.
Dennis
,
N. A.
, &
McCormick-Huhn
,
J. M.
(
2018
).
Item and associative memory decline in healthy aging
. In
J. T.
Wixted
(Ed.),
Stevens' handbook of experimental psychology and cognitive neuroscience, Vol. 1: Learning and memory
(pp.
323
362
).
New York
:
Wiley
.
Dennis
,
N. A.
,
Overman
,
A. A.
,
Gerver
,
C. R.
,
McGraw
,
K. E.
,
Rowley
,
M. A.
, &
Salerno
,
J. M.
(
2019
).
Different types of associative encoding evoke differential processing in both younger and older adults: Evidence from univariate and multivariate analyses
.
Neuropsychologia
,
135
,
107240
.
Diwadkar
,
V. A.
,
Carpenter
,
P. A.
, &
Just
,
M. A.
(
2000
).
Collaborative activity between parietal and dorso-lateral prefrontal cortex in dynamic spatial working memory revealed by fMRI
.
Neuroimage
,
12
,
85
99
.
Dulsky
,
S. G.
(
1935
).
The effect of a change of background on recall and relearning
.
Journal of Experimental Psychology
,
18
,
725
740
.
Egner
,
T.
,
Jamieson
,
G.
, &
Gruzelier
,
J.
(
2005
).
Hypnosis decouples cognitive control from conflict monitoring processes of the frontal lobe
.
Neuroimage
,
27
,
969
978
.
Eich
,
E.
(
1985
).
Context, memory, and integrated item/context imagery
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
11
,
764
770
.
Epstein
,
R. A.
, &
Ward
,
E. J.
(
2010
).
How reliable are visual context effects in the parahippocampal place area?
Cerebral Cortex
,
20
,
294
303
.
Etzel
,
J. A.
, &
Braver
,
T. S.
(
2013
).
MVPA permutation schemes: Permutation testing in the land of cross-validation
. In
Proceedings of the 2013 International Workshop on Pattern Recognition in Neuroimaging
(pp.
140
143
).
Philadelphia, PA
:
IEEE
.
Fellows
,
L. K.
, &
Farah
,
M. J.
(
2005
).
Dissociable elements of human foresight: A role for the ventromedial frontal lobes in framing the future, but not in discounting future rewards
.
Neuropsychologia
,
43
,
1214
1221
.
Fox
,
M. D.
,
Snyder
,
A. Z.
,
Vincent
,
J. L.
,
Corbetta
,
M.
,
Van Essen
,
D. C.
, &
Raichle
,
M. E.
(
2005
).
The human brain is intrinsically organized into dynamic, anticorrelated functional networks
.
Proceedings of the National Academy of Sciences, U.S.A.
,
102
,
9673
9678
.
Gaonkar
,
B.
, &
Davatzikos
,
C.
(
2012
).
Deriving statistical significance maps for SVM based image classification and group comparisons
. In
N.
Ayache
,
H.
Delingette
,
P.
Golland
, &
K.
Mori
(Eds.),
Medical image computing and computer-assisted intervention—MICCAI 2012
(
Vol. 7510
, pp.
723
730
).
Berlin, Germany
:
Springer
.
Gibson
,
E. J.
(
1969
).
Principles of perceptual learning and development
.
Science
,
168
,
958
959
.
Giovanello
,
K. S.
,
Schnyer
,
D.
, &
Verfaellie
,
M.
(
2009
).
Distinct hippocampal regions make unique contributions to relational memory
.
Hippocampus
,
19
,
111
117
.
Gisquet-Verrier
,
P.
, &
Riccio
,
D. C.
(
2012
).
Memory reactivation effects independent of reconsolidation
.
Learning & Memory
,
19
,
401
409
.
Godden
,
D. R.
, &
Baddeley
,
A. D.
(
1975
).
Context-dependent memory in two natural environments: On land and underwater
.
British Journal of Psychology
,
66
,
325
331
.
Gogos
,
A.
,
Gavrilescu
,
M.
,
Davison
,
S.
,
Searle
,
K.
,
Adams
,
J.
,
Rossell
,
S. L.
, et al
(
2010
).
Greater superior than inferior parietal lobule activation with increasing rotation angle during mental rotation: An fMRI study
.
Neuropsychologia
,
48
,
529
535
.
Goh
,
J. O. S.
,
Siong
,
S. C.
,
Park
,
D.
,
Gutchess
,
A.
,
Hebrank
,
A.
, &
Chee
,
M. W. L.
(
2004
).
Cortical areas involved in object, background, and object–background processing revealed with functional magnetic resonance adaptation
.
Journal of Neuroscience
,
24
,
10223
10228
.
Gould
,
R. L.
,
Brown
,
R. G.
,
Owen
,
A. M.
,
Ffytche
,
D. H.
, &
Howard
,
R. J.
(
2003
).
fMRI BOLD response to increasing task difficulty during successful paired associates learning
.
Neuroimage
,
20
,
1006
1019
.
Graf
,
P.
, &
Ryan
,
L.
(
1990
).
Transfer-appropriate processing for implicit and explicit memory
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
16
,
978
992
.
Gruppuso
,
V.
,
Lindsay
,
D. S.
, &
Masson
,
M. E. J.
(
2007
).
I'd know that face anywhere!
Psychonomic Bulletin & Review
,
14
,
1085
1089
.
Harris
,
I. M.
,
Egan
,
G. F.
,
Sonkkila
,
C.
,
Tochon-Danguy
,
H. J.
,
Paxinos
,
G.
, &
Watson
,
J. D. G.
(
2000
).
Selective right parietal lobe activation during mental rotation: A parametric PET study
.
Brain
,
123
,
65
73
.
Harrison
,
S. A.
, &
Tong
,
F.
(
2009
).
Decoding reveals the contents of visual working memory in early visual areas
.
Nature
,
458
,
632
635
.
Hayes
,
S. M.
,
Nadel
,
L.
, &
Ryan
,
L.
(
2007
).
The effect of scene context on episodic object recognition: Parahippocampal cortex mediates memory encoding and retrieval success
.
Hippocampus
,
17
,
873
889
.
Hayes
,
S. M.
,
Ryan
,
L.
,
Schnyer
,
D. M.
, &
Nadel
,
L.
(
2004
).
An fMRI study of episodic memory: Retrieval of object, spatial, and temporal information
.
Behavioral Neuroscience
,
118
,
885
896
.
Holdstock
,
J. S.
,
Hocking
,
J.
,
Notley
,
P.
,
Devlin
,
J. T.
, &
Price
,
C. J.
(
2009
).
Integrating visual and tactile information in the perirhinal cortex
.
Cerebral Cortex
,
19
,
2993
3000
.
Iidaka
,
T.
,
Matsumoto
,
A.
,
Nogawa
,
J.
,
Yamamoto
,
Y.
, &
Sadato
,
N.
(
2006
).
Frontoparietal network involved in successful retrieval from episodic memory: Spatial and temporal analyses using fMRI and ERP
.
Cerebral Cortex
,
16
,
1349
1360
.
Jenkinson
,
M.
,
Bannister
,
P.
,
Brady
,
M.
, &
Smith
,
S. M.
(
2002
).
Improved optimization for the robust and accurate linear registration and motion correction of brain images
.
Neuroimage
,
17
,
825
841
.
Johnson
,
J. D.
,
McDuff
,
S. G. R.
,
Rugg
,
M. D.
, &
Norman
,
K. A.
(
2009
).
Recollection, familiarity, and cortical reinstatement: A multivoxel pattern analysis
.
Neuron
,
63
,
697
708
.
Johnson
,
J. D.
, &
Rugg
,
M. D.
(
2007
).
Recollection and the reinstatement of encoding-related cortical activity
.
Cerebral Cortex
,
17
,
2507
2515
.
Jordan
,
K.
,
Heinze
,
H.-J.
,
Lutz
,
K.
,
Kanowski
,
M.
, &
Jäncke
,
L.
(
2001
).
Cortical activations during the mental rotation of different visual objects
.
Neuroimage
,
13
,
143
152
.
Kanwisher
,
N.
,
McDermott
,
J.
, &
Chun
,
M. M.
(
1997
).
The fusiform face area: A module in human extrastriate cortex specialized for face perception
.
Journal of Neuroscience
,
17
,
4302
4311
.
Kapadia
,
M. K.
,
Westheimer
,
G.
, &
Gilbert
,
C. D.
(
2000
).
Spatial distribution of contextual interactions in primary visual cortex and in visual perception
.
Journal of Neurophysiology
,
84
,
2048
2062
.
Kesner
,
R. P.
(
2009
).
The posterior parietal cortex and long-term memory representation of spatial information
.
Neurobiology of Learning and Memory
,
91
,
197
206
.
Kim
,
H.
(
2011
).
Neural activity that predicts subsequent memory and forgetting: A meta-analysis of 74 fMRI studies
.
Neuroimage
,
54
,
2446
2461
.
Kolers
,
P. A.
(
1973
).
Remembering operations
.
Memory & Cognition
,
1
,
347
355
.
Koshino
,
H.
,
Carpenter
,
P. A.
,
Keller
,
T. A.
, &
Just
,
M. A.
(
2005
).
Interactions between the dorsal and the ventral pathways in mental rotation: An fMRI study
.
Cognitive, Affective & Behavioral Neuroscience
,
5
,
54
66
.
Kravitz
,
D. J.
,
Saleem
,
K. S.
,
Baker
,
C. I.
, &
Mishkin
,
M.
(
2011
).
A new neural framework for visuospatial processing
.
Nature Reviews Neuroscience
,
12
,
217
230
.
Kriegeskorte
,
N.
,
Goebel
,
R.
, &
Bandettini
,
P.
(
2006
).
Information-based functional brain mapping
.
Proceedings of the National Academy of Sciences, U.S.A.
,
103
,
3863
3868
.
Kuhl
,
B. A.
, &
Chun
,
M. M.
(
2014
).
Successful remembering elicits event-specific activity patterns in lateral parietal cortex
.
Journal of Neuroscience
,
34
,
8051
8060
.
Lancaster
,
J. L.
,
Woldorff
,
M. G.
,
Parsons
,
L. M.
,
Liotti
,
M.
,
Freitas
,
C. S.
,
Rainey
,
L.
, et al
(
2000
).
Automated Talairach Atlas labels for functional brain mapping
.
Human Brain Mapping
,
10
,
120
131
.
Lee
,
S.-H.
,
Kravitz
,
D. J.
, &
Baker
,
C. I.
(
2012
).
Disentangling visual imagery and perception of real-world objects
.
Neuroimage
,
59
,
4064
4073
.
Levin
,
S. L.
,
Mohamed
,
F. B.
, &
Platek
,
S. M.
(
2005
).
Common ground for spatial cognition? A behavioral and fMRI study of sex differences in mental rotation and spatial working memory
.
Evolutionary Psychology
,
3
,
147470490500300116
.
Macken
,
W. J.
(
2002
).
Environmental context and recognition: The role of recollection and familiarity
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
28
,
153
161
.
Maillet
,
D.
, &
Rajah
,
M. N.
(
2014
).
Age-related differences in brain activity in the subsequent memory paradigm: A meta-analysis
.
Neuroscience & Biobehavioral Reviews
,
45
,
246
257
.
Mandler
,
G.
(
1980
).
Recognizing: The judgment of previous occurrence
.
Psychological Review
,
87
,
252
271
.
Martinez
,
A. M.
, &
Benavente
,
R.
(
1998
).
The AR face database (CVC Technical Report No. 24)
.
West Lafayette, IN
:
Purdue University
.
Mayes
,
A.
,
Montaldi
,
D.
, &
Migo
,
E.
(
2007
).
Associative memory and the medial temporal lobes
.
Trends in Cognitive Sciences
,
11
,
126
135
.
McDermott
,
K. B.
,
Jones
,
T. C.
,
Petersen
,
S. E.
,
Lageman
,
S. K.
, &
Roediger
,
H. L.
(
2000
).
Retrieval success is accompanied by enhanced activation in anterior prefrontal cortex during recognition memory: An event-related fMRI study
.
Journal of Cognitive Neuroscience
,
12
,
965
976
.
McDermott
,
K. B.
,
Szpunar
,
K. K.
, &
Christ
,
S. E.
(
2009
).
Laboratory-based and autobiographical retrieval tasks differ substantially in their neural substrates
.
Neuropsychologia
,
47
,
2290
2298
.
McKenzie
,
W. A.
, &
Tiberghien
,
G.
(
2004
).
Context effects in recognition memory: The role of familiarity and recollection
.
Consciousness and Cognition
,
13
,
20
38
.
Milivojevic
,
B.
,
Hamm
,
J. P.
, &
Corballis
,
M. C.
(
2009
).
Functional neuroanatomy of mental rotation
.
Journal of Cognitive Neuroscience
,
21
,
945
959
.
Minear
,
M.
, &
Park
,
D. C.
(
2004
).
A lifespan database of adult facial stimuli
.
Behavior Research Methods, Instruments & Computers
,
36
,
630
633
.
Mori
,
M.
, &
Graf
,
P.
(
1996
).
Nonverbal local context cues explicit but not implicit memory
.
Consciousness and Cognition
,
5
,
91
116
.
Morris
,
C. D.
,
Bransford
,
J. D.
, &
Franks
,
J. J.
(
1977
).
Levels of processing versus transfer appropriate processing
.
Journal of Verbal Learning and Verbal Behavior
,
16
,
519
533
.
Mumford
,
J. A.
,
Turner
,
B. O.
,
Ashby
,
F. G.
, &
Poldrack
,
R. A.
(
2012
).
Deconvolving BOLD activation in event-related designs for multivoxel pattern classification analyses
.
Neuroimage
,
59
,
2636
2643
.
Murnane
,
K.
, &
Phelps
,
M. P.
(
1993
).
A global activation approach to the effect of changes in environmental context on recognition
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
19
,
882
894
.
Murnane
,
K.
, &
Phelps
,
M. P.
(
1994
).
When does a different environmental context make a difference in recognition? A global activation model
.
Memory & Cognition
,
22
,
584
590
.
Murnane
,
K.
, &
Phelps
,
M. P.
(
1995
).
Effects of changes in relative cue strength on context-dependent recognition
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
21
,
158
172
.
Murnane
,
K.
,
Phelps
,
M. P.
, &
Malmberg
,
K.
(
1999
).
Context-dependent recognition memory: The ICE theory
.
Journal of Experimental Psychology: General
,
128
,
403
415
.
Nairne
,
J. S.
(
2002
).
Remembering over the short-term: The case against the standard model
.
Annual Review of Psychology
,
53
,
53
81
.
Nakashima
,
R.
, &
Yokosawa
,
K.
(
2011
).
Does scene context always facilitate retrieval of visual object representations?
Psychonomic Bulletin & Review
,
18
,
309
315
.
Naselaris
,
T.
,
Olman
,
C. A.
,
Stansbury
,
D. E.
,
Ugurbil
,
K.
, &
Gallant
,
J. L.
(
2015
).
A voxel-wise encoding model for early visual areas decodes mental images of remembered scenes
.
Neuroimage
,
105
,
215
228
.
Naveh-Benjamin
,
M.
(
2000
).
Adult age differences in memory performance: Tests of an associative deficit hypothesis
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
26
,
1170
1187
.
Naveh-Benjamin
,
M.
,
Guez
,
J.
,
Kilb
,
A.
, &
Reedy
,
S.
(
2004
).
The associative memory deficit of older adults: Further support using face–name associations
.
Psychology and Aging
,
19
,
541
546
.
Nixon
,
S. J.
, &
Kanak
,
N. J.
(
1981
).
The interactive effects of instructional set and environmental context changes on the serial position effect
.
Bulletin of the Psychonomic Society
,
18
,
237
240
.
Norman
,
K. A.
, &
O'Reilly
,
R. C.
(
2003
).
Modeling hippocampal and neocortical contributions to recognition memory: A complementary-learning-systems approach
.
Psychological Review
,
110
,
611
646
.
Oosterhof
,
N. N.
,
Connolly
,
A. C.
, &
Haxby
,
J. V.
(
2016
).
CoSMoMVPA: Multi-modal multivariate pattern analysis of neuroimaging data in MATLAB/GNU Octave
.
Frontiers in Neuroinformatics
,
10
,
27
.
Overman
,
A. A.
,
McCormick-Huhn
,
J. M.
,
Dennis
,
N. A.
,
Salerno
,
J. M.
, &
Giglio
,
A. P.
(
2018
).
Older adults' associative memory is modified by manner of presentation at encoding and retrieval
.
Psychology and Aging
,
33
,
82
92
.
Philiastides
,
M. G.
,
Ratcliff
,
R.
, &
Sajda
,
P.
(
2006
).
Neural representation of task difficulty and decision making during perceptual categorization: A timing diagram
.
Journal of Neuroscience
,
26
,
8965
8975
.
Phillips
,
P. J.
,
Moon
,
H.
,
Rizvi
,
S. A.
, &
Rauss
,
P. J.
(
2000
).
The FERET evaluation methodology for face-recognition algorithms
.
IEEE Transactions on Pattern Analysis and Machine Intelligence
,
22
,
1090
1104
.
Pihlajamäki
,
M.
,
Tanila
,
H.
,
Könönen
,
M.
,
Hänninen
,
T.
,
Hämäläinen
,
A.
,
Soininen
,
H.
, et al
(
2004
).
Visual presentation of novel objects and new spatial arrangements of objects differentially activates the medial temporal lobe subareas in humans
.
European Journal of Neuroscience
,
19
,
1939
1949
.
Podzebenko
,
K.
,
Egan
,
G. F.
, &
Watson
,
J. D. G.
(
2002
).
Widespread dorsal stream activation during a parametric mental rotation task, revealed with functional magnetic resonance imaging
.
Neuroimage
,
15
,
547
558
.
Polyn
,
S. M.
,
Natu
,
V. S.
,
Cohen
,
J. D.
, &
Norman
,
K. A.
(
2005
).
Category-specific cortical activity precedes retrieval during memory search
.
Science
,
310
,
1963
1966
.
Poppenk
,
J.
,
McIntosh
,
A. R.
,
Craik
,
F. I. M.
, &
Moscovitch
,
M.
(
2010
).
Past experience modulates the neural mechanisms of episodic memory formation
.
Journal of Neuroscience
,
30
,
4707
4716
.
Prince
,
S. E.
,
Daselaar
,
S. M.
, &
Cabeza
,
R.
(
2005
).
Neural correlates of relational memory: Successful encoding and retrieval of semantic and perceptual associations
.
Journal of Neuroscience
,
25
,
1203
1210
.
Ramanan
,
S.
, &
Bellana
,
B.
(
2019
).
A domain-general role for the angular gyrus in retrieving internal representations of the external world
.
Journal of Neuroscience
,
39
,
2978
2980
.
Ranganath
,
C.
, &
Paller
,
K. A.
(
2000
).
Neural correlates of memory retrieval and evaluation
.
Cognitive Brain Research
,
9
,
209
222
.
Ranganath
,
C.
, &
Ritchey
,
M.
(
2012
).
Two cortical systems for memory-guided behaviour
.
Nature Reviews Neuroscience
,
13
,
713
726
.
Rasch
,
B.
, &
Born
,
J.
(
2007
).
Maintaining memories by reactivation
.
Current Opinion in Neurobiology
,
17
,
698
703
.
Ratcliff
,
R.
, &
McKoon
,
G.
(
2008
).
The diffusion decision model: Theory and data for two-choice decision tasks
.
Neural Computation
,
20
,
873
922
.
Rhodes
,
M. G.
,
Castel
,
A. D.
, &
Jacoby
,
L. L.
(
2008
).
Associative recognition of face pairs by younger and older adults: The role of familiarity-based processing
.
Psychology and Aging
,
23
,
239
249
.
Ritchey
,
M.
,
Wing
,
E. A.
,
LaBar
,
K. S.
, &
Cabeza
,
R.
(
2013
).
Neural similarity between encoding and retrieval is related to memory via hippocampal interactions
.
Cerebral Cortex
,
23
,
2818
2828
.
Roediger
,
H. L.
, &
Challis
,
B. H.
(
1989
).
Hypermnesia: Improvements in recall with repeated testing
. In
C.
Izawa
(Ed.),
Current issues in cognitive processes: The Tutane Floweree Symposium on Cognition
(pp.
175
199
).
Hillsdale, NJ
:
Erlbaum
.
Roediger
,
H. L.
, &
Guynn
,
M. J.
(
1996
).
Retrieval processes
. In
E. L.
Bjork
&
R. A.
Bjork
(Eds.),
Human memory
(pp.
197
236
).
San Diego
:
Academic Press
.
RStudio Team
. (
2018
).
RStudio: Integrated development for R
.
Boston
:
RStudio, Inc
.
Rugg
,
M. D.
,
Henson
,
R. N. A.
, &
Robb
,
W. G. K.
(
2003
).
Neural correlates of retrieval processing in the prefrontal cortex during recognition and exclusion tasks
.
Neuropsychologia
,
41
,
40
52
.
Rugg
,
M. D.
,
Johnson
,
J. D.
,
Park
,
H.
, &
Uncapher
,
M. R.
(
2008
).
Encoding-retrieval overlap in human episodic memory: A functional neuroimaging perspective
.
Progress in Brain Research
,
169
,
339
352
.
Rugg
,
M. D.
,
Otten
,
L. J.
, &
Henson
,
R. N. A.
(
2002
).
The neural basis of episodic memory: Evidence from functional neuroimaging
.
Philosophical Transactions of the Royal Society of London, Series B: Biological Sciences
,
357
,
1097
1110
.
Schacter
,
D. L.
,
Verfaellie
,
M.
, &
Pradere
,
D.
(
1996
).
The neuropsychology of memory illusions: False recall and recognition in amnesic patients
.
Journal of Memory and Language
,
35
,
319
334
.
Sestieri
,
C.
,
Shulman
,
G. L.
, &
Corbetta
,
M.
(
2017
).
The contribution of the human posterior parietal cortex to episodic memory
.
Nature Reviews Neuroscience
,
18
,
183
192
.
Shepard
,
R. N.
, &
Metzler
,
J.
(
1971
).
Mental rotation of three-dimensional objects
.
Science
,
171
,
701
703
.
Siegel
,
A. L. M.
, &
Castel
,
A. D.
(
2018
).
Memory for important item–location associations in younger and older adults
.
Psychology and Aging
,
33
,
30
45
.
Smith
,
S. M.
(
1979
).
Remembering in and out of context
.
Journal of Experimental Psychology: Human Learning and Memory
,
5
,
460
471
.
Smith
,
S. M.
(
1982
).
Enhancement of recall using multiple environmental contexts during learning
.
Memory & Cognition
,
10
,
405
412
.
Smith
,
S. M.
(
1988
).
Environmental context—Dependent memory
. In
G. M.
Davies
&
D. M.
Thomson
(Eds.),
Memory in context: Context in memory
(pp.
13
34
).
Oxford
:
Wiley
.
Smith
,
S. M.
(
2002
).
Fast robust automated brain extraction
.
Human Brain Mapping
,
17
,
143
155
.
Solina
,
F.
,
Peer
,
P.
,
Batagelj
,
B.
,
Juvan
,
S.
, &
Kovač
,
J.
(
2003
).
Color-based face detection in the “15 seconds of fame” art installation
. In
International Conference on Computer Vision/Computer Graphics Collaboration for Model-based Imaging, Rendering, Image Analysis and Graphical Special Effects (MIRAGE 2003)
.
Paris
:
Institut National de Recherche en Informatique et en Automatique Rocquencourt
.
Spreng
,
R. N.
,
Mar
,
R. A.
, &
Kim
,
A. S. N.
(
2009
).
The common neural basis of autobiographical memory, prospection, navigation, theory of mind, and the default mode: A quantitative meta-analysis
.
Journal of Cognitive Neuroscience
,
21
,
489
510
.
Stelzer
,
J.
,
Chen
,
Y.
, &
Turner
,
R.
(
2013
).
Statistical inference and multiple testing correction in classification-based multi-voxel pattern analysis (MVPA): Random permutations and cluster size control
.
Neuroimage
,
65
,
69
82
.
Svoboda
,
E.
,
McKinnon
,
M. C.
, &
Levine
,
B.
(
2006
).
The functional neuroanatomy of autobiographical memory: A meta-analysis
.
Neuropsychologia
,
44
,
2189
2208
.
Takashima
,
A.
,
Nieuwenhuis
,
I. L. C.
,
Rijpkema
,
M.
,
Petersson
,
K. M.
,
Jensen
,
O.
, &
Fernández
,
G.
(
2007
).
Memory trace stabilization leads to large-scale changes in the retrieval network: A functional MRI study on associative memory
.
Learning & Memory
,
14
,
472
479
.
Thomson
,
D. M.
(
1972
).
Context effects in recognition memory
.
Journal of Verbal Learning and Verbal Behavior
,
11
,
497
511
.
Thomson
,
D. M.
, &
Tulving
,
E.
(
1970
).
Associative encoding and retrieval: Weak and strong cues
.
Journal of Experimental Psychology
,
86
,
255
262
.
Tiberghien
,
G.
, &
Cauzinille
,
E.
(
1979
).
Pre-decision and conditional search in long-term recognition memory
.
Acta Psychologica
,
43
,
329
343
.
Tibon
,
R.
,
Vakil
,
E.
,
Goldstein
,
A.
, &
Levy
,
D. A.
(
2012
).
Unitization and temporality in associative memory: Evidence from modulation of context effects
.
Journal of Memory and Language
,
67
,
93
105
.
Tulving
,
E.
, &
Thomson
,
D. M.
(
1973
).
Encoding specificity and retrieval processes in episodic memory
.
Psychological Review
,
80
,
352
373
.
Vaidya
,
C. J.
,
Zhao
,
M.
,
Desmond
,
J. E.
, &
Gabrieli
,
J. D. E.
(
2002
).
Evidence for cortical encoding specificity in episodic memory: Memory-induced re-activation of picture processing areas
.
Neuropsychologia
,
40
,
2136
2143
.
van Kesteren
,
M. T. R.
,
Rijpkema
,
M.
,
Ruiter
,
D. J.
, &
Fernández
,
G.
(
2010
).
Retrieval of associative information congruent with prior knowledge is related to increased medial prefrontal activity and connectivity
.
Journal of Neuroscience
,
30
,
15888
15894
.
Vilberg
,
K. L.
, &
Rugg
,
M. D.
(
2007
).
Dissociation of the neural correlates of recognition memory according to familiarity, recollection, and amount of recollected information
.
Neuropsychologia
,
45
,
2216
2225
.
Vilberg
,
K. L.
, &
Rugg
,
M. D.
(
2008
).
Memory retrieval and the parietal cortex: A review of evidence from a dual-process perspective
.
Neuropsychologia
,
46
,
1787
1799
.
Von Zerssen
,
G. C.
,
Mecklinger
,
A.
,
Opitz
,
B.
, &
Von Cramon
,
D. Y.
(
2001
).
Conscious recollection and illusory recognition: An event-related fMRI study
.
European Journal of Neuroscience
,
13
,
2148
2156
.
Wagner
,
A. D.
,
Shannon
,
B. J.
,
Kahn
,
I.
, &
Buckner
,
R. L.
(
2005
).
Parietal lobe contributions to episodic memory retrieval
.
Trends in Cognitive Sciences
,
9
,
445
453
.
Wang
,
J. X.
,
Rogers
,
L. M.
,
Gross
,
E. Z.
,
Ryals
,
A. J.
,
Dokucu
,
M. E.
,
Brandstatt
,
K. L.
, et al
(
2014
).
Targeted enhancement of cortical–hippocampal brain networks and associative memory
.
Science
,
345
,
1054
1057
.
Woolrich
,
M. W.
,
Ripley
,
B. D.
,
Brady
,
M.
, &
Smith
,
S. M.
(
2001
).
Temporal autocorrelation in univariate linear modeling of fMRI data
.
Neuroimage
,
14
,
1370
1386
.
Xue
,
G.
,
Dong
,
Q.
,
Chen
,
C.
,
Lu
,
Z.
,
Mumford
,
J. A.
, &
Poldrack
,
R. A.
(
2010
).
Greater neural pattern similarity across repetitions is associated with better memory
.
Science
,
330
,
97
101
.
Xue
,
G.
,
Dong
,
Q.
,
Chen
,
C.
,
Lu
,
Z.-L.
,
Mumford
,
J. A.
, &
Poldrack
,
R. A.
(
2013
).
Complementary role of frontoparietal activity and cortical pattern similarity in successful episodic memory encoding
.
Cerebral Cortex
,
23
,
1562
1571
.
Yonelinas
,
A. P.
(
2002
).
The nature of recollection and familiarity: A review of 30 years of research
.
Journal of Memory and Language
,
46
,
441
517
.
Yonelinas
,
A. P.
,
Otten
,
L. J.
,
Shaw
,
K. N.
, &
Rugg
,
M. D.
(
2005
).
Separating the brain regions involved in recollection and familiarity in recognition memory
.
Journal of Neuroscience
,
25
,
3002
3008
.
Yonelinas
,
A. P.
,
Widaman
,
K.
,
Mungas
,
D.
,
Reed
,
B.
,
Weiner
,
M. W.
, &
Chui
,
H. C.
(
2007
).
Memory in the aging brain: Doubly dissociating the contribution of the hippocampus and entorhinal cortex
.
Hippocampus
,
17
,
1134
1140
.
Zacks
,
J. M.
(
2008
).
Neuroimaging studies of mental rotation: A meta-analysis and review
.
Journal of Cognitive Neuroscience
,
20
,
1
19
.
Zacks
,
J. M.
,
Speer
,
N. K.
,
Swallow
,
K. M.
,
Braver
,
T. S.
, &
Reynolds
,
J. R.
(
2007
).
Event perception: A mind-brain perspective
.
Psychological Bulletin
,
133
,
273
293
.