Although the role of the hippocampus in spatial cognition is well accepted, it is unclear whether its involvement is restricted to the mnemonic domain or also extends to perception. We used fMRI to scan neurologically healthy participants during a scene oddity judgment task that placed no explicit demand on long-term memory. Crucially, a surprise recognition test was administered after scanning so that each trial could be categorized not only according to oddity accuracy but also according to subsequent memory. Univariate analyses showed significant hippocampal activity in association with correct oddity judgment, whereas greater parahippocampal place area (PPA) activity was observed during incorrect oddity trials, both irrespective of subsequent recognition performance. Consistent with this, multivariate pattern analyses revealed that a linear support vector machine was able to distinguish correct from incorrect oddity trials on the basis of activity in voxels within the hippocampus or PPA. Although no significant regions of activity were identified by univariate analyses in association with memory performance, a classifier was able to predict subsequent memory using voxels in either the hippocampus or PPA. Our findings are consistent with the idea that the hippocampus is important for processes beyond long-term declarative memory and that this structure may also play a role in complex spatial perception.
Since the discovery of place cells in the rat hippocampus (O'Keefe & Dostrovsky, 1971), there has been considerable data across species implicating a critical role for the hippocampus in spatial cognition (e.g., see Burgess, 2008; Muller, 1996). In humans, most of this evidence has been in the context of behavioral indices or experimental paradigms that explicitly require spatial memory, that is, the encoding and/or retrieval of spatial information. For instance, spatial navigation and maze tasks have been associated consistently with hippocampal activity (e.g., Morgan, Macevoy, Aguirre, & Epstein, 2011; Igloi, Doeller, Berthoz, Rondi-Reig, & Burgess, 2010; Doeller, King, & Burgess, 2008; Spiers & Maguire, 2006; Bohbot, Iaria, & Petrides, 2004; Ekstrom et al., 2003; Iaria, Petrides, Dagher, Pike, & Bohbot, 2003; Maguire et al., 1998; Ghaem et al., 1997), and critically, these tasks are known to be sensitive to hippocampal damage (e.g., Maguire, Nannery, & Spiers, 2006; Bohbot et al., 2004; Feigenbaum & Morris, 2004; Spiers, Burgess, Hartley, Vargha-Khadem, & O'Keefe, 2001; see also Rosenbaum, Winocur, Grady, Ziegler, & Moscovitch, 2007; Rosenbaum et al., 2000; Teng & Squire, 1999, for potential differences between old and new spatial navigation memories). In addition to this, studies in amnesic patients with hippocampal lesions and functional neuroimaging work in neurologically healthy individuals have underlined the importance of the hippocampus to recognition memory tasks involving scene stimuli or spatial information such as landmarks or spatial location/context (e.g., Bird, Vargha-Khadem, & Burgess, 2008; Ross & Slotnick, 2008; Taylor, Henson, & Graham, 2007; King, Burgess, Hartley, Vargha-Khadem, & O'Keefe, 2002; Burgess, Becker, King, & O'Keefe, 2001).
Recently, a number of studies have demonstrated that the involvement of the hippocampus in spatial cognition is not limited to the mnemonic domain and may extend to higher-order perception. These studies have employed visual discrimination tasks that do not place an obvious demand on memory processes because all the stimuli/information necessary to solve any given trial are presented to the participants throughout the duration of the trial (for a review, see Lee, Yeung, & Barense, 2012). One paradigm that has been used is different view scene oddity judgment. In this task, participants are shown two or three different views of the same virtual reality room and one different view of a different room on each trial and are instructed to select the odd one out. Hippocampal damage due to brain injury or neurodegenerative disease has been shown to significantly impair performance on this test but not oddity judgment tasks involving faces, objects, or simple visual features (e.g., color and size; Barense, Gaffan, & Graham, 2007; Lee, Buckley, et al., 2005, 2006). Furthermore, fMRI research has confirmed the involvement of the hippocampus when neurologically healthy participants are making odd-one-out decisions for scenes (Barense, Henson, Lee, & Graham, 2010; Lee, Scahill, & Graham, 2008). Collectively, these studies support the suggestion that the hippocampus is important for higher-order spatial processing regardless of whether mnemonic processing is involved (Gaffan, 2001, 2002).
The suggestion that the hippocampus may be important for spatial perception is inconsistent with the long-standing idea that the hippocampus and other surrounding medial-temporal lobe (MTL) structures function as an exclusive long-term memory system (Squire & Wixted, 2011; Squire, Stark, & Clark, 2004). Indeed, it is possible that findings demonstrating the involvement of the hippocampus in spatial discrimination tasks may, in fact, be accounted for by long-term memory processes. More specifically, although previous spatial discrimination tasks do not explicitly require participants to remember information across trials, it is, nevertheless, conceivable that accurate performance on these tasks is dependent on successful incidental encoding into long-term memory (Kim et al., 2011). Thus, according to this explanation, patients with hippocampal damage are poor at tasks such as scene oddity judgment due to an inability to remember the items on each trial, rather than an impairment in scene processing itself. Similarly, it is unclear whether hippocampal activity associated with scene oddity judgment (e.g., as revealed by fMRI) reflects incidental long-term memory encoding and not perceptual processes associated with selecting the odd scene on each trial.
The aim of this study, therefore, was to further elucidate the involvement of the hippocampus in scene discrimination and determine whether activity related to perceptual processing and mnemonic encoding can be disentangled in this structure. To achieve this, neurologically healthy participants were scanned using fMRI during a different view scene oddity judgment task and then given a surprise recognition memory test afterwards for the trials that were seen during scanning. Thus, each trial could be categorized according to oddity judgment accuracy as well as subsequent memory. Univariate and multivariate pattern analysis (MVPA) techniques were employed to investigate neural activity. If the involvement of the hippocampus in scene oddity judgment is related predominantly to mnemonic encoding, then significant increases in hippocampal activity in association with accurate oddity performance should only be observed for subsequently remembered and not forgotten trials. Alternatively, if perceptual and mnemonic process can be separated, then a distinct profile of activity (in terms of magnitude or multivariate patterns) should be associated with correct oddity trials irrespective of subsequent memory (and vice versa). For comparison, we also examined activity in the parahippocampal place area (PPA; Epstein & Kanwisher, 1998). The PPA is known to respond strongly to complex visual scenes in comparison with objects and faces and is believed to represent the layout of spatial scenes, predominantly in a viewpoint-specific manner (for a review, see Epstein, 2008). Moreover, it has been suggested that the PPA may play a primary role in scene perception and the encoding of spatial layouts rather than a more general mnemonic role in the encoding of spatial memories, for instance, as necessary for navigation (Epstein, Higgins, Jablonski, & Feiler, 2007; Epstein, Harris, Stanley, & Kanwisher, 1999). There is evidence to indicate that PPA dysfunction does not underlie hippocampal damaged patients' poor performance on spatial discrimination tasks such as scene oddity judgment (Lee & Rudebeck, 2010a); however, it is unclear what role this region may play in these tests. Earlier functional neuroimaging work has demonstrated significant PPA activity during scene oddity judgment relative to oddity judgment for faces, objects, and/or size (Barense et al., 2010; Lee et al., 2008). The relationship between oddity accuracy and PPA activity has, however, not been investigated, making it unclear whether greater PPA activity during scene oddity is associated with successful performance.
Fifteen neurologically healthy right-handed participants ranging in age from 21 to 28 years (nine women; mean age = 25.13 years, SD = 1.81) took part in this study. All participants possessed normal or corrected-to-normal vision and gave informed written consent after the nature of the study and its possible consequences were explained to them. This work received ethical approval from the Oxfordshire Research Ethics Committee A (Ref: 07/H0604/115).
Imaging data were acquired at the Oxford Centre for Clinical Magnetic Resonance Research using a 3-T Siemens (Malvern, PA) Tim TRIO MRI scanner and 12-channel head coil. Five 4-D data sets (four for the experimental task and one for a functional localizer task to identify the PPA) were obtained for each participant using an EPI pulse sequence to acquire T2*-weighted image volumes with BOLD contrast. For the experimental EPI data sets, partial brain volumes were acquired (Figure 1), each consisting of 27 interleaved axial-oblique slices centered on the hippocampus and angled 5° away from the axis of the hippocampus (in-plane resolution = 1.96 × 1.96 mm2, slice thickness = 2 mm with no gaps between slices, matrix = 112 × 112 × 27; repetition time [TR] = 2.16 sec, echo time [TE] = 27 msec, field of view [FOV] = 220 × 220 mm2, flip angle = 90°). Each experimental EPI run lasted 922.32 sec and consisted of 423 scans as well as four dummy scans acquired at the beginning to accommodate MRI saturation effects. For the functional localizer, whole-brain volumes were acquired, each consisting of 46 axial-oblique slices angled away from the eyeballs to prevent image ghosting (in-plane resolution = 3 × 3 mm2, slice thickness = 3 mm with no gaps between slices, matrix = 64 × 64 × 46, TR = 3 sec, TE = 30 msec, FOV = 192 × 192 mm2, flip angle = 89°). The functional localizer scan was 552 sec in duration, consisting of 180 scans and 4 initial dummy scans. A T1 structural scan was acquired for every subject using a 3-D MPRAGE sequence (voxel resolution = 1 × 1 × 1 mm, TR = 2.04 sec, TE = 4.7 msec, flip angle = 8°, FOV = 192 × 192, matrix size = 174 × 192 × 192). For the correction of EPI data distortion, magnetic field maps were also obtained using a dual-echo 2-D flash sequence (TR = 4.88 sec; TE1 = 5.19 msec, TE2 = 7.69 msec, flip angle = 90/180°, FOV = 224 × 224, matrix size = 64 × 64 × 44).
During fMRI scanning, a desktop computer and LCD projector (1024 × 768 pixel resolution) were used to project visual stimuli onto a white screen located at the head of the MRI subject bed. This screen could be seen via an angled mirror placed above the participant's eyes in the scanner. The participants held a button box in their right hand for making responses.
There were two components to the experimental task: (1) a spatial oddity judgment task that the participants carried out while undergoing fMRI and (2) a surprise subsequent recognition memory test that was administered outside the scanner following MRI data collection (Figure 2). Both components were programmed and presented using the Presentation software package (Neurobehavioral Systems, Inc., Albany, CA), and all stimuli were created using a commercially available computer game (Deus Ex; Ion Storm, Austin, TX) and a freeware editor (Deus Ex SDK v1112f). The spatial oddity judgment task was an adaptation of a paradigm used in previous work (Lee, Buckley et al., 2005, 2006). On each trial, participants were presented with three color images of 3-D virtual reality indoor scenes. Two of these images were different views of the same scene, whereas the third image was a different view of another scene that was similar in appearance (e.g., same wall/floor/ceiling textures and general spatial layout) and differed only by the dimensions or placement of one or two aspects of the scene such as a window, wall, or room cavity. Each set of three scenes was presented for 5.5 sec, after which a fixation cross appeared for 2 sec. The participants were asked to determine which scene was the odd one out and to indicate their responses during the presentation of the fixation cross using one of three prespecified buttons on the response box. To discourage guessing, the participants were asked to press a fourth button whenever they were unsure. These trials were subsequently categorized as incorrect trials. A jittered intertrial interval (ITI) with mean of 10 sec followed each fixation cross/response period, and there were 50 trials in each EPI run, leading to 200 trials across all four runs. The order of trials was pseudorandomized in each run, and the four run trial orders were counterbalanced across participants. In an effort to provide balanced classes for MVPA, task difficulty was determined via behavioral piloting, with the aim of participants achieving an approximately equal number of correct and incorrect oddity judgment responses (50% each), but still performing above chance (33%).
The postscanning recognition memory test was administered on a 15-in. screen laptop computer in a room adjacent to the scanner suite approximately 10 min after the last scan was completed and allowed us to classify each trial during scanning as either subsequently remembered or forgotten. In this task, the participants were shown all of the 200 oddity judgment trials that were presented during scanning randomly intermixed with 100 foil trials that had not been presented before (i.e., three images were presented on each recognition memory trial). Crucially, these foil trials contained the same wall/floor/ceiling textures and colors as those used in the scanning trials to emphasize memory on the basis of spatial layout rather than individual textures or colors. The participants were instructed to indicate whether each presented oddity trial was old (seen during scanning) or new (never seen before) using prespecified keys on the laptop keyboard. If the trial was considered old, participants were also asked whether they recollected seeing the trial during scanning (i.e., recall of specific contextual information) or whether the trial was familiar (i.e., a “feeling of knowing”). Given the relatively low memory performance of the participants on the recognition test, however (see Behavioral results), recollect and familiar trials were combined into a single category of remembered trials in all subsequent imaging analyses.
Functional Localizer Task
To locate the PPA in our participants, 15-sec blocks of grayscale images of scenes, objects, faces, scrambled scenes, scrambled objects, and scrambled faces were presented during a scanning run independent of the experimental task. Each block consisted of 20 images, and each image was presented for 300 msec (ITI = 450 msec). There were four blocks of each stimulus type, and the order of the different blocks was pseudorandomized. The participants were instructed to press a prespecified button on a response box held in the right hand when they noticed a repeating image (i.e., 1-back task). A fixation cross was presented at the start of the task and then every three blocks throughout (nine total times 15 sec).
Imaging Data Analysis
Preprocessing and univariate statistical analyses were carried out using the FMRIB Software Library (www.fmrib.ox.ac.uk/fsl/). MVPA analyses were conducted with a custom MVPA framework (www.people.inf.ethz.ch/bkay/downloads/) written in MATLAB (www.mathworks.com/products/Matlab/) in conjunction with functions from SPM8 (www.fil.ion.ucl.ac.uk/spm/) for image handling and the Princeton MVPA toolbox (www.csbmb.princeton.edu/mvpa/).
Experimental Data Image Preprocessing
Preprocessing of EPI data involved (1) realignment of all images to the central volume using rigid body registration; (2) unwarping using the acquired field maps to correct for distortions due to magnetic field inhomogeneities; (3) segmentation of brain matter from nonbrain matter using a mesh deformation approach; (4) spatial smoothing using a relatively small Gaussian kernel of FWHM 4.0 mm to increase signal-to-noise ratio but maintain spatial resolution across the MTL structures; (5) grand mean intensity normalization using a single multiplicative factor; (6) high-pass temporal filtering using Gaussian-weighted least-squares straight line fitting (sigma = 90 sec); and (7) registration to standard Montreal Neurological Institute (MNI) space using the MNI 152 template as well as nonlinear registration (Jenkinson, Bannister, Brady, & Smith, 2002; Jenkinson & Smith, 2001). Notably, the size of our smoothing kernel is in keeping with existing studies that have either omitted or applied minimal spatial smoothing to EPI data sets comprising high-resolution voxels (≤2 mm3; Carr, Rissman, & Wagner, 2010).
Experimental Data Univariate Analysis
A general linear model (GLM) was fit in prewhitened space for each EPI session for each subject. This GLM consisted of four explanatory variables (EVs) and their temporal derivatives (Woolrich, Ripley, Brady, & Smith, 2001), namely, (1) correct oddity and subsequently remembered, (2) correct oddity and subsequently forgotten, (3) incorrect oddity and subsequently remembered, and (4) incorrect oddity and subsequently forgotten. Each EV was convolved with a double-gamma hemodynamic response function and temporally filtered with the same high-pass filter as that used for preprocessing. Consequently, a parameter estimate image was created for each of the four EVs, as well as a number of planned contrasts that were set up to investigate differential activity between the EVs. These contrasts included (1) the main effect of oddity judgment ([correct oddity and subsequently remembered + correct oddity and subsequently forgotten] vs. [incorrect oddity and subsequently remembered + incorrect oddity and subsequently forgotten]); (2) the main effect of subsequent memory ([correct oddity and subsequently remembered + incorrect oddity and subsequently remembered] vs. [correct oddity and subsequently forgotten + incorrect oddity and subsequently forgotten]); and (3) the interaction between oddity judgment and memory ([correct oddity and subsequently remembered − incorrect oddity and subsequently remembered] vs. ([correct oddity and subsequently forgotten − incorrect oddity and subsequently forgotten]). To examine activity across all four EPI sessions for each participant, a second-level analysis was carried out for every participant using a standard weighted fixed effects model with each EPI session as a higher level regressor. The output from these analyses were then fed into a third higher-level group analysis with a Bayesian mixed effects model, accounting for both within-session fixed effects variance and between-session/subject random effects variance (Woolrich et al., 2009).
Because of our a priori hypotheses concerning the hippocampus and the role of the PPA in scene processing (Epstein, 2008), two separate bilateral masks of the hippocampus (2882 voxels) and PPA (1031 voxels; see ROIs) were used to constrain our findings (i.e., small volume correction, s.v.c.). Within the hippocampus, all clusters of activity that surpassed a threshold of p < .001 (uncorrected for multiple comparisons) and were larger than 7 voxels in size were identified as significant. For the PPA, clusters of activity at p < .001 (uncorrected for multiple comparisons) of at least 5 voxels in size were considered significant. These thresholds are equivalent to a map-wise false positive rate of p < .05 (calculated with the AFNI programs “3dFWHM” to estimate the spatial correlation of the residuals in each individual participant and “AlphaSim” to conduct 5000 Monte Carlo simulations). This approach to false-positive correction was adopted for the experimental EPI data because limited spatial smoothing (FWHM = 4 mm) was applied, resulting in the final smoothness of the data failing to meet the conditions of Random Field Theory as implemented for multiple comparison correction (Nichols & Hayasaka, 2003; Petersson, Nichols, Poline, & Holmes, 1999). All coordinates (x, y, z) are reported in MNI space.
Experimental Data Multivariate Analysis
Parameter estimate volumes were created for each participant by fitting a GLM in prewhitened space for each EPI session. Each trial (50 per EPI session) was specified as an individual boxcar EV and convolved with a double-gamma hemodynamic function and temporally filtered with the same high-pass filter as that used for preprocessing. The four EPI sessions for each subject were then concatenated to create a single volume of 200 parameter estimate maps. These maps were processed further by standardizing the beta coefficients within each voxel and by normalizing all trials such that all voxels would form a vector of unit length. The final images were used in two classification analyses. The goal of the first analysis was to assess the context-dependent amount of information that could be extracted from activity patterns in trial-wise images. A linear support vector machine (LIBSVM; Chang & Lin, 2011) was trained and tested in a leave-one-out cross-validation scheme to decode (i) whether a correct oddity decision had been made, irrespective of subsequent memory; (ii) whether a trial was subsequently remembered, irrespective of oddity accuracy; and (iii) whether a correct oddity decision was made, considering subsequently remembered trials only. All analyses were based on activity in voxels in the hippocampus or PPA (see ROIs). Within each cross-validation fold, the training set was balanced by oversampling the minority class, and nested cross-validation on the training data was used to optimize the regularization hyperparameter C. To minimize the potentially confounding effects of an autocorrelated signal, the two trials surrounding the current test trial were excluded from the training set. Subject-wise classification accuracies were then submitted to a one-sample one-tailed t test, representing random effects inference to determine whether classification performance at the group level was significantly above chance (50% accuracy).
The aim of the second multivariate analysis was to study the spatial deployment of jointly informative voxels. Here, we embedded the above SVM into a searchlight procedure (Kriegeskorte, Goebel, & Bandettini, 2006) by passing a local sphere (radius = 3 voxels) across the ROIs (see below) and evaluating the classification performance afforded by voxels contained in each sphere. This procedure yielded a map of t scores, expressing how significantly each voxel contributed to a distributed encoding of a cognitive state across the whole group of participants.
It is worth highlighting that, across both classification analyses, it is the statistical significance, and not the absolute magnitude, of prediction performance that is of primary interest. The goal of this study was to investigate the relationship between brain activity and cognitive function, which is evidenced by the significance with which the group classification accuracy is above chance, not by its magnitude, because the significance takes into account both mean and variability in the group.
The hippocampal mask for both univariate and multivariate analyses was created using the Harvard-Oxford Cortical and Subcortical Structural Atlases. The PPA mask was defined on the basis of the functional localizer data. Each subject's functional localizer data were first preprocessed in an identical manner to the spatial oddity fMRI data, except for the use of a larger smoothing kernel (FWHM = 8 mm) and high-pass temporal smoothing at 120 sec. A GLM was fit to prewhitened data for each subject, with one EV plus its temporal derivative (Woolrich et al., 2001) for each stimulus category. Each EV was convolved with a double-gamma hemodynamic function and temporally filtered with the same high-pass filter as that used for preprocessing. The contrast “scenes + (faces − objects)” was used to identify the PPA. For the multivariate statistical analysis of the spatial oddity data, subject-specific PPA masks were created by identifying voxels that survived a statistical threshold of p < .001 (uncorrected for multiple comparisons) within the posterior parahippocampal gyrus as defined by the Harvard-Oxford Cortical atlas. For the univariate statistical analysis of the spatial oddity data, the group PPA mask was created by feeding the individual functional localizer data analyses into a second-level group analysis with a mixed effects model. Voxels surviving a random field-based voxel-wise intensity threshold of p < .05 (corrected for multiple comparisons) within the posterior parahippocampal gyrus as defined by the Harvard-Oxford Cortical atlas were incorporated into the group PPA mask.
On average, the proportion correct on the spatial oddity judgment trials in the scanner was 0.562 (SD = 0.057). Perhaps owing to the difficulty of this task, subsequent memory performance was relatively poor. The mean proportion of recollect hit (H) responses was 0.208 (SD = 0.119), with a recollect false alarm (FA) rate of 0.052 (SD = 0.092). Although the mean proportion of familiar H responses was higher at 0.402 (SD = 0.085), this was tempered by a greater proportion of familiar FA responses of 0.293 (SD = 0.074). Consequently, both recollect and familiar responses were combined into a single category of remember responses, giving P(H) = 0.610 (SD = 0.097) and P(FA) = 0.345 (SD = 0.097; mean d′ = 0.710, SD = 0.327). There was no significant correlation between oddity accuracy and subsequent memory performance as measured by d′ (r = .42, p = .12). Figure 3 illustrates the mean proportion of each trial type as categorized according to oddity accuracy and subsequent memory.
Univariate Analysis Results
The subtraction “correct oddity minus incorrect oddity” revealed three significant clusters of activity in the hippocampus: one in the right posterior hippocampus (max Z = 3.62; 32, −36, −8; 13 voxels) and two clusters more anteriorly in the right (max Z = 3.84; 22, −10, −20; 36 voxels) and left (max Z = 3.56; −16, −14, −22; 16 voxels) hemispheres (see Figure 4A). Figure 4B illustrates the mean percent signal change at the most significant voxel in each of these clusters of activity. As seen in this graph, activity for these voxels in all three clusters was greater for correct versus incorrect oddity trials for both subsequently remembered and forgotten trials. Correct oddity judgment was not associated with significant activity in the PPA at the chosen statistical threshold for this region (p < .001, uncorrected, cluster size ≥ 5 voxels, equivalent of map-wise false positive rate of p < .05). Even when a liberal threshold of p < .05, uncorrected, was applied to explore the data further, no clusters of activity were observed in the PPA in either hemisphere.
The reverse subtraction of “incorrect oddity minus correct oddity” revealed no significant clusters of activity in the hippocampus or PPA at p < .001, uncorrected, although there was one voxel that survived this threshold in the left PPA (Z = 3.12; −28, −48, −14). Applying a threshold of p < .05, uncorrected, to investigate the data further did not yield any clusters of activity in the hippocampus. In the PPA, however, there was one large cluster of activity in the left PPA (max Z = 3.12; −28, −48, −14; 179 voxels) and one smaller cluster of activity in the right PPA (max Z = 2.52; 28, −46, −16; 23 voxels; see Figure 4C). Use of the 3dFWHM and AlphaSim programs revealed that the left PPA cluster survived a map-wise false positive rate of p ≤ .0002, whereas the right PPA cluster possessed a map-wise false positive rate of p < .3. Figure 4D depicts the mean percent signal change at the most significant voxel in left PPA cluster and illustrates that there was greater activity for incorrect versus correct oddity trials for subsequently remembered and forgotten trials (note that the relatively smaller fluctuations in BOLD signal at this voxel compared with those observed in the hippocampus reflect the lower level of statistical significance of the difference in activity between correct vs. incorrect oddity judgment in the left PPA).
The subtraction “remembered minus forgotten” did not reveal any significant clusters of activity in the hippocampus or the PPA at p < .001, uncorrected. To explore the data further, a threshold of p < .05, uncorrected, revealed one cluster of activity in the right hippocampus (max Z = 2.54; 20, −22, −14; 13 voxels, equivalent map-wise false positive rate p < .3) and one cluster of activity in the right PPA (max Z = 2.09; 22, −36, −20; 13 voxels, equivalent map-wise false positive rate p < .5). The reverse subtraction “forgotten minus remembered” did not reveal any significant clusters of activity in the hippocampus or the PPA at p < .001, uncorrected. A threshold of p < .05, uncorrected, revealed clusters of activity in the hippocampus only (Cluster 1 max Z = 2.41; −16, −4, −24; 17 voxels; Cluster 2 max Z = 2.50; 30, −40, −2; 13 voxels; Cluster 3 max Z = 2.34; −30, −22, −24; 10 voxels), although none of these survived a map-wise false-positive rate of p < .05.
Finally, the interaction contrast ([correct oddity and subsequently remembered − incorrect oddity and subsequently remembered] − [correct oddity and subsequently forgotten − incorrect oddity and subsequently forgotten]) revealed no clusters of activity in the hippocampus or the PPA at p < .001, uncorrected, or p < .05, uncorrected. The reverse contrast did not reveal any regions of activity in either region at p < .001, uncorrected, although a liberal threshold (p < .05, uncorrected) yielded clusters in the hippocampus bilaterally (Cluster 1 max Z = 2.45; −28, −26, −14; 55 voxels; Cluster 2 max Z = 2.49; 24, −10, −24; 17 voxels, Cluster 3 max Z = 2.39; −16, −38, −6; 11 voxels) and the right (max Z = 2.25; 32, −58, −10; 15 voxels) and left PPA (max Z = 2.17; −24, −60, −10; 104 voxels). None of these clusters, however, survived a map-wise false positive rate of p < .05.
Multivariate Analysis Results
Using a linear SVM on trial-specific parameter estimate images, we obtained a mean classification accuracy for correct versus incorrect oddity judgment, irrespective of subsequent memory, of 56.93% (SD = 6.08) in the hippocampus and 55.77% (SD = 4.73) in the PPA. One-sample t tests revealed that both of these were significantly above chance (hippocampus: t(14) = 4.42, p < .001, one-tailed; PPA: t(14) = 4.72, p < .001, one-tailed), with a paired-sample t test revealing no significant difference between the classification accuracies in the hippocampus and PPA (t(14) = 0.75, p = .47, two-tailed). Mean classification accuracies for subsequently remembered versus forgotten trials irrespective of oddity accuracy were 53.47% (SD = 5.24) in the hippocampus and 56.94% (SD = 7.17) in the PPA. One-sample t tests revealed that both of these were significantly above chance (hippocampus: t(14) = 2.57, p = .011, one-tailed; PPA: t(14) = 3.75, p = .0010, one-tailed), with a significant difference between the two (t(14) = 2.46, p = .028, two-tailed).
To examine potential interaction effects, we investigated classification accuracy for correct versus incorrect oddity judgment for subsequently remembered trials only. We found that classification performance was very similar to that obtained when both subsequently remembered and forgotten trials were considered together: 56.41% (SD = 6.51) in the hippocampus and 55.45% (SD = 7.52) in the PPA. Both of these accuracies were significantly above chance (hippocampus: t(14) = 3.81, p = .0010, one-tailed; PPA: t(14) = 2.80, p = .0070, one-tailed), with no significant difference between the two (t(14) = 0.50, p = .63, two-tailed).
Thresholding the hippocampal searchlight map for correct versus incorrect oddity classification (irrespective of subsequent memory) at p < .001, uncorrected, revealed two clusters of voxels in the left hemisphere (Cluster 1 max t = 5.11; −20, −18, −20; 7 voxels; Cluster 2 max t = 4.95; −22, −26, −18; 22 voxels; both clusters survive mapwise false positive rate of p < .05; Figure 5). No voxels were identified at p < .001, uncorrected, for the PPA searchlight map for the same classification decision. Using a more lenient threshold of p < .025, uncorrected, to explore the map further identified two separate clusters in the left PPA, although neither survived a mapwise false positive rate of p < .05 (Cluster 1 max t = 3.75; −20, −48, −14; 7 voxels; Cluster 2 max t = 3.25; −24, −48, −4; 22 voxels).
For the remembered versus forgotten classification decision (irrespective of oddity accuracy), a threshold of p < .001, uncorrected, only identified a single voxel in the hippocampal searchlight map (t = 3.81; 18, −12, −14). This grew to a larger cluster when a lower threshold of p < .025, uncorrected, was applied (15 voxels), with an additional group of voxels in the right hippocampus (max t = 3.71; 14, −38, −2; 8 voxels). For the PPA searchlight map, a threshold of p < .001, uncorrected, identified two groups of two voxels in the left hemisphere within 1 voxel of each another (Cluster 1 max t = 5.71; −24, −46, −16; 2 voxels; Cluster 2 max t = 4.44; −24, −16, −12; 2 voxels). These clusters merged into a larger cluster at p < .025, uncorrected (max t = 5.71; −24, −46, −16; 27 voxels), with a second cluster also identified in the right hemisphere (max t = 3.61; 26, −54, −18; 8 voxels). None of these clusters, however, survived a mapwise false positive rate of p < .05.
A number of recent studies have demonstrated that the hippocampus is involved in complex spatial discrimination tasks such as oddity judgment for scene images presented from different viewpoints (Barense et al., 2010; Lee, Buckley, et al., 2005, 2006; Lee et al., 2008). There has been uncertainty over the interpretation of these data: Whereas some argue that these findings reflect the well accepted role of the hippocampus in long-term declarative memory (e.g., Kim et al., 2011; Squire & Wixted, 2011), others propose that they support a role for the hippocampus in complex spatial perception (e.g., Lee et al., 2012; Murray, Bussey, & Saksida, 2007). By using fMRI to scan neurologically healthy participants during a different view scene oddity judgment task and assessing recognition memory for individual trials after scanning, we have provided evidence consistent with the latter suggestion. Univariate analyses revealed significantly greater hippocampal activity for correct versus incorrect scene oddity trials, and critically, this difference was not dependent on whether the trials were subsequently remembered. Indeed, there was no interaction effect between oddity accuracy and subsequent memory. Using MVPA, we found that a linear SVM was significantly above chance when classifying trials according to oddity accuracy (irrespective of subsequent memory) on the basis of hippocampal voxels. Moreover, a searchlight analysis identified two clusters of hippocampal voxels that contributed significantly to this classification across participants.
The fact that hippocampal activity was greater for correct oddity trials in comparison with incorrect trials even when those trials were subsequently forgotten suggests that successful episodic memory encoding is not necessary for accurate scene oddity judgment. Our findings point, therefore, toward a role for the hippocampus in spatial processing beyond the domain of long-term declarative memory. Given a number of studies implicating the involvement of the hippocampus in working memory for relational/spatial information (e.g., Cashdollar et al., 2009; Hannula & Ranganath, 2008; Hartley et al., 2007; Hannula, Tranel, & Cohen, 2006; Olson, Page, Moore, Chatterjee, & Verfaellie, 2006), it is plausible that our observation of significant hippocampal activity reflects the spatial working memory processes that are necessary for successful scene oddity performance (e.g., maintaining and comparing multiple images in working memory). Although we do not necessarily disagree with this interpretation, we have reasons to suggest that our data are indicative of a role for the hippocampus in high-order spatial perception (for further discussion, see Lee et al., 2012). First, in a recent fMRI study, we found that working memory demand modulated hippocampal activity when complex scene stimuli were involved (i.e., 3-D virtual reality rooms) but not when images possessing relatively simple spatial information were presented (i.e., a 2-D spatial array of shapes; Lee & Rudebeck, 2010b). Thus, the hippocampus does not appear to play a general role in spatial working memory and the type of spatial information involved is a critical factor in determining hippocampal involvement. Second, recent eye movement data suggest that a primary working memory impairment is unlikely to account entirely for the poor performance of patients with hippocampal damage on different view scene oddity judgment (Erez, Lee, & Barense, submitted). In brief, the ratio of saccades that hippocampal lesioned patients produced within a given scene in comparison with those across all scenes presented on a trial was similar to that of matched controls, suggesting intact trans-saccadic working memory in these patients.
It is important to note that although we are proposing that the hippocampus may play a role in complex scene perception, we are not suggesting that this structure is not critical for mnemonic processing. Instead, the hippocampus contributes to both perceptual and mnemonic processes providing that these processes place a sufficient demand on the types of representations for which the hippocampus is critical. As discussed in detail in a recent review (Lee et al., 2012), we suggest that the hippocampus represents distinct, complex conjunctions of spatial and temporal information, in keeping with its role in spatial cognition (e.g., Burgess, 2008; O'Keefe & Nadel, 1978) and processing sequences and temporal context (e.g., MacDonald, Lepage, Eden, & Eichenbaum, 2011; Tubridy & Davachi, 2011; Jenkins & Ranganath, 2010; Kesner, Gilbert, & Barua, 2002). Although the precise composition of these representations remains to be clarified, it is plausible that they underlie successful performance on spatial discrimination tasks that require viewpoint independent processing (Lee, Buckley, et al., 2005) or discriminating highly similar scene images (Lee, Bussey, et al., 2005), as well as a range of memory tasks (declarative/nondeclarative; long-term/working memory) involving spatial scene stimuli (e.g., Bird et al., 2008; Hartley et al., 2007; Graham et al., 2006).
Because of the combination of having to maximize the number of scene oddity trials and implement a relatively long ITI for classifier training, we did not have sufficient scan time to incorporate oddity trials involving other types of stimuli (e.g., objects). Although our hippocampal findings pertaining to correct scene oddity judgment may reflect the recruitment of processes that are not stimulus-specific such as match–mismatch detection (e.g., Kumaran et al., 2007; Vinogradova, 2001), we suggest that this is unlikely to account entirely for our data. Our present findings converge with previous fMRI work in which greater hippocampal activity has been observed during oddity judgment for scenes in comparison with objects and/or faces (Barense et al., 2010; Lee et al., 2008). In addition to this, patients with selective hippocampal damage have been shown to demonstrate scene, but not face or object, oddity deficits (Barense et al., 2007; Lee, Buckley, et al., 2005).
As a comparison with the hippocampus, the current study also examined the involvement of the PPA to different view scene oddity judgment. Consistent with the finding that hippocampal damaged patients who are significantly impaired on this task do not appear to suffer from dysfunction to the PPA (Lee & Rudebeck, 2010a), we found that correct oddity trials were not associated with significantly greater activity in the PPA in comparison with incorrect trials. Interestingly, however, the reverse comparison revealed PPA activity in association with incorrect trials, and furthermore, a classifier was significantly above chance when distinguishing correct and incorrect trials on the basis of voxels within the PPA. A searchlight analysis did not, however, identify any clusters of PPA voxels at p < .001, uncorrected, suggesting that, dissimilar to the hippocampus, there were no voxels within the PPA that were consistently associated with significantly above chance classification across all participants. Although it cannot be determined for certain from our current data why greater PPA activity was associated with incorrect spatial oddity trials, some insight may be gleaned by considering the differential contributions of the hippocampus and PPA to spatial cognition. In keeping with its suggested function as a cognitive map (e.g., O'Keefe & Nadel, 1978) and contribution to spatial navigation (e.g., Maguire et al., 2006; Iaria et al., 2003; Maguire et al., 1998), there is considerable data suggesting that the hippocampus is critical for viewpoint independent spatial processing (e.g., Lavenex, Amaral, & Lavenex, 2006; King et al., 2002; Holdstock et al., 2000; Morris, Schenk, Tweedie, & Jarrard, 1990). In contrast, the PPA and the wider surrounding parahippocampal cortex appear to be involved in viewpoint dependent processing (e.g., Bohbot & Corkin, 2007; Epstein, Graham, & Downing, 2003). Because our scene oddity judgment task required participants to process the locations and dimensions of spatial features across multiple viewpoints, it is not surprising that significantly greater hippocampal activity was observed for correct versus incorrect trials. It is conceivable, however, that an error was more likely to occur when participants failed to process the absolute properties of the spatial features in each scene image (i.e., viewpoint independent processing) and made a decision based on the relative spatial properties in each image (i.e., viewpoint dependent processing), leading to greater PPA activity in association with incorrect oddity trials. For instance, in Figure 1A, a participant could make a decision on the basis of the absence of the yellow grated window and erroneously choose the bottom right image. In contrast, if the effect of viewpoint was taken into account, then a participant would realize that the window is being obscured in the viewpoint depicted in the bottom right image and that, in fact, the bottom left image is the odd-one-out due to a shifting in the position of the pillar closest to the viewer.
Surprisingly, univariate analyses revealed no significant differences in activity in the hippocampus or PPA between subsequently remembered and forgotten stimuli, in contrast to a number of existing studies that have found subsequent memory effects in the MTL for spatial scene stimuli (e.g., Qin, van Marle, Hermans, & Fernandez, 2011; Poppenk, McIntosh, Craik, & Moscovitch, 2010; Preston et al., 2010; Brewer, Zhao, Desmond, Glover, & Gabrieli, 1998). One potential explanation for this is that the high difficulty of our spatial oddity judgment task compromised incidental memory encoding, leading to relatively poor subsequent memory performance (P(H) − P(FA) = 0.27; mean d′ = 0.71). Indeed, there was a low proportion of recollect responses in the subsequent recognition task and even the higher proportion of familiar responses was tempered by a greater FA rate. Perhaps reflecting the greater sensitivity of multivariate analyses over a univariate approach, we found that a classifier was significantly above chance when classifying remembered versus forgotten trials on the basis of either hippocampal or PPA voxels (irrespective of oddity accuracy), with mean accuracy for the latter being significantly greater than that for the former. Searchlight analyses for consistently informative voxels failed, however, to reveal any clusters that survived a corrected statistical threshold. Our classification findings are consistent with a recent study by Watanabe and colleagues (2011) in which an SVM was also found to be able to predict subsequent recognition on the basis of fMRI data collected during an incidental episodic memory encoding task, despite the absence of any significant univariate findings the MTL (see also Kuhl, Rissman, & Wagner, 2012; Xue et al., 2010). Notably, there are a number of methodological differences between this study and the current investigation. For instance, whereas we adopted an anatomical ROI approach to feature selection for classifier training (i.e., all voxels within the hippocampus/PPA were selected), Watanabe et al. (2011) used the searchlight procedure (Kriegeskorte et al., 2006) to identify clusters of interest across the entire MTL in each subject for classifier training and prediction. Using this method, Watanabe and colleagues were able to identify clusters of voxels associated with a high degree of classification accuracy for subsequent memory in each subject in the hippocampus and/or the parahippocampal cortex, which led to significant prediction accuracies in almost all participants. Our study demonstrates that a searchlight approach to feature selection is not necessary and that the use of anatomically defined ROIs can also yield significant prediction accuracies for subsequent memory based on encoding-related fMRI activity.
It is interesting to note that our group searchlight analyses only revealed significant clusters of informative voxels (p < .05 corrected) for oddity accuracy classification in the hippocampus, and no significant clusters (at a corrected threshold) were found in association with subsequent memory classification. This difference may reflect the variability of the cognitive processes underlying correct spatial oddity judgment and successful subsequent memory. For example, because successful performance on the scene oddity task is dependent solely on the participants' ability to detect a difference in spatial layout, the cognitive processes contributing to a correct oddity decision were likely to be similar across participants, leading to a more consistent pattern of hippocampal activity across all subjects in association with accurate oddity performance. In contrast, a wider range of processes/factors were likely to have contributed to successful recognition memory (e.g., recollection, familiarity, memory strength, etc.), potentially leading to a greater variability in the pattern of activity across the participants pertaining to successful subsequent memory.
Finally, our finding that a classifier was significantly above chance in distinguishing remembered and forgotten trials on the basis of PPA voxels suggests that an important role for this region in mnemonic processing (e.g., as required for spatial navigation) cannot be ruled out. Although parahippocampal cortex activity has been shown to be predictive of subsequent memory in univariate fMRI studies involving scene stimuli or contextual information (e.g., Diana, Yonelinas, & Ranganath, 2010; Hayes, Nadel, & Ryan, 2007; Brewer et al., 1998), there has been some uncertainty as to whether the PPA itself contributes to long-term memory. For example, one PET study demonstrated the involvement of the PPA in recognizing previously studied scenes (Kohler, Crane, & Milner, 2002), whereas other fMRI studies have failed to report strong changes in PPA activity when the familiarity of the scenes presented during scanning is manipulated (Epstein et al., 1999, 2007). The latter work has led to the proposal that, although the PPA contributes primarily to scene perception, more anterior parahippocampal cortex may subserve mnemonic processing (Epstein, 2008). Our data suggest that the PPA may, in fact, be important for spatial memory and that this involvement may be reflected in local patterns, rather than absolute levels, of neuronal activity as identified by multivariate fMRI analyses.
In conclusion, we have used a combination of univariate and multivariate analyses of fMRI data collected during scene oddity judgment to demonstrate that the hippocampus is important for successful performance independent of its role in episodic memory. Our findings are consistent with the suggestion that the hippocampus is involved in processes beyond long-term declarative memory and that this structure may be involved in processing complex representations of spatial information.
We thank all participants for their time, the University of Oxford Centre for Clinical Magnetic Resonance Research staff for their assistance, and Danielle Douglas and Hilary Watson for helpful discussion. This work was supported by the Wellcome Trust (#082315 ACHL) and the Natural Science and Engineering Research Council of Canada (#412309-2011, #402651-2011 ACHL).
Reprint requests should be sent to Andy C. H. Lee, Department of Psychology, University of Toronto Scarborough, 1265 Military Trail, Toronto M1C 1A4, Canada, or via e-mail: firstname.lastname@example.org.