Abstract

Induced forgetting occurs when accessing an item in memory appears to harm memory representations of categorically related items. However, it is possible that the actual memory representations are unharmed. Instead, people may just change how they make decisions. Specifically, signal detection theory suggests this apparent forgetting may be due to participants shifting their decision criterion. Here, we used behavioral and electrophysiological measures to determine whether induced forgetting is truly due to changes in how items are represented or simply due to a shifting criterion. Participants' behavior and brain activity showed that induced forgetting was due to changes in the strength of the underlying representations, weighing against a criterion shift explanation of induced forgetting.

INTRODUCTION

In a variety of research domains, researchers have proposed that simple shifts in decision threshold may underlie seemingly complex cognitive phenomena (Wixted, 2016, 2020; Witt, Taylor, Sugovic, & Wixted, 2015; Wixted & Stretch, 2000). According to signal detection theory (Macmillan & Creelman, 2004; Green & Swets, 1966), people's old–new recognition judgments are determined by a decision-making framework that sets a threshold for the amount of memory strength that a memory representation must have to prompt them to respond that it is remembered (Wickelgren, 1968). The decision threshold can be illustrated as a vertical line intersecting distributions of activity elicited by old items and new items in a recognition memory task (Figure 1, left). Items to the right of the line will be responded to as old, whereas items on the left of the line will be responded to as new. Importantly, behavioral responses are driven by both the memory strength of any given item and the placement of the criterion. This study measured brain activity during a laboratory-forgetting paradigm to determine whether criterion shifting might underlie effects that we often attribute to differences in memory fidelity.

Figure 1.

(Left) Typical signal detection theory illustration of signal (target) and noise (distractor) distributions, intersected by the criterion on a familiarity scale. Participants respond to all signals above (to the right of) the criterion as old, resulting in many hits and some false alarms. Participants respond to all signals below (to the left of) the criterion as new, resulting in many correct rejections and some misses. Note that recognition performance may be better illustrated by an unequal variance model in which the signal is more variable than the noise (Glanzer, Kim, Hilford, & Adams, 1999; Ratcliff, Sheu, & Gronlund, 1992). Nevertheless, leaders in the field agree that equal variance is a reasonable representation of signal detection predictions (Singer & Wixted, 2006; Wixted & Stretch, 2004; Hicks & Marsh, 1998; Hirshman, 1995). (Right) Criterion shift hypothesis explanation of induced forgetting. The left distribution represents new items and the right distribution represents old items. The pink line represents the hypothetical criterion for related items. The blue line represents the hypothetical criterion for baseline items. The location of these criteria relative to one another accounts for more hits for baseline items relative to related items. The placement of these two lines, representing criterion shifting, is how a criterion shift could underlie behavioral evidence of forgetting rather than a true difference in memory strength. The criterion shift hypothesis predicts that participants should false alarm to more baseline than related items.

Figure 1.

(Left) Typical signal detection theory illustration of signal (target) and noise (distractor) distributions, intersected by the criterion on a familiarity scale. Participants respond to all signals above (to the right of) the criterion as old, resulting in many hits and some false alarms. Participants respond to all signals below (to the left of) the criterion as new, resulting in many correct rejections and some misses. Note that recognition performance may be better illustrated by an unequal variance model in which the signal is more variable than the noise (Glanzer, Kim, Hilford, & Adams, 1999; Ratcliff, Sheu, & Gronlund, 1992). Nevertheless, leaders in the field agree that equal variance is a reasonable representation of signal detection predictions (Singer & Wixted, 2006; Wixted & Stretch, 2004; Hicks & Marsh, 1998; Hirshman, 1995). (Right) Criterion shift hypothesis explanation of induced forgetting. The left distribution represents new items and the right distribution represents old items. The pink line represents the hypothetical criterion for related items. The blue line represents the hypothetical criterion for baseline items. The location of these criteria relative to one another accounts for more hits for baseline items relative to related items. The placement of these two lines, representing criterion shifting, is how a criterion shift could underlie behavioral evidence of forgetting rather than a true difference in memory strength. The criterion shift hypothesis predicts that participants should false alarm to more baseline than related items.

One apparently robust memory phenomenon that could be explained by shifts in criterion rather than true changes in memory strength is induced forgetting (Maxcey, Dezso, Megla, & Schneider, 2019; Maxcey, 2016; Maxcey & Woodman, 2014). Induced forgetting is the forgetting of information held in visual long-term memory as a function of remembering semantically related information. In the typical induced forgetting task (Figure 2), participants are presented with multiple exemplars from the same categories (e.g., six pictures of different chairs, six pictures of different water bottles). Then, participants practice recognizing three of the chairs in an old–new recognition judgment task. Finally, participants' memory is tested for all six chairs and all six water bottles. The three chairs that were not practiced are called related objects because they were related to objects that were practiced, but they were not practiced themselves. When tested, participants show worse memory for these related chairs than the water bottles, which were also not practiced. In this example, the water bottles are known as baseline objects because they serve as a baseline for memory of objects from the study phase whose category was not involved in the practice phase. Neither the related objects nor the baseline objects were practiced. Both related and baseline objects were only shown once in the study phase. However, participants consistently respond that they remembered related objects at lower rates than baseline objects. The explanation for this forgetting is that recognizing some of the chairs during the practiced phase induced the forgetting of the related chairs (Fukuda, Pall, Chen, & Maxcey, 2020; Maxcey, McCann, & Stallkamp, 2020; Scotti, Janakiefski, & Maxcey, 2020; Maxcey, Janakiefski, Megla, Smerdell, & Stallkamp, 2019; Maxcey, Glenn, & Stansberry, 2018; Rugo, Tamler, Woodman, & Maxcey, 2017; Maxcey, Bostic, & Maldonado, 2016; Maxcey & Bostic, 2015; Anderson, Bjork, & Bjork, 1994).

Figure 2.

An example of the stimuli and overview of the different phases of the experiment. This figure illustrates this study but is also useful in understanding the basic paradigm used to detect induced forgetting described in the Introduction. In the study phase, a central fixation dot was presented for 500 msec, followed by stimulus presentation in which a single object was presented on the screen for 2000 msec. Participants were instructed to maintain fixation for the duration of the trial while attempting to memorize all images presented for a later memory test. There was a 2000-msec intertrial interval in which participants could blink or move their eyes between trials. In the practice phase, half of the objects from half of the categories in the study phase were randomly presented a total of three times. An equal number of novel objects from each category was also presented during this practice phase. In the test phase, participants performed an old–new recognition task with an equal number of practiced, related, novel, and baseline objects presented.

Figure 2.

An example of the stimuli and overview of the different phases of the experiment. This figure illustrates this study but is also useful in understanding the basic paradigm used to detect induced forgetting described in the Introduction. In the study phase, a central fixation dot was presented for 500 msec, followed by stimulus presentation in which a single object was presented on the screen for 2000 msec. Participants were instructed to maintain fixation for the duration of the trial while attempting to memorize all images presented for a later memory test. There was a 2000-msec intertrial interval in which participants could blink or move their eyes between trials. In the practice phase, half of the objects from half of the categories in the study phase were randomly presented a total of three times. An equal number of novel objects from each category was also presented during this practice phase. In the test phase, participants performed an old–new recognition task with an equal number of practiced, related, novel, and baseline objects presented.

Here, we asked whether induced forgetting reflects a true change in what is stored in memory, as in an actual weakening of the underlying memory signal, or if forgetting simply appears to occur behaviorally because of the placement of the decision threshold when responding to baseline and related items (Figure 1, right).

This Study

Our goal in this study was to test the criterion-shifting explanation of induced forgetting by recording participants' EEG and measuring their ERPs elicited by pictures in an induced forgetting task. We recorded participants' ERPs because this method allowed us to measure neural activity related to the strength of participants' memory representations, rather than relying on participants' overt behavioral responses. This allows us to determine whether induced forgetting is due to an underlying change in memory strength, or whether criterion shifts underlie this forgetting effect.

Recording participants' ERPs will distinguish between true forgetting and criterion shifts because the amplitude of participants' frontal ERPs can provide a more direct measure of memory strength than behavior alone. Specifically, the FN400 tracks the fidelity of long-term memory storage, with this sensitivity being sufficient to be measured on a single trial (Fukuda & Woodman, 2015), and it provides more finely graded responses than participants' behavior in recognition memory experiments (Curran, DeBuse, & Leynes, 2007; Rugg & Curran, 2007; Azimian-Faridani & Wilding, 2006; Rugg et al., 1998).

Predictions

In this study, we used the amplitude of participants' FN400 as our measure of memory strength for the different types of objects in the induced forgetting paradigm to test the following predictions. According to the memory strength hypothesis, induced forgetting occurs because of a true underlying suppression of the related objects (Spitzer & Bäuml, 2007). If participants report forgetting the related objects because their memory representation is truly weakened by recognizing the practiced objects, then we should see that these memories are weaker than those of the baseline objects (Figure 3, left). Support for the memory strength hypothesis would be shown by an FN400 that is reliably more negative for related objects, indicating less memory, than for baseline objects.

Figure 3.

Hypothetical test phase FN400 predicted by the memory strength hypothesis (left) and criterion shift hypothesis (right). The hypothetical grand-averaged ERP waveforms from electrode Fz (over the frontal lobe, along the midline) show a reliable FN400 difference between baseline and related (shaded green) according to the memory strength hypothesis, illustrating a true change in memory strength because of induced forgetting. The hypothetical waveforms representing the criterion shift hypothesis show no reliable FN400 difference between baseline and related items, illustrating that a criterion shift underlies behavioral evidence of forgetting, not a true difference in memory strength.

Figure 3.

Hypothetical test phase FN400 predicted by the memory strength hypothesis (left) and criterion shift hypothesis (right). The hypothetical grand-averaged ERP waveforms from electrode Fz (over the frontal lobe, along the midline) show a reliable FN400 difference between baseline and related (shaded green) according to the memory strength hypothesis, illustrating a true change in memory strength because of induced forgetting. The hypothetical waveforms representing the criterion shift hypothesis show no reliable FN400 difference between baseline and related items, illustrating that a criterion shift underlies behavioral evidence of forgetting, not a true difference in memory strength.

According to the criterion shift hypothesis, induced forgetting is not due to a difference in the underlying memory strength but rather a difference in decision thresholds for responding old to related and baseline objects (Macmillan & Creelman, 2004; Green & Swets, 1966). If participants report forgetting the related objects from the practiced category because of criterion shifts, then we should see no difference between the strength of memory representations, as shown by statistically indistinguishable FN400 between these two object types (Figure 3, right).

In addition to the ERP predictions, the criterion shift and memory strength hypotheses also make opposing behavioral predictions. According to the criterion shift hypothesis, the difference in criteria for baseline and related items should have an impact on both hits and false alarms (Figure 1, right). The predicted pattern of hits is the signature of induced forgetting, with more hits for baseline than related items, and is predicted by both hypotheses. However, a difference in false alarms between items from related and baseline categories is only predicted by the criterion shift hypothesis. Specifically, the predicted leftward shift of criterion for baseline items that captures more hits would also lead to more false alarms to baseline items (see blue–gray shading on Figure 1, right) than related items (see pink–gray shading on Figure 1, right). The memory strength hypothesis does not predict a reliable difference between false alarms because the forgetting is not due to a shift in criterion, but rather a true change in the strength of the underlying memory representation.

Note that even the criterion shift hypothesis involves changes to the underlying memory strength of practiced items. In Figure 3, the practiced items are more positive than baseline and related items for both hypotheses. This is because practiced items have been seen four times throughout the experiment before the test phase, compared with the baseline and related items that have been seen only once in the study phase. This clearly leads to a stronger memory signal for the practiced items. The dissociation between the practice effect (the difference between practiced and baseline items) is known to be independent from induced forgetting (Storm & Levy, 2012); therefore, it is possible that, although underlying memory strength drives the practice effect, a criterion shift drives induced forgetting. Here, we focus on the mechanism underlying induced forgetting.

METHODS

Participants

We ran 22 participants from Vanderbilt University and the surrounding community (14 women, 8 men; Mage = 24.5 years, SDage = 4.7 years) through procedures approved by the Vanderbilt University institutional review board after informed consent was obtained. Participants were compensated at a rate of $15 per hour. All participants reported normal or corrected-to-normal visual acuity, normal color vision, and no history of neurological problems.

Stimuli and Procedure

Stimuli were presented using MATLAB (The MathWorks, Inc.) and Psychophysics Toolbox (Brainard, 1997; Pelli, 1997) on a 24-in. LED gaming monitor (ASUS VG 248; 120-Hz refresh rate). Participants sat approximately 75 cm from the screen. Stimuli were photographs of everyday objects (see Figure 2 for examples) centered on a white background. Each image subtended approximately 7° × 7°. The memoranda were drawn from our previous studies (e.g., Maxcey, Dezso, et al., 2019; Maxcey et al., 2018) and were supplemented using Google Images to provide a larger stimulus set. The images were composed of 32 categories with 21 images in each category for a total of 672 images. (The stimulus set can be found at osf.io/8xatw/).

Figure 2 shows an overview of the experiment. In the study phase, we instructed participants to memorize each presented object for a later memory test. The phase lasted until 12 objects from each of the 32 object categories were presented in random order. Each trial started with a black fixation dot (6.89 cd/m2) in the center of a gray screen (30.5 cd/m2) for 500 msec, followed by the stimulus presentation for 2000 msec. The fixation dot remained on the screen during stimulus presentation to encourage participants to refrain from blinking or making eye movements. There was a 2000-msec intertrial interval after stimulus presentation during which participants could blink and move their eyes. Participants received a break every 64 images until all 384 images were presented.

During the practice phase, half of the originally studied objects (i.e., six objects) from half of the studied categories (i.e., 16 categories) were randomly presented. Practiced objects were presented three separate times during this phase. An equal number of new images (i.e., 96 images) in each of the practiced categories were also presented. Before the practice phase started, we told participants to practice studying all of the images.

An equal number of all object types (i.e., 96 practiced, 96 related, and 96 baseline objects) were tested in the test phase. An equal number of novel (new) objects were also presented (i.e., 96 objects). Novel objects were divided evenly among all categories (i.e., three objects from each of the 32 categories) and were randomly interleaved with the old items. Participants responded using the F and J keys on the keyboard, with the mapping of the keys to old versus new being counterbalanced across participants. Trials were terminated after the keyboard response and followed by a 2000-msec intertrial interval between test trials. We informed participants that 75% of the objects would be old before the test phase began.

EEG Acquisition

The experiment took place in an electrically shielded, soundproof booth as we recorded the EEG during all phases of the experiment. The EEG data were recorded from a 20-channel cap (Electro-Cap International) with channels located according to the International 10–20 electrode sites (F3, F4, C3, C4, P3, P4, PO3, PO4, O1, O2, PO7, PO8, T3, T4, T5, T6, Fz, Cz, Pz). We kept impedance values below 4 kΩ during recording. Data were referenced online to the right mastoid and rereferenced off-line to the average of the left and right mastoid electrodes, and the ground electrode was placed at site Fpz. We placed EOG electrodes approximately 1 cm lateral to the outer canthi of each eye, in addition to an electrode underneath the right eye, to monitor eye movements and blinks. All channels were band-pass filtered from 0.01 to 100 Hz and digitized at 250 Hz.

EEG Analysis

Artifact Rejection

We first rejected trials containing blinks, amplifier saturation, or excessive noise in the EEG by running each participant's data through the EEGLAB Toolbox function eegthresh.m (Delorme & Makeig, 2004). Any trials with voltages greater than +100 μV or less than −100 μV were rejected. Next, we used a split-half sliding window (window size = 200 msec, step size = 10 msec, threshold = 10 μV), as used in Adam, Robison, and Vogel (2018), on the remaining trials to further reject any trials with eye movements. This approach placed a 200-msec window every 10 msec from the beginning to the end of a trial in the difference horizontal EOG (HEOG) signal (left hemisphere − right hemisphere). The trial was rejected if the difference HEOG signal from the first half to the second half of the window was greater than 10 μV. An average of 5.21% of study trials and 8.24% of test trials were rejected for each participant.

FN400 Analysis

We first baseline-corrected the EEG data by subtracting the mean of the 200-msec preceding stimulus onset. We measured the amplitude of participants' ERPs across the midline electrodes Fz, Cz, and Pz, following our previous studies looking at FN400, verifying that our effects were maximal at Fz (Servant, Cassey, Woodman, & Logan, 2018; Reinhart & Woodman, 2014). We used a measurement window from 350 to 550 msec following stimulus onset to calculate mean amplitude, similar to previous studies (e.g., Drew, Williams, Jones, & Luria, 2018), and we plotted these mean voltages across the head using the topoplot.m function from the EEGLAB Toolbox (see Figure 5). Analyses were performed on baseline-corrected, unfiltered data so that our measurements were not contaminated by filtering (JASP Team, 2020). For visualization purposes only, trials were low-pass filtered using the EEGLAB Toolbox function eegfilt.m (Delorme & Makeig, 2004) with a half-amplitude low-pass cutoff at 30 Hz.

Behavioral Data Analysis

The primary dependent variable for our recognition data was d′ (Verde, Macmillan, & Rotello, 2006). We also report hit rate and false alarm data in Table 1. A preplanned repeated-measures t test determined whether there was a difference between false alarms to novel objects from practiced and baseline categories, as predicted by the criterion shift hypothesis (Figure 1, right), accompanied by JZS Bayes factor (scale r on effect size = .707) to quantify support for the null or alternative hypothesis (Rouder, Speckman, Sun, Morey, & Iverson, 2009). JZS Bayes factor represents the relative probability of the data under one model (the null hypothesis) compared with another model (the alternative hypothesis). For example, if JZSNULL = 2, then the null hypothesis is two times more likely than the alternative.

Table 1.

Mean (SD) Behavioral Responses to Objects by Object Type in the Test Phase

 PracticedBaselineRelated
Hit 0.91 (0.08) 0.59 (0.15) 0.55 (0.17) 
d′ 2.30 (1.38) 0.74 (0.09) 0.62 (0.34) 
False alarm 0.34 (0.20) 0.31 (0.15) 0.34 (0.20) 
 PracticedBaselineRelated
Hit 0.91 (0.08) 0.59 (0.15) 0.55 (0.17) 
d′ 2.30 (1.38) 0.74 (0.09) 0.62 (0.34) 
False alarm 0.34 (0.20) 0.31 (0.15) 0.34 (0.20) 

Induced forgetting (baseline-related) is reliable across all measures (hit rate, p = .027, and d′, p = .037). False alarms (SD) to novel objects from baseline and practiced/related categories are not reliably different (p = .271).

RESULTS

Behavioral Results

Mean behavioral data from the test phase are presented in Table 1. We first tested the behavioral prediction of the criterion shift hypothesis. As shown in Figure 1, the criterion shift hypothesis says that induced forgetting is a product of placing the criterion for baseline items to the left, capturing more hits, and related items to the right, capturing fewer hits. If this criterion placement explains induced forgetting, then false alarms will be more frequent for baseline items relative to related items (see Figure 1). We tested this prediction by comparing false alarm rates for related categories (.34) and baseline categories (.31) and found no reliable difference, t(21) = 1.130, p = .271, JZSNULL = 2.55, and a trend in the opposite direction. Having ruled out the behavioral prediction of the criterion shift hypothesis, we next turn to the electrophysiological data to see if our measures of brain activity converge with these behavioral results.

Electrophysiological Results

Participants' grand-averaged ERP waveforms are shown in Figure 4 (and distributions in Figure 5). The ERP waveforms demonstrate the graded effect shown in the predictions figure (Figure 3, left) where related objects elicited potentials that were more negative than those elicited by baseline objects.1 The amplitude of the FN400 was smaller following the presentation of related than baseline items, t(21) = 1.928, p = .034, d = 0.41. These results mirror the conclusions drawn from the behavioral results in that they are inconsistent with the criterion shift hypothesis. The finding of a more negative potential elicited by related items compared with baseline items, even though both were shown only once during the study phase and never during the practice phase, is consistent with the account in which this induced forgetting is due to a true difference in the underlying memory representations of these two object types.

Figure 4.

Test phase FN400. These grand-averaged ERP waveforms were measured at electrode Fz and were elicited by the presentation of each object type in the test phase. The measurement window used to calculate the FN400 amplitude is shown in the dashed box (i.e., 350–550 msec). Consistent with the memory strength hypothesis, there was a reliable FN400 difference between baseline and related (shaded green), illustrating a true change in memory strength because of induced forgetting.

Figure 4.

Test phase FN400. These grand-averaged ERP waveforms were measured at electrode Fz and were elicited by the presentation of each object type in the test phase. The measurement window used to calculate the FN400 amplitude is shown in the dashed box (i.e., 350–550 msec). Consistent with the memory strength hypothesis, there was a reliable FN400 difference between baseline and related (shaded green), illustrating a true change in memory strength because of induced forgetting.

Figure 5.

Waveforms across midline electrodes and topographic maps of voltage across the head showing the typical FN400 distribution for each trial type, as well as the difference between baseline and related items. The left column reproduces the waveforms shown in Figure 4 at the top (Fz), with electrodes Cz and Pz below it. Right column shows the topographical maps of raw voltage for baseline-minus-novel, related-minus-novel, and baseline-minus-related for the FN400 interval, 350–550 msec. The top two distributions confirm that we are measuring the same FN400 previously observed during recognition memory tasks (Rugg & Curran, 2007). The baseline-minus-related map might be interpreted as suggesting an inferior frontal source implicating inhibitory mechanisms, but we can see in the component maps that this is due to a minor rightward shift in FN400 on baseline trials. The component voltage maps are shown in Appendix Figure 1 for those who are interested.

Figure 5.

Waveforms across midline electrodes and topographic maps of voltage across the head showing the typical FN400 distribution for each trial type, as well as the difference between baseline and related items. The left column reproduces the waveforms shown in Figure 4 at the top (Fz), with electrodes Cz and Pz below it. Right column shows the topographical maps of raw voltage for baseline-minus-novel, related-minus-novel, and baseline-minus-related for the FN400 interval, 350–550 msec. The top two distributions confirm that we are measuring the same FN400 previously observed during recognition memory tasks (Rugg & Curran, 2007). The baseline-minus-related map might be interpreted as suggesting an inferior frontal source implicating inhibitory mechanisms, but we can see in the component maps that this is due to a minor rightward shift in FN400 on baseline trials. The component voltage maps are shown in Appendix Figure 1 for those who are interested.

Next, we confirmed that the difference in the amplitude of the FN400 between baseline and related items, indicative of forgetting, was truly due to the repetition of practiced items during the practice phase. To this end, we analyzed the ERPs from the first phase, the study phase, to confirm there was no differentiation in waveforms before practice. The ERP waveforms in Figure 6 (Fz time locked to the study item onset, waveforms for baseline, related, and to-be-practiced objects) demonstrate that all three object types were overlapping during the study phase, before practice induced the forgetting of related items and boosted memory for practiced items, F(3, 63) = 0.530, p = .663, ηp2 = .025. This rules out the alternative explanation that a physical stimulus confound caused baseline objects to be more memorable than related.

Figure 6.

Study phase FN400. These grand-averaged ERP waveforms were measured at electrode Fz and were elicited by the presentation of each object during the study phase. These waveforms demonstrate that the object types were indistinguishable at study, showing that differences in test phase waveforms are not due to a physical stimulus confound.

Figure 6.

Study phase FN400. These grand-averaged ERP waveforms were measured at electrode Fz and were elicited by the presentation of each object during the study phase. These waveforms demonstrate that the object types were indistinguishable at study, showing that differences in test phase waveforms are not due to a physical stimulus confound.

ERPs are prone to contamination by eye movements. People make different patterns of eye movements when viewing pictures they remember versus pictures that they do not report remembering (Loftus & Mackworth, 1978). This means that it is possible that our ERP effects could be artifacts of differential eye movement behavior. Given that the critical comparison here is between items that are better remembered (i.e., baseline) than some other items (i.e., related), we also analyzed HEOG to ensure that eye movements were not impacting our results. Figure 7 demonstrates that the test-elicited ERPs from HEOG in response to the different test items (i.e., baseline, related, practiced, and novel) were not reliably different, F(3, 63) = 0.477, p = .720, ηp2 = .021. This rules out the alternative explanation that eye movements contaminate our results.

Figure 7.

HEOG (recorded as the difference between electrodes placed 1 cm lateral to each eye) while viewing the test items centered at fixation. The overlapping nature of these traces show that eye movements do not account for the electrophysiological differences measured across the head.

Figure 7.

HEOG (recorded as the difference between electrodes placed 1 cm lateral to each eye) while viewing the test items centered at fixation. The overlapping nature of these traces show that eye movements do not account for the electrophysiological differences measured across the head.

To provide further evidence for the memory strength hypothesis, we confirmed that the change in memory strength indexed by the FN400 was correlated with the behavioral evidence of induced forgetting. Recall that induced forgetting is defined by worse memory for related items relative to baseline items and can thus be quantified by a difference score (i.e., d′ baseline − d′ related) that can be used in a correlation with the amplitude of the FN400. We found that the amplitude difference (μV) of the FN400 between baseline and related objects could reliably predict a participant's behavioral measure of induced forgetting, r(20) = .401, p = .032. This confirms that the FN400 was tracking behavioral measures of forgetting.2 Furthermore, within-condition correlations between behavior and amplitude show that behavioral recognition is correlated with the amplitude of the FN400 (practiced amplitude and d′, r(20) = .63, p < .001; baseline amplitude and d′, r(20) = .59, p = .002; and related amplitude and d′, r(20) = .52, p = .007), as shown by previous studies (Kutas & Federmeier, 2011; Rugg & Curran, 2007; Voss & Paller, 2006, 2007). These within-condition correlations are consistent with our interpretation that performance at recognizing these different types of objects was due to differences in memory as measured by modulations of the FN400.

DISCUSSION

Signal detection theory is one of the oldest, most far-reaching theoretical accounts in experimental psychology (Wixted, 2020; Peterson, Birdsall, & Fox, 1954). Of particular importance for this study, signal detection theory makes simple, testable predictions about how decision-making is performed during recognition tasks. Here, we tested the criterion shift hypothesis of induced forgetting. This hypothesis proposes that rather than detecting true forgetting, behavioral evidence of induced forgetting is due to participants shifting their decision criteria in response to the two critical object categories in this task (i.e., baseline and related). On the other hand, the memory strength hypothesis proposes induced forgetting occurs because of a true underlying suppression of the related objects. These opposing hypotheses make two specific predictions, one tested with behavioral data and one requiring electrophysiological data to resolve. First, if criterion shifting is occurring for these two object types, behavioral responses should show increased false alarms for baseline object categories (Figure 1, right). Here, we found no reliable difference between false alarms for baseline or related object categories, consistent with the memory strength hypothesis. Second, if criterion shifting explains apparent forgetting, rather than a change in the underlying memory strength of related objects, then a frontal ERP component, known as the FN400, which tracks the fidelity of long-term memory storage (Fukuda & Woodman, 2015; Maxcey, Fukuda, Song, & Woodman, 2015; Rugg & Curran, 2007; Rugg et al., 1998), should show no reliable difference in amplitude across these two object types (Figure 3). Again, supporting the memory strength hypothesis, electrophysiological markers of the contents of long-term memory showed forgetting of related objects relative to baseline objects (Figure 4). We show that this effect is due to the second phase of the experiment (Figure 6) and not due to eye movements (Figure 7).

Existing electrophysiological studies have not convincingly demonstrated evidence for or against the underlying memory strength explanation of induced forgetting. However, previous electrophysiological studies have tested other hypotheses related to induced forgetting. Next, we briefly review the foundational work laid out in previous studies on this topic, including the hypotheses they were testing and a short explanation that underlines the novelty of the present work. First, we will discuss the four extant ERP studies that used an induced forgetting paradigm to measure activity during the practice phase, with the goal of measuring inhibition or other mechanisms during this middle phase of these paradigms. Then, we will discuss the only other ERP study we are aware of that reports activity measured during the test phase of an induced forgetting paradigm and how those previous findings are related to what we show here.

To probe the electrophysiological correlates of induced forgetting, Johansson, Aslan, Bäuml, Gäbel, and Mecklinger (2007) measured ERPs during the practice phase when participants were instructed to either relearn a presented word or retrieve a word when given the first two to three letters of the word (i.e., stem completion). The authors encouraged participants to withhold oral responses during the practice phase to avoid muscle artifacts. The electrophysiological data from the practice phase were then sorted by participants' responses from the final test phase. The authors observed a sustained positivity over the frontal electrode sites during practice (i.e., the phase that induces forgetting) predicted later forgetting. The purpose of analyzing ERPs during the practice phase was to measure potential inhibitory processes that might operate during practice. Thus, this study did not test the hypothesis that decision-making thresholds might explain why participants appear to not remember certain items, but instead interpreted modulations measured during the practice phase as consistent with proposals that memory representations compete for access to retrieval. Our experiment examined the effects of restudying pictures in contrast to this previous study that used restudy as a control condition in which competition was hypothesized to not exist.

In the next step in this line of work, Hellerstedt and Johansson (2014) again focused on ERPs recorded from the practice phase to further test the hypothesis that competition modulates how much we forget a representation. They found that competitors more strongly linked to their category cues (determined by taxonomic frequency) elicited a greater FN400 response during practice than weak competitive cue associations. Hellerstedt and Johansson did detect a modulation of the FN400 during this practice phase. However, their approach did not allow them to test the criterion shift hypothesis. Instead, they concluded the FN400 was modulated by taxonomic frequency of the competitor to its category cue, similar to previous studies on linguistic processing that found an influence of word frequency on the FN400 (Kutas & Federmeier, 2000). The next two studies that we describe measured activity in specific frequency bands to test hypotheses about the cognitive operations at play during the practice phase of these induced forgetting experiments.

Staudigl, Hanslmayr, and Bäuml (2010) asked whether theta oscillations reflect interference during memory retrieval in the practice phase. They relied on previous work showing that induced forgetting of words occurs following retrieval but not restudy (Bäuml & Aslan, 2004; Anderson, Bjork, & Bjork, 2000; Ciranni & Shimamura, 1999) and used a restudy task as their baseline condition, against which they compared theta band activity during retrieval (using logic similar to that of Johansson et al., 2007, when measuring the FN400 during practice). The authors concluded that theta oscillations index interference in episodic memory, linking ERPs elicited from one condition in the practice phase and behavioral performance from another condition in the test phase. As mentioned above, this study provides a novel departure from this previous work by using restudy of pictures to induce forgetting (Maxcey, Janakiefski, et al., 2019) and measuring both behavioral responses and brain activity at test from completely separate object categories to distinguish between competing explanations of the cognitive processes giving rise to forgetting.

The other study to focus on theta activity was published by Hanslmayr, Staudigl, Aslan, and Bäuml (2010) and measured theta power during what they called competitive and noncompetitive memory retrieval in the practice phase. During the practice phase, participants had to either recall the category name (e.g., FR____: Apple) or the exemplar name (e.g., FRUIT: Ap____). The former practice task (e.g., FR____: Apple) was considered noncompetitive because it did not match the test phase task, which required participants to report the exemplar name (e.g., FRUIT: Ap____). The latter practice task (e.g., FRUIT: Ap____) was considered competitive because it matched the test phase task. Only the competitive task induced forgetting. Replicating Staudigl et al. (2010), Hanslmayr et al. found that theta power was stronger during the practice phase when competition was present. Similar to the points made above, the use of verbal memoranda, the focus on practice phase activity, the absence of a baseline condition similar to that used here, and measuring frequency band power instead of ERPs are methodological distinctions between the Hanslmayr et al. (2010) study and this study.

One extant study did examine neural activity at test like what was done here. Spitzer, Hanslmayr, Opitz, Mecklinger, and Bäuml (2009) examined the effect of retrieval practice of words on ERP components and oscillatory brain activity. They found no impact of induced forgetting on the FN400 with words. Spitzer et al. suggested that their failure to measure induced forgetting with the FN400 was due to the verbal memoranda they employed because the FN400 is modulated as a function of typicality members of a category (Kutas & Federmeier, 2000). Unlike Spitzer et al., here we successfully measured induced forgetting using the FN400 and visual memoranda. Moreover, we found that simply restudying pictures led to forgetting, replicating previous behavioral work (Maxcey, Janakiefski, et al., 2019), and demonstrating how the present work deviates from previous investigations. In the present work, the control condition was not accompanied by special instructions that may have changed people's strategies, but instead the control objects (i.e., baseline objects) are randomly interleaved with those that the experimental design targets for forgetting.

Collectively, the five EEG and ERP studies to our knowledge that have examined some aspect of induced forgetting have not addressed the simple question herein, “Is there electrophysiological evidence of a change in memory strength for forgotten items relative to remembered items?” Four of the five studies (Hellerstedt & Johansson, 2014; Hanslmayr et al., 2010; Staudigl et al., 2010; Johansson et al., 2007) reported ERPs from the practice phase, not the test phase. The one study that did measure behavioral induced forgetting and ERPs on the same test phase trials, Spitzer et al. (2009), found no impact of induced forgetting on the FN400 using words. Thus, based on the existing work, we would likely conclude that the criterion shift hypothesis is correct, and there is no evidence for the modulation of memory strength underlying induced forgetting at the final memory test. This study draws the novel conclusion that decision-making thresholds cannot explain the patterns of behavior measured when memory is ultimately tested.

A major challenge to modeling memory and forgetting is determining the underlying mechanism(s) responsible for forgetting. Existing theories propose forgetting may be due to inhibition (Storm & Levy, 2012; Anderson et al., 2004), context (Maxcey et al., in press; Jonker, Seli, & MacLeod, 2013), or familiarity (Shiffrin & Steyvers, 1997; Raaijmakers & Shiffrin, 1980, 1981). In this study, both behavioral and electrophysiological measures weigh against a criterion shift explanation of induced forgetting. Having established that induced forgetting is due to changes in memory strength, future work can confidently address what causes the forgetting of related memories.

Studies supporting criterion shifts as an explanation for recognition memory performance (Chen, Starns, & Rotello, 2015; Verde & Rotello, 2003; Miller & Wolford, 1999; Hirshman, 1995) may have incorrectly confused effects of retrieval for effects of decision-making (Verde & Rotello, 2007). Indeed signal detection theory is not a model of memory. It is a model of decision-making based on signal strength that has been employed to explain performance in memory tasks (Singer & Wixted, 2006). Signal detection theory is not operationalizing memory, as some have argued (Franks & Hicks, 2016), because what is being measured is downstream decision-making, not the actual memory signal. If signal detection theory can successfully account for performance in a memory task, then the locus of the effect is not in memory, because all of the memory processes are handled by other parts of the system that compute memory strength and simply served up to the threshold as an integer value.

Signal detection explanations of memory performance, which otherwise appear parsimonious (e.g., Miller & Wolford, 1999), must rule out a role of memory. To truly operationalize, memory requires a measurement of the actual memory signal, such as the FN400 (Fukuda & Woodman, 2015; Rugg & Curran, 2007; Rugg et al., 1998). Here, we provide a novel electrophysiological approach to confirm that the action of a memory phenomenon, induced forgetting, is occurring in memory, not during downstream decision-making. It may be the case that criterion shifts are still occurring in this task, because the FN400 does not account for all the variance in induced forgetting. However, if criterion shifts are occurring, they occur after a true change in memory strength given our current understanding of the FN400 signature. Future work should employ a similar neuroscientific approach, measuring brain activity to validly operationalize memory, as a precursor to determining the utility of the signal detection theory approach.

APPENDIX

Appendix Figure 1.

The topographic voltage maps showing mean voltage measured from 350 to 550 msec post stimulus onset for each type of test object shown. The posterior mound of activity shows the P3b component over parietal electrodes. The FN400 can be seen as the negativity surrounding electrode Fz. Note that the distribution of voltage elicited by the baseline objects had a greater positivity over right frontal electrodes than did the related object response, resulting in a significant effect of electrode F(18, 378) = 4.612, p < .001, ηp2 = .180) and a significant interaction of Object × Electrode F(18, 378) = 1.702, p = .037, ηp2 = .075) in an ANOVA with the factors of Object type (related, baseline) and Electrode (19 electrode sites on the head). Although it may seem tempting to attribute this frontal activation to inhibition, it is notable that the rightward activity is on baseline trials, not related trials, inconsistent with this simple interpretation.

Appendix Figure 1.

The topographic voltage maps showing mean voltage measured from 350 to 550 msec post stimulus onset for each type of test object shown. The posterior mound of activity shows the P3b component over parietal electrodes. The FN400 can be seen as the negativity surrounding electrode Fz. Note that the distribution of voltage elicited by the baseline objects had a greater positivity over right frontal electrodes than did the related object response, resulting in a significant effect of electrode F(18, 378) = 4.612, p < .001, ηp2 = .180) and a significant interaction of Object × Electrode F(18, 378) = 1.702, p = .037, ηp2 = .075) in an ANOVA with the factors of Object type (related, baseline) and Electrode (19 electrode sites on the head). Although it may seem tempting to attribute this frontal activation to inhibition, it is notable that the rightward activity is on baseline trials, not related trials, inconsistent with this simple interpretation.

Acknowledgments

This work was supported by grants from the National Institutes of Health (R01-EY019882, R01-MH110378, and P30-EY08126) to G. F. W.

Reprint requests should be sent to Ashleigh M. Maxcey, Department of Psychology, Vanderbilt University, Wilson Hall, 111 21st Ave S, Nashville, TN 37212, or via e-mail: ashleigh.m.maxcey@vanderbilt.edu.

Funding Information

Geoffrey F. Woodman, National Institutes of Health (http://dx.doi.org/10.13039/100000002), grant numbers: P30-EY08126, R01-EY019882, R01-MH110378.

Diversity in Citation Practices

A retrospective analysis of the citations in every article published in this journal from 2010 to 2020 has revealed a persistent pattern of gender imbalance: Although the proportions of authorship teams (categorized by estimated gender identification of first author/last author) publishing in the Journal of Cognitive Neuroscience (JoCN) during this period were M(an)/M = .408, W(oman)/M = .335, M/W = .108, and W/W = .149, the comparable proportions for the articles that these authorship teams cited were M/M = .579, W/M = .243, M/W = .102, and W/W = .076 (Fulvio et al., JoCN, 33:1, pp. 3–7). Consequently, JoCN encourages all authors to consider gender balance explicitly when selecting which articles to cite and gives them the opportunity to report their article's gender citation balance. The authors of this article report its proportions of citations by gender category to be as follows: M/M = .667, W/M = .13, M/W = .074, and W/W = .13.

Notes

1. 

These analyses consist of the brain responses to all stimuli from these object types, regardless of behavior. We analyzed all of the data because we were interested in how the brain responded to the test items, regardless of participants' behavioral response. However, we also analyzed the data from only correct trials. These data show the same pattern observed across all trials, F(3, 63) = 18.66, p < .001, ηp2 = .470, t test isolating baseline-related, t(21) = 2.258, p = .017, d = 0.481, demonstrating the generality of these observations.

2. 

We note that the response-locked P3 (i.e., the P3b), the lateralized readiness potential, and several other ERP components that play roles in the initiation and completion of manual button presses are more tightly linked to participants' behavioral responses than the memory-related component that we focused on here (Woodman, 2010), but these were not directly relevant for the hypotheses tested in this study, as none of these components appear to represent something like a criterion value.

REFERENCES

Adam
,
K. C. S.
,
Robison
,
M. K.
, &
Vogel
,
E. K.
(
2018
).
Contralateral delay activity tracks fluctuations in working memory performance
.
Journal of Cognitive Neuroscience
,
30
,
1229
1240
.
Anderson
,
M. C.
,
Bjork
,
E. L.
, &
Bjork
,
R. A.
(
2000
).
Retrieval-induced forgetting: Evidence for a recall-specific mechanism
.
Psychonomic Bulletin & Review
,
7
,
522
530
.
Anderson
,
M. C.
,
Bjork
,
R. A.
, &
Bjork
,
E. L.
(
1994
).
Remembering can cause forgetting: Retrieval dynamics in long-term memory
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
20
,
1063
1087
.
Anderson
,
M. C.
,
Ochsner
,
K. N.
,
Kuhl
,
B.
,
Cooper
,
J.
,
Robertson
,
E.
,
Gabrieli
,
S. W.
, et al
(
2004
).
Neural systems underlying the suppression of unwanted memories
.
Science
,
303
,
232
235
.
Azimian-Faridani
,
N.
, &
Wilding
,
E. L.
(
2006
).
The influence of criterion shifts on electrophysiological correlates of recognition memory
.
Journal of Cognitive Neuroscience
,
18
,
1075
1086
.
Bäuml
,
K.-H.
, &
Aslan
,
A.
(
2004
).
Part-list cuing as instructed retrieval inhibition
.
Memory & Cognition
,
32
,
610
617
.
Brainard
,
D. H.
(
1997
).
The Psychophysics Toolbox
.
Spatial Vision
,
10
,
433
436
.
Chen
,
T.
,
Starns
,
J. J.
, &
Rotello
,
C. M.
(
2015
).
A violation of the conditional independence assumption in the two-high-threshold model of recognition memory
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
41
,
1215
1222
.
Ciranni
,
M. A.
, &
Shimamura
,
A. P.
(
1999
).
Retrieval-induced forgetting in episodic memory
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
25
,
1403
1414
.
Curran
,
T.
,
DeBuse
,
C.
, &
Leynes
,
P. A.
(
2007
).
Conflict and criterion setting in recognition memory
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
33
,
2
17
.
Delorme
,
A.
, &
Makeig
,
S.
(
2004
).
EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis
.
Journal of Neuroscience Methods
,
134
,
9
21
.
Drew
,
T.
,
Williams
,
L. H.
,
Jones
,
C. M.
, &
Luria
,
R.
(
2018
).
Neural processing of repeated search targets depends upon the stimuli: Real world stimuli engage semantic processing and recognition memory
.
Frontiers in Human Neuroscience
,
12
,
460
.
Franks
,
B. A.
, &
Hicks
,
J. L.
(
2016
).
The reliability of criterion shifting in recognition memory is task dependent
.
Memory & Cognition
,
44
,
1215
1227
.
Fukuda
,
K.
,
Pall
,
S.
,
Chen
,
E.
, &
Maxcey
,
A. M.
(
2020
).
Recognition and rejection each induce forgetting
.
Psychonomic Bulletin & Review
,
27
,
520
528
.
Fukuda
,
K.
, &
Woodman
,
G. F.
(
2015
).
Predicting and improving recognition memory using multiple electrophysiological signals in real time
.
Psychological Science
,
26
,
1026
1037
.
Glanzer
,
M.
,
Kim
,
K.
,
Hilford
,
A.
, &
Adams
,
J. K.
(
1999
).
Slope of the receiver-operating characteristic in recognition memory
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
25
,
500
513
.
Green
,
D. M.
, &
Swets
,
J. A.
(
1966
).
Signal detection theory and psychophysics
.
New York
:
Wiley
.
Hanslmayr
,
S.
,
Staudigl
,
T.
,
Aslan
,
A.
, &
Bäuml
,
K.-H.
(
2010
).
Theta oscillations predict the detrimental effects of memory retrieval
.
Cognitive, Affective, & Behavioral Neuroscience
,
10
,
329
338
.
Hellerstedt
,
R.
, &
Johansson
,
M.
(
2014
).
Electrophysiological correlates of competitor activation predict retrieval-induced forgetting
.
Cerebral Cortex
,
24
,
1619
1629
.
Hicks
,
J. L.
, &
Marsh
,
R. L.
(
1998
).
A decrement-to-familiarity interpretation of the revelation effect from forced-choice tests of recognition memory
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
24
,
1105
1120
.
Hirshman
,
E.
(
1995
).
Decision processes in recognition memory: Criterion shifts and the list-strength paradigm
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
21
,
302
313
.
JASP Team
. (
2020
).
JASP (Version 0.14) [Computer software]
.
Retrieved from https://jasp-stats.org/.
Johansson
,
M.
,
Aslan
,
A.
,
Bäuml
,
K.-H.
,
Gäbel
,
A.
, &
Mecklinger
,
A.
(
2007
).
When remembering causes forgetting: Electrophysiological correlates of retrieval-induced forgetting
.
Cerebral Cortex
,
17
,
1335
1341
.
Jonker
,
T. R.
,
Seli
,
P.
, &
MacLeod
,
C. M.
(
2013
).
Putting retrieval-induced forgetting in context: An inhibition-free, context-based account
.
Psychological Review
,
120
,
852
872
.
Kutas
,
M.
, &
Federmeier
,
K. D.
(
2000
).
Electrophysiology reveals semantic memory use in language comprehension
.
Trends in Cognitive Sciences
,
4
,
463
470
.
Kutas
,
M.
, &
Federmeier
,
K. D.
(
2011
).
Thirty years and counting: Finding meaning in the N400 component of the event-related brain potential (ERP)
.
Annual Review of Psychology
,
62
,
621
647
.
Loftus
,
G. R.
, &
Mackworth
,
N. H.
(
1978
).
Cognitive determinants of fixation location during picture viewing
.
Journal of Experimental Psychology: Human Perception and Performance
,
4
,
565
572
.
Macmillan
,
N. A.
, &
Creelman
,
C. D.
(
2004
).
Detection theory: A user's guide
(2nd ed.).
New York
:
Psychology Press
.
Maxcey
,
A. M.
(
2016
).
Recognition-induced forgetting is not due to category-based set size
.
Attention, Perception, & Psychophysics
,
78
,
187
197
.
Maxcey
,
A. M.
, &
Bostic
,
J.
(
2015
).
Activating learned exemplars in children impairs memory for related exemplars in visual long-term memory
.
Visual Cognition
,
23
,
643
558
.
Maxcey
,
A. M.
,
Bostic
,
J.
, &
Maldonado
,
T.
(
2016
).
Recognition practice results in a generalizable skill in older adults: Decreased intrusion errors to novel objects belonging to practiced categories
.
Applied Cognitive Psychology
,
30
,
643
649
.
Maxcey
,
A. M.
,
De Leon
,
V.
,
Janakiefski
,
L.
,
Megla
,
E.
,
Stallkamp
,
S.
,
Torres
,
R.
, et al
(
in press
).
Induced forgetting of pictures across shifts of context
.
Journal of Experimental Psychology: Human Perception and Performance
.
Maxcey
,
A. M.
,
Dezso
,
B.
,
Megla
,
E.
, &
Schneider
,
A.
(
2019
).
Unintentional forgetting is beyond cognitive control
.
Cognitive Research: Principles and Implications
,
4
,
25
.
Maxcey
,
A. M.
,
Fukuda
,
K.
,
Song
,
W. S.
, &
Woodman
,
G. F.
(
2015
).
Using electrophysiology to demonstrate that cueing affects long-term memory storage over the short term
.
Psychonomic Bulletin & Review
,
22
,
1349
1357
.
Maxcey
,
A. M.
,
Glenn
,
H.
, &
Stansberry
,
E.
(
2018
).
Recognition-induced forgetting does not occur for temporally grouped objects unless they are semantically related
.
Psychonomic Bulletin & Review
,
25
,
1087
1103
.
Maxcey
,
A. M.
,
Janakiefski
,
L.
,
Megla
,
E.
,
Smerdell
,
M.
, &
Stallkamp
,
S.
(
2019
).
Modality-specific forgetting
.
Psychonomic Bulletin & Review
,
26
,
622
633
.
Maxcey
,
A. M.
,
McCann
,
M.
, &
Stallkamp
,
S.
(
2020
).
Recognition-induced forgetting is caused by episodic, not semantic, memory retrieval tasks
.
Attention, Perception, & Psychophysics
,
82
,
1539
1547
.
Maxcey
,
A. M.
, &
Woodman
,
G. F.
(
2014
).
Forgetting induced by recognition of visual images
.
Visual Cognition
,
22
,
789
808
.
Miller
,
M. B.
, &
Wolford
,
G. L.
(
1999
).
Theoretical commentary: The role of criterion shift in false memory
.
Psychological Review
,
106
,
398
405
.
Pelli
,
D. G.
(
1997
).
The VideoToolbox software for visual psychophysics: Transforming numbers into movies
.
Spatial Vision
,
10
,
437
442
.
Peterson
,
W.
,
Birdsall
,
T.
, &
Fox
,
W.
(
1954
).
The theory of signal detectability
.
Transactions of the IRE Professional Group on Information Theory
,
4
,
171
212
.
Raaijmakers
,
J. G. W.
, &
Shiffrin
,
R. M.
(
1980
).
SAM: A theory of probabilistic search of associative memory
. In
H. B.
Gordon
(Ed.),
Psychology of learning and motivation
(
Vol. 14
, pp.
207
262
).
Cambridge, MA
:
Academic Press
.
Raaijmakers
,
J. G. W.
, &
Shiffrin
,
R. M.
(
1981
).
Search of associative memory
.
Psychological Review
,
88
,
93
134
.
Ratcliff
,
R.
,
Sheu
,
C.-F.
, &
Gronlund
,
S. D.
(
1992
).
Testing global memory models using ROC curves
.
Psychological Review
,
99
,
518
535
.
Reinhart
,
R. M. G.
, &
Woodman
,
G. F.
(
2014
).
Causal control of medial–frontal cortex governs electrophysiological and behavioral indices of performance monitoring and learning
.
Journal of Neuroscience
,
34
,
4214
4227
.
Rouder
,
J. N.
,
Speckman
,
P. L.
,
Sun
,
D.
,
Morey
,
R. D.
, &
Iverson
,
G.
(
2009
).
Bayesian t tests for accepting and rejecting the null hypothesis
.
Psychonomic Bulletin & Review
,
16
,
225
237
.
Rugg
,
M. D.
, &
Curran
,
T.
(
2007
).
Event-related potentials and recognition memory
.
Trends in Cognitive Sciences
,
11
,
251
257
.
Rugg
,
M. D.
,
Mark
,
R. E.
,
Walla
,
P.
,
Schloerscheidt
,
A. M.
,
Birch
,
C. S.
, &
Allan
,
K.
(
1998
).
Dissociation of the neural correlates of implicit and explicit memory
.
Nature
,
392
,
595
598
.
Rugo
,
K. F.
,
Tamler
,
K. N.
,
Woodman
,
G. F.
, &
Maxcey
,
A. M.
(
2017
).
Recognition-induced forgetting of faces in visual long-term memory
.
Attention, Perception, & Psychophysics
,
79
,
1878
1885
.
Scotti
,
P. S.
,
Janakiefski
,
L.
, &
Maxcey
,
A. M.
(
2020
).
Recognition-induced forgetting of schematically related pictures
.
Psychonomic Bulletin & Review
,
27
,
357
365
.
Servant
,
M.
,
Cassey
,
P.
,
Woodman
,
G. F.
, &
Logan
,
G. D.
(
2018
).
Neural bases of automaticity
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
44
,
440
464
.
Shiffrin
,
R. M.
, &
Steyvers
,
M.
(
1997
).
A model for recognition memory: REM—Retrieving effectively from memory
.
Psychonomic Bulletin & Review
,
4
,
145
166
.
Singer
,
M.
, &
Wixted
,
J. T.
(
2006
).
Effect of delay on recognition decisions: Evidence for a criterion shift
.
Memory & Cognition
,
34
,
125
137
.
Spitzer
,
B.
, &
Bäuml
,
K.-H.
(
2007
).
Retrieval-induced forgetting in item recognition: Evidence for a reduction in general memory strength
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
33
,
863
875
.
Spitzer
,
B.
,
Hanslmayr
,
S.
,
Opitz
,
B.
,
Mecklinger
,
A.
, &
Bäuml
,
K.-H.
(
2009
).
Oscillatory correlates of retrieval-induced forgetting in recognition memory
.
Journal of Cognitive Neuroscience
,
21
,
976
990
.
Staudigl
,
T.
,
Hanslmayr
,
S.
, &
Bäuml
,
K.-H.
(
2010
).
Theta oscillations reflect the dynamics of interference in episodic memory retrieval
.
Journal of Neuroscience
,
30
,
11356
11362
.
Storm
,
B. C.
, &
Levy
,
B. J.
(
2012
).
A progress report on the inhibitory account of retrieval-induced forgetting
.
Memory & Cognition
,
40
,
827
843
.
Verde
,
M. F.
,
Macmillan
,
N. A.
, &
Rotello
,
C. M.
(
2006
).
Measures of sensitivity based on a single hit rate and false alarm rate: The accuracy, precision, and robustness of d′, Az, and A
.
Perception & Psychophysics
,
68
,
643
654
.
Verde
,
M. F.
, &
Rotello
,
C. M.
(
2003
).
Does familiarity change in the revelation effect?
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
29
,
739
746
.
Verde
,
M. F.
, &
Rotello
,
C. M.
(
2007
).
Memory strength and the decision process in recognition memory
.
Memory & Cognition
,
35
,
254
262
.
Voss
,
J. L.
, &
Paller
,
K. A.
(
2006
).
Fluent conceptual processing and explicit memory for faces are electrophysiologically distinct
.
Journal of Neuroscience
,
26
,
926
933
.
Voss
,
J. L.
, &
Paller
,
K. A.
(
2007
).
Neural correlates of conceptual implicit memory and their contamination of putative neural correlates of explicit memory
.
Learning & Memory
,
14
,
259
267
.
Wickelgren
,
W. A.
(
1968
).
Unidimensional strength theory and component analysis of noise in absolute and comparative judgments
.
Journal of Mathematical Psychology
,
5
,
102
122
.
Witt
,
J. K.
,
Taylor
,
J. E. T.
,
Sugovic
,
M.
, &
Wixted
,
J. T.
(
2015
).
Signal detection measures cannot distinguish perceptual biases from response biases
.
Perception
,
44
,
289
300
.
Wixted
,
J. T.
(
2016
).
From the pigeon lab to the courtroom
.
Comparative Cognition & Behavior Reviews
,
11
,
1
23
.
Wixted
,
J. T.
(
2020
).
The forgotten history of signal detection theory
.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
46
,
201
233
.
Wixted
,
J. T.
, &
Stretch
,
V.
(
2000
).
The case against a criterion shift account of false memory
.
Psychological Review
,
107
,
368
376
.
Wixted
,
J. T.
, &
Stretch
,
V.
(
2004
).
In defense of the signal detection interpretation of remember/know judgments
.
Psychonomic Bulletin & Review
,
11
,
616
641
.
Woodman
,
G. F.
(
2010
).
A brief introduction to the use of event-related potentials in studies of perception and attention
.
Attention, Perception, & Psychophysics
,
72
,
2031
2046
.