To test how preexisting long-term memory influences visual STM, this study takes advantage of individual differences in participants' prior familiarity with Pokémon characters and uses an ERP component, the contralateral delay activity (CDA), to assess whether observers' prior stimulus familiarity affects STM consolidation and storage capacity. In two change detection experiments, consolidation speed, as indexed by CDA fractional area latency and/or early-window (500–800 msec) amplitude, was significantly associated with individual differences in Pokémon familiarity. In contrast, the number of remembered Pokémon stimuli, as indexed by Cowan's K and late-window (1500–2000 msec) CDA amplitude, was significantly associated with individual differences in Pokémon familiarity when STM consolidation was incomplete because of a short presentation of Pokémon stimuli (500 msec, Experiment 2), but not when STM consolidation was allowed to complete given sufficient encoding time (1000 msec, Experiment 1). Similar findings were obtained in between-group analyses when participants were separated into high-familiarity and low-familiarity groups based on their Pokémon familiarity ratings. Together, these results suggest that stimulus familiarity, as a proxy for the strength of preexisting long-term memory, primarily speeds up STM consolidation, which may subsequently lead to an increase in the number of remembered stimuli if consolidation is incomplete. These findings thus highlight the importance of research assessing how effects on representations (e.g., STM capacity) are in general related to (or even caused by) effects on processes (e.g., STM consolidation) in cognition.
How does information passively stored in long-term memory (LTM) influence processes and representations actively engaged in STM? For instance, does preexisting LTM increase the amount of information that can be simultaneously maintained in STM? Although this is one of the central questions concerning the architecture of human memory (Jonides et al., 2008; Atkinson & Shiffrin, 1968), previous research has yielded mixed findings (see Luck, 2008, for a review), ranging from significant boosts of STM capacity (i.e., the number of retained representations) for familiar over unfamiliar stimuli (e.g., Xie & Zhang, 2017a; Curby, Glazek, & Gauthier, 2009; Jackson & Raymond, 2008) to null results (e.g., Huang, 2011; Olson & Jiang, 2004; Pashler, 1988). Although some of these conflicting findings can potentially be accounted for by factors concerning the strength of LTM representations due to differences in familiarization procedures (Beck & van Lamsweerde, 2011), it may also be important to consider how these STM capacity effects (an issue about representation) are related to the effects of LTM on processes in STM, such as STM consolidation (Xie & Zhang, 2017b; Blalock, 2015; Phillips, 1971).
STM consolidation is the process by which fragile sensory memories (Cappiello & Zhang, 2016) are transformed into durable STM representations (Vogel, Woodman, & Luck, 2006). Previous research has demonstrated robust effects of LTM on the speed of STM consolidation in that familiar stimuli with stronger preexisting LTM can be consolidated into STM faster than less familiar stimuli (Xie & Zhang, 2017b; Blalock, 2015; Lefton & Spragins, 1974; Phillips, 1971). In theory, these consolidation speed effects across different studies could potentially account for the conflicting results on the effects of LTM on STM storage capacity. That is, within a limited amount of encoding time, more familiar stimuli may be encoded into STM because of their faster STM consolidation speed (Xie & Zhang, 2017b; Blalock, 2015; Lefton & Spragins, 1974; Phillips, 1971). However, the increases in the amount of information consolidated into STM because of faster consolidation speed for familiar stimuli will decrease as uninterrupted consolidation time increases, eventually leading to a comparable storage capacity for familiar and unfamiliar information (e.g., Huang, 2011; Olson & Jiang, 2004; Pashler, 1988).
This consolidation speed hypothesis has received some preliminary supports from a recent behavioral study (Xie & Zhang, 2017b), which took advantage of the large individual differences in participants' prior experience with Pokémon characters without the need for intensive laboratory familiarization or training procedures. In the first experiment of Xie and Zhang (2017b), consolidation masks were presented at different intervals after the onset of to-be-remembered Pokémon characters to manipulate the amount of time allowed for visual STM consolidation. Consolidation speed, estimated from the memory performance across different memory-to-mask SOAs, was higher for participants who were more familiar with Pokémon stimuli, as compared with those who were less familiar with Pokémon stimuli. In an additional condition where consolidation was uninterrupted but still incomplete because of a short duration of the memory array, participants with higher Pokémon familiarity retained more familiar Pokémon stimuli in visual STM than did participants with lower Pokémon familiarity. This capacity effect was absent when consolidation was allowed to complete because of a sufficiently long duration of the memory array in the second experiment of Xieand Zhang (2017b).
Although these findings are highly consistent with the consolidation speed hypothesis, they are open to multiple interpretations. First, the comparable storage capacity regardless of stimulus familiarity given sufficient encoding time was found in the second experiment of Xie and Zhang (2017b) without any demonstration of the consolidation speed effect (i.e., faster consolidation for familiar stimuli). It was thus unclear whether this absence of capacity effect resulted from the same storage capacity that was reached for both familiar and unfamiliar stimuli or the lack of effects on consolidation speed in the first place. This potential problem reflects a methodological limitation of behavioral tests of STM consolidation and storage capacity. That is, the consolidation speed effects and the capacity effects (and their relationships) must be assessed across different experimental conditions. Specifically, although a short presentation duration and masking stimuli used to disrupt the consolidation process make it possible to characterize STM consolidation speed, these manipulations will inevitably decrease the amount of information that can be encoded into STM (Xie & Zhang, 2017b; Blalock, 2015; Vogel et al., 2006; Enns & Di Lollo, 2000). Consequently, the intact STM storage capacity will need to be assessed in a condition without significant consolidation interruption. Second, given that behavioral performance in the change detection task could be determined by both the storage capacity of retained representations and the difficulty of the comparisons of the memory and test items (Awh, Barton, & Vogel, 2007; Mitroff, Simons, & Levin, 2004), the behavioral effects of Xie and Zhang (2017b) could result from comparison errors during retrieval instead of STM storage capacity.
To rule out these alternative interpretations, and more importantly to directly test the consolidation speed hypothesis, it is thus pivotal to provide independent measures of STM consolidation and storage simultaneously in the same experimental condition. The contralateral delay activity (CDA), an ERP component that manifests as a sustained negative deflection over parietal-occipital electrodes during visual STM delay period, is ideal for this purpose. First, the size of the CDA increases during STM delay period (Vogel & Machizawa, 2004), reflecting the increase in the amount of information encoded into STM over time (Zhang & Luck, 2008) for both simple and complex visual stimuli (see Fukuda, Awh, & Vogel, 2010, for a review). Second, CDA amplitude is load sensitive and asymptotes at a memory set size consistent with behavioral estimates of STM storage capacity (Vogel, McCollough, & Machizawa, 2005; Vogel & Machizawa, 2004). Last, the CDA is refined to the delay period, void of any effects of retrieval and decisional factors that occur after the onset of the test display (Awh et al., 2007; Mitroff et al., 2004).
In contrast to CDA amplitude, CDA latency reflects the time course, instead of the outcome, of STM consolidation. Measures of CDA latency can thus be used to infer the speed of information encoding into STM (i.e., STM consolidation speed). Supporting the use of CDA latency as the electrophysiological index for STM consolidation, the rate of STM consolidation derived from CDA latency is highly consistent with that derived from the psychophysical masked consolidation procedure (Perez & Vogel, 2012). However, CDA amplitude and latency are not independent of each other. Specifically, CDA maximum latency (the latency that CDA reaches maximum amplitude) or fractional area latency (e.g., 50% fractional area latency, the time point when the CDA reaches 50% of total area under the waveform) is dependent on STM capacity (and hence CDA amplitude; Perez & Vogel, 2012), which will be problematic when both consolidation effects and capacity effects are expected. To deal with this issue, the early-window CDA amplitude measured in a time window before CDA reaches asymptote can be used to infer the amount of information that has already been consolidated into STM (Qi, Ding, & Li, 2014). That is, the larger the early-window CDA amplitude is, the faster sensory information can be consolidated into STM within a given amount of time. In contrast, the late-window CDA amplitude measured in a time window after CDA has reached asymptote (indicating consolidation has stopped when capacity is reached) can be used to infer the outcome of STM consolidation (Vogel et al., 2005; Vogel & Machizawa, 2004). Note, CDA fractional latency is still reported across experiments given its direct relationship with consolidation speed, but it would be less interpretable when a concurrent capacity effect is observed.
This study thus combined the behavioral methods and Pokémon stimuli (see Figure 1) from Xie and Zhang (2017a, 2017b) and the CDA to directly assess how stimulus familiarity influences visual STM consolidation speed and its subsequent effects on STM storage capacity. Different CDA measures were used to track how fast and how much information about Pokémon characters has been consolidated into STM during the delay period. Experiment 1 allowed sufficient encoding time (1000-msec memory array) for STM consolidation to complete. According to the consolidation speed hypothesis, familiar stimuli should be consolidated into STM faster than less familiar stimuli, leading to a smaller CDA latency and larger early-window CDA amplitude. In contrast, the late-window CDA amplitude should be comparable, because similar capacity should be reached for familiar and less familiar stimuli (Xie & Zhang, 2017b). Experiment 2, however, interrupted STM consolidation before its completion due to insufficient encoding time (500-msec memory array) for this type of stimuli (Xie & Zhang, 2017a). Given that familiar stimuli may be consolidated into STM faster than less familiar stimuli before the consolidation process was interrupted (manifesting as larger early-window CDA amplitude for familiar information), more information about familiar stimuli should be successfully encoded into and maintained in STM (manifesting as larger late-window CDA amplitude for familiar information). Nevertheless, because CDA latency measures (e.g., the fractional area latency) can be confounded by differences in the late-window CDA amplitude (Perez & Vogel, 2012), the CDA latency effect is not expected in Experiment 2.
Sixty-four participants between 18 and 35 years old were recruited from University of California, Riverside. Thirty-two students participated in Experiments 1 (mean ± SD: 19.56 ± 1.11 years old, 16 men) and the other 32 participated in Experiment 2 (19.73 ± 1.29 years old, 20 men). All participants had normal or corrected-to-normal visual acuity and reported having normal color vision. Informed consent was obtained at the beginning of the study. Six additional individuals participated the study but were excluded from further analysis because of excessive eye blinks or eye movements during the experiment (more than 35% of trials were rejected because of ocular artifacts). All participants completed the study before the release of the mobile game Pokemon Go.
Stimuli were presented on a CRT monitor, calibrated using an X-Rite I1Pro spectrophotometer (X-Rite, Inc., Grand Rapids, MI), with a gray background (6.1 cd/m2) at a viewing distance of 75 cm. Psychophysics Toolbox (Brainard, 1997) in MATLAB (The MathWorks, Cambridge, MA) was used for stimulus presentation and response collection. For the change detection task, 65 unique first-evolution Pokémon characters were chosen separately for the first generation and the fifth generation (hereafter referred to as the recent generation) Pokémon, yielding a total of 130 different characters that covered all Pokémon types (www.pokemon.com, Nintendo, Japan). These Pokémon characters do not significantly differ in types (e.g., “fire,” “water,” and “grass”), visual style, perceptual and conceptual distinctiveness (as defined in Konkle, Brady, Alvarez, & Oliva, 2010), and complexity (see Xie & Zhang, 2017a, 2017b, for details). Each Pokémon character was rescaled to 4.2° × 4.2° in visual angle. On each trial, eight Pokémon characters were randomly chosen, without replacement, from 65 Pokémon characters within each generation. They were presented at eight equally spaced locations on an invisible circle of 6.5° radius, with four in each visual field.
Change Detection Task
Each trial started with an 800-msec fixation, followed by a 200-msec arrow cue presented above the fixation circle at the center of the screen. The arrow was equally likely to point to the left or right side (see Figure 1). Immediately following the cue, a memory array containing eight Pokémon characters divided in both visual fields were presented for 1000 msec. These Pokémon characters were randomly chosen from the same Pokémon generation. Participants were instructed at the beginning of the experiment to only remember the Pokémon characters on the cue side while maintaining fixation at the center of the screen throughout the trials and hold the memory items through a 1000-msec blank delay interval. A test array then appeared containing one Pokémon character, whose location was randomly chosen from the four possible locations on the cued side, and seven empty squares at the remaining locations. On half of the trials, the tested Pokémon was identical to the one presented in the memory array at the same location (i.e., no change). On the other half of the trials, the tested Pokémon was a new Pokémon character that did not appear in the memory array (i.e., change), although it was from the same generation as those in the memory array. Participants pushed a right-side button on a gamepad if they detected a change or a left-side button on the gamepad to report no change of the test item. Accuracy was emphasized over speed. Each participant completed 200 trials (100 change and 100 no-change) for each Pokémon generation condition (first vs. recent), yielding a total of 400 trials that were divided into five experimental blocks. All experimental factors, including Pokémon generation, change versus no-change trials, cued side (left vs. right), and test location (out of four possible locations on the cued side), were randomly intermixed within each block.
The stimulus and procedure for Experiment 2 were the same as those in Experiment 1 with the following exceptions. That is, the memory array was presented for 500 msec, followed by a delay interval of 1500 msec. The resulting 2000-msec SOA between memory and test arrays was the same as that in Experiment 1 (see Figure 1).
Electrophysiological Recordings and Data Processing
Raw scalp EEG was recorded continuously from 32 Ag/AgCL electrodes mounted in an elastic cap using Biosemi ActiveTwo (Biosemi B.V., Amsterdam, the Netherlands) and later analyzed offline using the EEGLAB toolbox (Delorme & Makeig, 2004) and ERPLAB toolbox (Lopez-Calderon & Luck, 2014). Electrodes were arranged based on the International 10/20 System with an emphasis on posterior electrodes (FP1/2, F3/4, F7/8, Fz, T7/8, C3/4, Cz, P1/2, P3/4, P5/6, P7/8, P9/10, Pz, PO3/4, PO7/8, POz, O1/2, Oz, and Lz). The common mode sense electrode and the driven right leg electrode were placed at site C1 and site C2, respectively. The horizontal EOG was recorded from a pair of electrodes placed lateral to the external canthi, and the vertical EOG was recorded from a pair of electrodes placed above and below the right eye. These EOG electrodes were used to detect horizontal eye movements, eyeblinks, and vertical eye movements. All electrodes were digitized at a sampling rate of 500 Hz and were referenced to the right mastoid electrode during online recording and later to the average of the bilateral mastoids in offline data analyses. Trials with horizontal eye movements, blinks, or excessive muscle movements were excluded from the analysis. Artifact rejection was performed on individual trials. Participants with ocular artifacts rate of 35% and above were excluded from further data analyses (six participants). After artifact rejection, 75% ± 9% trials averaged across participants were included in data analyses, leading to at least 65 trials for each experimental condition per subject. The residual horizontal eye movement traces after artifact rejection were plotted in Figure A1. As shown in the figure, the resulting residual horizontal eye movements time-locked to the memory array onset (0 msec) were minimal throughout the entire trial duration from the cue onset (−200 msec). More importantly, the absolute deflection in voltage during this time window (−200 to 2000 msec) between left and right eye electrodes was less than 1.2 μV in both experiments, corresponding to an average deviation of less than 0.04° from the central fixation point (Lins, Picton, Berg, & Scherg, 1993). These results together suggested that the observed patterns in CDA could not be attributed to eye movements.
Unfiltered EEGs on these trials were averaged, separately for each participant at each condition, and time-locked to the onset of the memory array. The resulting ERPs elicited by the memory array were then collapsed across lateral positions of the electrodes (left or right) and to-be-remembered hemifield (left or right) to obtain waveforms contralateral and ipsilateral to the to-be-remembered stimuli. CDA waveforms were then constructed as the difference waveforms between contralateral and ipsilateral ERPs averaged from five pairs of posterior electrode sites (P3/P4, P5/P6, P7/P8, PO3/PO4, /PO7/PO8/) relative to the 200-msec prestimulus baseline (Qi et al., 2014; Störmer, Li, Heekeren, & Lindenberger, 2013). Three key measures of the CDA waveform were assessed for each participant under each experimental condition. They were (1) CDA 50% fractional area latency that was the time point dividing the area under the CDA curve from 300 to 2000 msec poststimulus onset into two equal regions, (2) early-window CDA amplitude measured between 300 and 800 msec after the onset of the memory array (Qi et al., 2014), and (3) late-window CDA amplitude measured between 1500 and 2000 msec after the onset of the memory array. Note that the two time windows were selected based on the rising phase and the asymptote phase of the CDA in the current study and also based on some previous studies (e.g., Qi et al., 2014). Furthermore, the two measurement windows were chosen to be equal in length (500 msec), yielding comparable measurement reliability for the statistical comparisons of the mean CDA amplitude.
Following the change detection task, participants provided visual familiarity rating for all 130 Pokémon characters used in the study. In this rating task, a Pokémon was presented at the center of the screen on each trial, and participants reported their familiarity to the presented Pokémon using a 6-point familiarity scale (from not familiar at all to very familiar), which was shown at the bottom of the screen. No EEG was recorded in this familiarity rating task. The average rating of familiarity across 65 Pokémon in each generation of Pokémon was used to index the level of phenomenological visual familiarity for the corresponding Pokémon generation of a participant (Xie & Zhang, 2017b).
Similar to previous studies (Xie & Zhang, 2017b), the familiarity ratings across participants from the 6-point scale tended to be higher and more variable for the first-generation Pokémon (Experiment 1: median = 4.45, interquartile range [IQR] 3.15, 5.70; Experiment 2: median = 4.60, IQR 2.86, 5.46) than the recent-generation Pokémon (Experiment 1: median = 1.62, IQR 1.17, 2.45; Experiment 2: median = 1.30, IQR 1.31, 2.08). Furthermore, on average, over 78% of participants provided ratings smaller than 3 on the 6-point familiarity scale for recent-general Pokémon stimuli, in contrast to only 26% of the participants providing ratings smaller than 3 for first-general Pokémon stimuli. The consistently low familiarity rating for the recent-generation Pokémon suggested that most participants were less familiar with the recent-generation Pokémon (ps < .0001, Wilcoxon sign-rank tests for comparisons of medians). Thus, median-splits were performed on familiarity ratings of the first-generation Pokémon across participants, yielding a group of participants with higher familiarity and a group of participants with lower familiarity for the first-generation Pokémon (also see Xie & Zhang, 2017a, 2017b). As a result, in a subsequent 2 (Subject group: high vs. low familiarity) × 2 (Pokémon generation: first vs. recent) mixed-effect repeated-measure ANOVAs, there were significant interactions between Pokémon generation and Subject group on familiarity ratings (Experiment 1: F(1, 30) = 40.96, p < .001, ηp2 = .577; Experiment 2: F(1, 30) = 12.81, p = .001, ηp2 = .299). These interactions were driven by a larger difference on Pokémon familiarity between the high-familiarity group and the low-familiarity group for the first-generation Pokémon (Experiment 1: t(30) = 8.94, p < .001, Cohens' d = 3.26; Experiment 2: t(30) = 9.73, p < .001, Cohens' d = 3.55), as compared with that for the recent-generation Pokémon (Experiment 1: t(30) = 1.35, p = .19, Cohen's d = 0.49; Experiment 2: t(30) = 3.40, p = .002, Cohen's d = 1.24). Furthermore, these two-way interactions were similar across the two experiments, as indicated by a nonsignificant three-way interaction of Experiment, Pokémon generation, and Subject group, F(1, 60) = 1.99, p = .16, ηp2 = .032, in a 2 (Experiment: Experiment 1 vs. Experiment 2) × 2 (Subject group: high vs. low familiarity) × 2 (Pokémon generation: first vs. recent) mixed-effect repeated-measure ANOVA. Note that the two groups of participants did not significantly differ in age or sex ratio in either experiment (all ps > .16).
Change detection performance was assessed using Cowan's K (set size × [hit rate − false alarm rate]), which represents the number of remembered stimuli (Rouder, Morey, Morey, & Cowan, 2011; Cowan, 2001). A similar 2 (subject group: high vs. low familiarity) × 2 (Pokémon generation: first vs. recent) mixed-effect repeated-measure ANOVA was performed on Cowan's K. Furthermore, we also examined whether the individual differences in relative familiarity from recent- to first-generation Pokémon was associated with the increase in K from recent- to first-generation Pokémon (Xie & Zhang, 2017b).
CDA measures were analyzed in a similar manner as Cowan's K. That is, of primary interest, separate 2 (Subject group: high vs. low familiarity) × 2 (Pokémon generation: first vs. recent) mixed-effect repeated-measure ANOVAs were applied to the 50% fractional area latency, the early-window CDA amplitude, and the late-window CDA amplitude for different experiments. Additional correlation analyses were performed based on the difference scores of the measures between first- and recent-generation Pokémon conditions to test whether individual differences in relative familiarity between Pokémon generations was associated with effects on CDA measures between Pokémon generations.
As shown in Figure 2A, behavioral performance assessed as Cowan's K in the change detection task seemed to be comparable across conditions and subject groups. A 2 (Subject group: high vs. low familiarity) × 2 (Pokémon generation: first vs. recent) mixed-effect repeated-measure ANOVA yielded nonsignificant main effects of Pokémon generation, F(1, 30) = 2.98, p = .095, ηp2 = .090, and Subject group (F < 1) on Cowan's K. More importantly, there was no significant interaction (F < 1) between Pokémon generation and Subject group, in sharp contrast to the significant two-way interaction between Pokémon generation and Subject group on familiarity ratings (see Data Analysis in Methods). In addition, individual differences in the changes of familiarity ratings from recent- to first-generation Pokémon was not significantly correlated with individual differences in the changes of K from recent- to first-generation Pokémon across participants (r = .17 [−.19, .49], p = .35; Figure 2B). These results, consistent with the findings in the second experiment of Xie and Zhang (2017b), provided little evidence that prior familiarity with the memory items could substantially increase STM storage capacity when STM encoding time was sufficient.
Contralateral Delay Activity
The nonsignificant effect on K was mirrored by nonsignificant effect on the late-window CDA amplitude after CDA reached asymptote (1500–2000 msec; Figure 3A, B, and E). That is, no significant main effect of Pokémon generation, main effect of Subject group, or interaction was observed (all ps > .24). However, CDA latency (Figure 3G) and the early-window CDA amplitude (Figure 3C) showed different patterns. Although the main effects of Subject group and Pokémon generation were not significant (all ps > .19), there was a significant interaction effect between Subject group and Pokémon generation on the early-window (300–800 msec) CDA amplitude, F(1, 30) = 7.41, p = .011, ηp2 = .198. Specifically, for first-generation Pokémon stimuli, the high-familiarity group showed larger early-window CDA amplitude as compared with the low-familiarity group, t(30) = 2.38, p = .024, Cohen's d = 0.87. In contrast, the early-window CDA amplitude was comparable t < 1) between these two subject groups for recent-generation Pokémon. Consistent with these results, there was a nonsignificant correlation between changes in familiarity ratings between Pokémon generations and changes in the late-window CDA amplitudes between Pokémon generations (r = .21 [−.15, .52], p = .25; Figure 3F), in contrast to a significant correlation between relative familiarity and relative early-window CDA amplitude between Pokémon generations (r = −.44 [−.68, −.11], p = .012; note that CDA is a negative-going component; Figure 3D).
Similarly, there was a significant Subject group and Pokémon generation interaction effect on the 50% fractional area latency, F(1, 30) = 7.31, p = .011, ηp2 = .196 (Figure 3G), even though neither the main effect of Pokémon generation (F < 1) nor the main effect of Subject group on the 50% fractional area latency, F(1, 30) = 3.29, p = .080, ηp2 = .104, was significant. Specifically, for first-generation Pokémon, participants in the high-familiarity group reached 50% area under the CDA waveform earlier than those in the low-familiarity group, t(30) = 2.68, p = .012, Cohen's d = 0.98. This pattern was absent for recent-general Pokémon (t < 1). More importantly, there was a significant negative correlation between relative familiarity and relative 50% fractional area latency between Pokémon generations (r = −.51 [−.73, −.20], p = .0028, Figure 3H). That is, the morefamiliar the first-generation Pokémon was relative to recent-generation Pokémon, the faster the first-generation Pokémon could be consolidated into visual STM, as compared with the recent-generation Pokémon.
With a shorter memory array duration of 500 msec, significant differences in Cowan's K started to emerge in Experiment 2 (Figure 4). Of primary interest, there was a significant Subject group and Pokémon generation interaction effect on Cowan's K, F(1, 30) = 7.66, p = .010, ηp2 = .203, such that the number of retained Pokémon stimuli differed significantly between the two subject groups for first-generation, t(30) = 2.37, p = .024, Cohen's d = 0.87, but not for recent-generation Pokémon (t < 1). These was also a marginally significant main effect of Pokémon generation, F(1, 30) = 3.44, p = .074, ηp2 = .103, such that participants tended to remember more familiar first-generation Pokémon than less familiar recent-generation Pokémon (K: 1.90 ± 0.49 vs. 1.81 ± 0.56), but the main effect of Subject group was not significant, F(1, 30) = 1.96, p = .17, ηp2 = .061. Furthermore, the difference in Cowan's K between Pokémon generations, which partialled out the individual difference in general visual STM capacity, was also significantly correlated with the relative Pokémon familiarly between generations (r = .44 [.11, .69], p = .011). Note that these findings were highly consistent with the results of a previous study (Xie & Zhang, 2017a).
The significant Subject group and Pokémon generation interaction effect in Cowan's K was supported by a subject group and Pokémon generation interaction effect in both the early-window, F(1, 30) = 4.47, p = .043, ηp2 = .130 (Figure 5C) and the late-window CDA amplitude, F(1, 30) = 5.24, p = .029, ηp2 = .149 (Figure 5E). That is, the high-familiarity group, as compared with the low-familiarity group, had larger CDA amplitudes for first-generation Pokémon stimuli (early-window: t(30) = 2.08, p = .046, Cohen's d = 0.76; late-window: t(30) = 2.29, p = .029, Cohen's d = 0.84). These patterns were absent for recent-generation Pokémon (ts < 1). Across participants, changes in familiarity between Pokémon generations significantly correlated with changes in CDA amplitude from recent- to first-generation Pokémon for both the early-window (r = −.55 [−.76, −.25], p = .0011; Figure 5D) and the late-window CDA amplitudes (r = −.43 [−.68, −.10], p = .014; Figure 5F).
The difference in late-window CDA amplitude suggested that the low-familiarity group could not fully consolidate their visual STM representations with the limited exposure, whereas the high-familiarity group could better consolidate the memory items into STM. Because CDA 50% fractional latency from 300 to 2000 msec would be contingent on late-window CDA amplitude (e.g., the time to reach 50% area under an ERP waveform will increase as the area under the waveform increases), this measure was not expected to be indicative of consolidation speed. Indeed, no significant main effect of subject group, Pokémon generation, or interaction between these two factors was obtained (ps > .22; Figure 5G). In addition, no significant correlation was observed between changes in familiarity and changes in CDA 50% fractional area latency from recent- to first-generation Pokémon (r = .25 [−.11, .55], p = .17; Figure 5H). Nonetheless, the significant difference in early-window CDA amplitude replicated the consolidation speed findings observed in Experiment 1.
This study tested how preexisting LTM, operationalized as stimulus familiarity previously acquired through real-life experience, affected visual STM consolidation speed (processing) and storage capacity (representation). STM consolidation speed was estimated using measures of CDA latency and the early-window CDA amplitude. The number of retained memory representations was estimated using the late-window amplitude. The results in Experiment 1 showed that familiar information elicited smaller CDA fractional area latencies and larger early-window CDA amplitudes, suggesting that more familiar information can be consolidated into STM within a given encoding time window (i.e., faster consolidation). In contrast, comparable capacity (similar late-window CDA amplitude) was observed regardless of stimulus familiarity, because the same storage capacity could be reached because of the sufficient time for the completion of STM consolidation. Critically, differences in stimulus familiarity in Experiment 1 could significantly predict differences in the two ERP estimates of consolidation speed (early-window CDA amplitude and CDA fractional latency) across participants, but not differences in STM storage capacity (K and late-window CDA amplitude). In Experiment 2, although STM consolidation was incomplete because of a shorter duration of the memory array, the consolidation speed effect still manifested in the early-window CDA amplitude. More importantly, faster consolidation of familiar stimuli was accompanied by larger storage capacity (K and the late-window CDA amplitude) and differences in stimulus familiarity could significantly predict differences in STM capacity across participants. Together, these findings provide strong support for the consolidation speed hypothesis regarding the effect of preexisting LTM on STM.
The consolidation speed hypothesis, although consistent with the findings that more information can be retained for familiar stimuli under certain circumstances (e.g., when STM encoding is short in duration or interrupted by masking stimuli), is fundamentally different from the time-limit hypothesis in which STM storage capacity is determined by STM processing speed (e.g., Baddeley, Thomson, & Buchanan, 1975). That is, the consolidation speed hypothesis does not imply that STM storage capacity is inherently determined by the rate of consolidation. Instead, effects on the number of retained stimuli can be by-product/artifact of effects on consolidation speed when faster consolidation leads to more information encoded into STM within the limit of STM storage capacity (Jannati, McDonald, & Di Lollo, 2015). Consequently, the consolidation speed hypothesis is in line with the capacity-limit hypothesis that predicts an upper limit in the amount of information represented and maintained in STM that is independent of consolidation speed (Zhang & Luck, 2008; Cowan, 2001; Luck & Vogel, 1997).
The current results rule out some alternative explanations for Xie and Zhang (2017b) and provide the first electrophysiological evidence, to the best of our knowledge, to integrate previously reported behavioral effects of preexisting LTM on STM consolidation (Blalock, 2015; Howe & Brandau, 1983; Lefton & Spragins, 1974; Phillips, 1971) and storage capacity (Curby et al., 2009; Jackson & Raymond, 2008). Given that overall effects based on behavioral measures in STM tasks could result from factors orthogonal to STM storage capacity, such as comparison errors during the response period (Awh et al., 2007), the present ERP findings are of great importance to directly relate STM consolidation effects to capacity effects, as the CDA measures are obtained before behavioral responses. More importantly, these findings could also reconcile some previous conflicting findings on STM capacity for stimuli with preexisting LTM representations (Huang, 2011; Curby et al., 2009; Jackson & Raymond, 2008; Luck, 2008; Olson & Jiang, 2004; Pashler, 1988). That is, the presence or absence of STM storage capacity effects may be contingent on whether STM consolidation can complete or not. The capacity effect may occur if performance is limited by the amount of information that can be consolidated before capacity is reached, either because of consolidation masking or insufficient duration of the memory array (e.g., Xie & Zhang, 2017a, 2017b). If, however, sufficient STM encoding time is permitted, then similar capacity will be reached for both familiar and less familiar stimuli (Xie & Zhang, 2017b). Last, the STM consolidation speed effect of familiarity could also account for improved performance for familiar stimuli in working memory tasks that tap more on the processing aspect (e.g., n-back task; Reder, Liu, Keinath, & Popov, 2015), beyond the storage capacity aspect of working memory, namely STM.
Furthermore, the current findings also add to the growing literature for the benefits of stimulus familiarity on cognitive processing. For example, familiarity can improve memory for visual features (e.g., orientation and location) that are orthogonal to intrinsic features of visual familiarity (Balas, Cox, & Conwell, 2007). It can also improve LTM encoding, leading to deeper encoding (Craik & Lockhart, 1972), more efficient chunking (Gobet et al., 2001), and better binding between item and contextual information (Reder et al., 2013; Poppenk & Norman, 2012). Consequently, facilitation of STM by preexisting LTM is one critical assumption for some previous explanations of the effects of familiarity on learning (Reder et al., 2015; Reder, Paynter, Diana, Ngiam, & Dickison, 2007).
In terms of the underlying mechanisms, the consolidation effect of stimulus familiarity on STM may be supported by enhanced neural processing of expertise and familiarity-related information (James & James, 2013; Stern, Sherman, Kirchhoff, & Hasselmo, 2001). For instance, a recent study showed that long-time exposure to Pokémon stimuli could modulate fusiform face area (FFA) activities such that participants with higher Pokémon familiarity showed greater FFA activation when viewing Pokémon stimuli (James & James, 2013). Furthermore, familiar information maintained temporally in working memory can activate hippocampus (Stern et al., 2001), providing a direct interface between STM and LTM. It is possible that these familiarity-dependent or expertise-related neural effects could account for the current STM consolidation effects. That is, the changes in neural activities could reflect a more efficient construction of internal representations for familiar stimuli (Zimmer, Popp, Reith, & Krick, 2012) and thereby speeding up STM consolidation via the functional connectivity of FFA and other brain regions supporting active information maintenance in STM (see Gazzaley, Rissman, & D'Esposito, 2004, for details). If this is the case, the CDA latency effect should be generalized to other visual objects of expertise as well (Sørensen & Kyllingsbæk, 2012; Curby & Gauthier, 2007, 2009; Curby et al., 2009). These predictions remain to be tested in future research.
Could the between-group differences in CDA amplitude account for the observed effects? We find this alternative account unlikely, because both groups of participants have produced highly comparable CDA waveforms for the recent-generation Pokémon, with which most participants were unfamiliar. This serves as a good control for generic individual differences in CDA waveforms, suggesting that the observed effect in early CDA 50% fractional area latency for familiar information is not likely to be confounded by generic individual differences. Furthermore, individual differences in relative Pokémon familiarity (first- vs. recent-generation) were strongly correlated with individual differences in relative CDA 50% fractional area latency (first- vs. recent-generation) in Experiment 1, suggesting that the observed effects are not likely to be an artifact due to dichotomizing participants into two groups.
Some exceptions to the present findings are expected. First, novelty can also improve memory performance (Mayer, Kim, & Park, 2011) in that the salience in the novel stimulus can attract attentional resources upon the initial perceptual encoding, which could in turn facilitate STM consolidation (Schmidt, Vogel, Woodman, & Luck, 2002) and boost the amount of information that can be retained in STM. Second, when chunking is possible (Cowan, 2001), capacity effects could occur independent of the consolidation effects. Lastly, although it was demonstrated in several previous behavioral studies that the familiarity-based enhancement of STM performance could not be entirely attributed to verbal encoding (Xie & Zhang, 2017a, 2017b; Jackson & Raymond, 2008), this study did not use the articulatory suppression procedure to prevent verbal (subvocal) encoding or conceptual encoding (Endress & Potter, 2012). It remains possible that verbal encoding, in addition to visual encoding, could potentially allow participants to recruit more LTM without changing STM consolidation speed, leading to increases in the amount of retained information.
In conclusion, the current study provides strong behavioral and ERP evidence for faster STM consolidation of information with strong preexisting LTM. These findings could help us understand how prior knowledge and expertise facilitate learning in the general population and potentially provide additional tools to address learning difficulties in various clinical populations (Reder et al., 2007, 2015).
Both authors contributed to concept development, experimental design, and manuscript preparation. W. Xie programmed the task and performed data analyses. We thank Marcus Cappiello, Julieanne Ong, Debora Handojo, Miriam Rizk, Hanna Boparai, Adrian Pardo, and Aaron Koay for assistance in data collection.
Reprint requests should be sent to Weiwei Zhang, Department of Psychology, University of California, Riverside, 900 University Ave., Riverside, CA 92521, or via e-mail: firstname.lastname@example.org.