Abstract

Over 350 years ago, Descartes proposed that the neural basis of consciousness must be a brain region in which sensory inputs are combined. Using fMRI, we identified at least one such area for working memory, the limited information held in mind, described by William James as the trailing edge of consciousness. Specifically, a region in the left intraparietal sulcus was found to demonstrate load-dependent activity for either visual stimuli (colored squares) or a combination of visual and auditory stimuli (spoken letters). This result was replicated across two experiments with different participants and methods. The results suggest that this brain region, previously well known for working memory of visually presented materials, actually holds or refers to information from more than one modality.

INTRODUCTION

Extensive research on working memory suggests it is consistent with William James' concept of primary memory (James, 1890), the trailing edge of the conscious present. When one presents multiple items and immediately tests how many can be remembered, people can remember at most about four items, no matter whether these are visual objects in an array (Luck & Vogel, 1997; Sperling, 1960) or words in a list (Darwin, Turvey, & Crowder, 1972), provided that rehearsal and grouping strategies are not possible (Cowan, 2001). The similarity in capacity limits across different types of stimuli might be attributed to similar modules for visual and spoken items, as asserted in an earlier theory of working memory (Baddeley, 1986). Another possibility, though, is that a single mental faculty (whether neurophysiologically represented by a single brain area or a network of areas) supports information from multiple modalities. They could hold information in an abstract, conceptual form, or they could hold pointers to modality-specific regions that store the original information (e.g., see Ruchkin, Grafman, Cameron, & Berndt, 2003).

Such multimodal or abstract working memory areas could be limited in their capacity (Chein & Fiez, 2010; Baddeley, 2001; Cowan, 1995). That possibility is reinforced by the finding of a capacity limit that applies to bisensory arrays that include spoken digits and colored squares (Saults & Cowan, 2007).

Brain regions underlying abstract, conscious recollection would have to combine inputs from different sensory channels, as Descartes suggested in 1640 (Cottingham, Stoothoff, Murdoch, & Kenny, 1991). To search for such regions, we adapted the bisensory working memory procedure for use in fMRI. Bisensory procedures have been used, in which an acoustic working memory task was combined with a visual processing task (Klemen, Büchel, Bühler, Menz, & Rose, 2010; Rissman, Gazzaley, & D'Esposito, 2009), but, to our knowledge, we are the first to combine acoustic verbal and visual nonverbal working memory tasks on the same trial in an fMRI procedure.

We examined increases in the BOLD signal strength, an index of neural activation (Christ, Van Essen, Watson, Brubaker, & McDermott, 2009; Bandettini, Jesmanowicz, Wong, & Hyde, 1993), as visual or spoken stimuli or both were to be held in mind during a waiting period before a recognition test. The methods were adapted from previous single-modality tests of working memory using fMRI (Jonides et al., 2008; Cohen et al., 1997). In two experiments, participants were presented with a ready signal and then visual or spoken memoranda or a combination of the two, a waiting period of several seconds, and finally a probe item to be judged present or absent from the set of memoranda.

On the basis of prior research, it was expected that the left and/or right intraparietal sulcus (IPS) would be involved in abstract or multimodal working memory. Regions in both the left and right IPS have shown increased activation in response to an increase in the number of visual memoranda. Unlike other brain areas, activity in the IPS has been shown to reach an asymptote when no more items can be retained in working memory (Xu & Chun, 2006; Todd & Marois, 2004). Other research shows that the IPS also responds to printed verbal memoranda (Majerus et al., 2006, 2010). These findings suggest that the IPS might represent various items from multiple modalities up to the working memory limit. It might be part of a broader parietal lobe basis of one's current focus of attention (Cowan, 1995; Posner & Petersen, 1990).

From prior research, it is clear that various other brain areas also are involved in working memory maintenance in some way (Jonides et al., 2008; Cohen et al., 1997), although the IPS may be unique in the capacity-limited nature of its working memory activation. The present research goal is simply to establish whether at least one brain area responds to both visual nonverbal and acoustic verbal working memory items in a manner robust enough to survive across two experiments with important, theoretically motivated differences in their experimental methods. If the left and/or right IPS turn out to be among those areas, an added benefit will be convergence from the literature, indicating that these special capacity-limited regions are also regions for multimodal (or amodal) working memory maintenance.

EXPERIMENT 1

In the first experiment, the items to be retained in memory included only visual items, either two colored squares (2vis condition) or four colored squares (4vis condition), or both visual and acoustic items, two colored squares plus two spoken letters (2vis2aud condition). The question was whether there are brain areas that show an increase not only between the 2vis and 4vis conditions but also between the 2vis and 2vis2aud conditions. In the latter comparison, both visual and acoustic items contribute to the memory load.

Two additional precautions were taken in Experiment 1 to ensure that abstract working memory was observed (Figure 1A). First, every trial included an array of four colored squares and, simultaneously, a list of four spoken letters, making the initial perception of stimuli identical across conditions. The stimuli were followed by a cue, indicating which items were to be retained in working memory during the following 8-sec waiting period, the maintenance period during which the MRI signal was of special interest. Second, the retention cue was followed by a meaningless, bisensory pattern or mask to eliminate any lingering sensory memory of how each item looked or sounded (Saults & Cowan, 2007).

Figure 1. 

Procedure in two fMRI experiments on intersensory working memory. (A) Experiment 1. Four colored squares and four spoken letters were presented on each trial, and a subsequent cue indicated which ones to retain. The cue arrows pointed to the colored squares to be retained, and a series of two digits indicated the serial positions of spoken letters to be retained; when no letters were to be retained, two asterisks were presented instead. After a postperceptual mask and an 8-sec blank period, a probe item, a color or letter, was to be judged present or absent from the memoranda. When the probe was a color, it appeared in the same location as an array item and was the same color as that item or was a color not found in the array. When the probe was a letter (not shown), it was printed in the center of the screen. (B) Experiment 2 differed from the first experiment in several ways. All presented stimuli were to be remembered and included two colored squares, two spoken letters, two of each, or four colored squares. The probe was a color, located as in the first experiment, or a spoken letter (not shown). Participants in this experiment whispered “the” repeatedly during the presentation to suppress verbal rehearsal.

Figure 1. 

Procedure in two fMRI experiments on intersensory working memory. (A) Experiment 1. Four colored squares and four spoken letters were presented on each trial, and a subsequent cue indicated which ones to retain. The cue arrows pointed to the colored squares to be retained, and a series of two digits indicated the serial positions of spoken letters to be retained; when no letters were to be retained, two asterisks were presented instead. After a postperceptual mask and an 8-sec blank period, a probe item, a color or letter, was to be judged present or absent from the memoranda. When the probe was a color, it appeared in the same location as an array item and was the same color as that item or was a color not found in the array. When the probe was a letter (not shown), it was printed in the center of the screen. (B) Experiment 2 differed from the first experiment in several ways. All presented stimuli were to be remembered and included two colored squares, two spoken letters, two of each, or four colored squares. The probe was a color, located as in the first experiment, or a spoken letter (not shown). Participants in this experiment whispered “the” repeatedly during the presentation to suppress verbal rehearsal.

Methods

Participants

The final sample of 16 participants included 9 men and 7 women, ranging from 18 to 24 years. Another five participants were excluded from the analyses because of excessive head motion in the scanner.

Behavioral Apparatus, Task, and Stimuli

Visual stimuli were displayed in the scanner by computer using an LCD projector, and responses were recorded using a fiber-optic switch. Sounds were presented through MRI-compatible headphones (Optime 1, MR Confon, Magdeburg, Germany) at a comfortable listening level.

Six practice trials outside the MRI scanner were followed by 120 test trials during the scans. Each condition in the experiment (2vis, 4vis, and 2vis2aud; see text) occurred equally often in the experiment. The 2vis2aud condition ended in a probe of a visual item in half the trials and a probe of an acoustic item in the other half. In each condition, the probe was the same as one of the studied items half the time and the task was to indicate whether it was the same as a studied item or not by pressing one of two response buttons.

The order of conditions was separately randomized for each participant. The timing of events for each trial is shown in Figure 1A. The visual display area subtended approximately 3.8° vertically and 3.6° horizontally. Individual colored squares were 0.45° and appeared in the study array at the four corners of a diamond shape with a horizontal diameter (from the center of the leftmost square to that of the rightmost square) of 3.14° and a vertical diameter of 3.34°. There was, thus, a separation of 2.28° between the centers of any two adjacent squares in the display. The four squares were randomly selected without replacement from the easily discriminable set {red, blue, violet, green, yellow, black, white, and cyan}. Four spoken letters were selected without replacement from the set {b, c, f, h, j, l, q, r} and were presented at a rate of 750 msec per item starting at the beginning of the array.

In the postcue, arrows pointed to the colors to be retained in memory and, when two letters were to be retained, digits appeared in the center of the display to indicate their serial positions. The visual masking stimulus was the same as in Saults and Cowan (2007), with identical multicolored squares at the four locations in which colored squares had occurred. The acoustic mask, presented at the same time as the visual mask, was a superimposition of all eight of the spoken letters used in the experiment. When the probe stimulus was a colored square, it was either identical to the square that occupied the same location or was not found at all in the studied array. Similarly, when the probe was a letter printed in the center of the screen, it was either the same as one of the spoken letters in the studied list or was a letter not included in that list. A randomly selected period between 0 and 16 sec was inserted between trials for purposes of trial jittering (see below).

Neuroimaging Data Acquisition

Scans were obtained on a 3T Siemens Trio scanner with a standard eight-channel head coil. For alignment purposes, a set of structural images was collected first using standard high-resolution (1 mm3) T1- and T2-weighted 3-D pulse sequences (MPRAGE and SPACE, respectively). Next, functional images were collected using a T2*-weighted EPI pulse sequence. For each functional run, sets of 32 contiguous, 4.0-mm-thick axial slice images (TR = 2000 msec, TE = 30 msec, 4.0 × 4.0 mm in-plane resolution) were acquired parallel to the anterior–posterior commissure plane; this procedure offered whole-brain coverage at a high signal-to-noise ratio. There were 10 functional runs, each beginning with 5 TRs and ending with 10 TRs of fixation. In each run, 12 task trials, each lasting for 24 sec, were intermixed with 36 TRs of fixation for jittering purposes (total run length = 195 TRs).

Processing and Analysis of fMRI Data

Functional imaging data were preprocessed and analyzed using BrainVoyager QX software (version 1.10; Brain Innovation, Maastricht, the Netherlands). Preprocessing steps included slice scan time correction, 3-D motion correction, transformation to standardized atlas space (Talairach & Tournoux, 1988), and spatial smoothing (6 mm FWHM) to accommodate variations in activation loci across participants.

Statistical analysis was conducted using a random effects general linear model approach. Estimated parameter values were derived separately for the three trial types (2vis, 4vis, and 2vis2aud) using a finite impulse–response model. Condition-specific responses were modeled for 20 time points (40 sec) immediately following the onset of the ready signal. Consistent with finite impulse–response modeling, no assumptions regarding the shape or timing of the hemodynamic response were made.

To minimize any effect of the processing of the cue, we concentrated on the BOLD signal associated with neural activity near the end of the waiting period by examining a 4-sec period starting 18 sec after the onset of the ready signal. We defined the ROIs according to a conjunction analysis (Nichols, Brett, Andersson, Wager, & Poline, 2005), as those that displayed a working memory load effect not only with a purely visual load (i.e., a difference between the 2vis and 4vis conditions) but also with a mixed audiovisual load (i.e., a difference between the 2vis and 2vis2aud conditions). The resulting statistical maps were corrected for multiple comparisons using cluster-size thresholding (Goebel, Esposito, & Formisano, 2006; Forman et al., 1995), with a resulting α level equivalent of p < .05 (false discovery rate [FDR] corrected). The corresponding voxel level and cluster extent thresholds for the critical contrast map in this experiment [(4vis > 2vis) && (2vis2aud > 2vis)] was t(15) > 2.2, with a cluster size > 25 voxels.

Results and Discussion

Behavioral Results

Table 1 shows the behavioral results in terms of both proportion correct and the k measure of items held in working memory, which corrects for guessing (Cowan, 2001). It is clear that the performance was good in all conditions and in a range comparable to the most closely related behavioral study (Saults & Cowan, 2007).

Table 1. 

Behavioral Results in Both Experiments

Measure
Study Condition
2vis
2aud
4vis
2vis2aud (vis probe)
2vis2aud (aud probe)
2vis2aud (total)
Experiment 1 
Same 0.91 (.03) – 0.91 (.02) 0.82 (.03) 0.84 (.05) 0.83 (.03) 
Changed 0.96 (.02) – 0.95 (.03) 0.94 (.02) 0.94 (.03) 0.94 (.02) 
k 1.74 (.08) – 3.45 (.16) 1.53 (.08) 1.56 (.13) 3.09 (.16) 
 
Experiment 2 
Same 0.94 (.02) 0.93 (.03) 0.85 (.03) 0.93 (.03) 0.95 (.02) 0.94 (.02) 
Changed 0.93 (.02) 0.97 (.01) 0.88 (.03) 0.96 (.02) 0.96 (.02) 0.96 (.01) 
k 1.76 (.06) 1.79 (.07) 2.90 (.17) 1.76 (.06) 1.81 (.06) 3.58 (.09) 
Measure
Study Condition
2vis
2aud
4vis
2vis2aud (vis probe)
2vis2aud (aud probe)
2vis2aud (total)
Experiment 1 
Same 0.91 (.03) – 0.91 (.02) 0.82 (.03) 0.84 (.05) 0.83 (.03) 
Changed 0.96 (.02) – 0.95 (.03) 0.94 (.02) 0.94 (.03) 0.94 (.02) 
k 1.74 (.08) – 3.45 (.16) 1.53 (.08) 1.56 (.13) 3.09 (.16) 
 
Experiment 2 
Same 0.94 (.02) 0.93 (.03) 0.85 (.03) 0.93 (.03) 0.95 (.02) 0.94 (.02) 
Changed 0.93 (.02) 0.97 (.01) 0.88 (.03) 0.96 (.02) 0.96 (.02) 0.96 (.01) 
k 1.76 (.06) 1.79 (.07) 2.90 (.17) 1.76 (.06) 1.81 (.06) 3.58 (.09) 

Same refers to proportion correct on trials in which the probe was found in the studied set; Changed is the proportion correct on trials in which the probe was not in the studied set. The k measure of items in working memory (Cowan, 2001) is defined as k = n (Same + Changed − 1), where n is the number of items in the tested set (n = 2 for the 2vis and the 2aud conditions; n = 4 for all other conditions).

Neuroimaging Results

The previously described conjunction analysis revealed nine potential ROIs, including the left and right IPS, showing significantly greater activation for both the 4vis and the 2vis2aud conditions as compared with the lesser load, 2vis condition. In Figure 2, these areas are shown partly in blue, where they did not overlap with the areas observed in Experiment 2, and partly in red, where they did overlap.

Figure 2. 

In both experiments, brain regions demonstrating greater activation for the 4vis condition as compared with the 2vis condition in conjunction with greater activation for the 2vis2aud condition as compared with the 2vis condition [(4vis > 2vis) && (2vis2aud > 2vis)]. Experiment 1 results are shown in blue, Experiment 2 results are in green, and the overlap of the resulting two maps is in red. Results are viewed on the inflated surface of an exemplar brain. In all instances, p < .05, FDR-corrected.

Figure 2. 

In both experiments, brain regions demonstrating greater activation for the 4vis condition as compared with the 2vis condition in conjunction with greater activation for the 2vis2aud condition as compared with the 2vis condition [(4vis > 2vis) && (2vis2aud > 2vis)]. Experiment 1 results are shown in blue, Experiment 2 results are in green, and the overlap of the resulting two maps is in red. Results are viewed on the inflated surface of an exemplar brain. In all instances, p < .05, FDR-corrected.

The results of this experiment suggest that there exist areas of the brain that respond to both acoustic–verbal and visual–nonverbal memory loads. This appears to contradict the notion that working memory storage is completely separate for different modalities or domains of storage. It does so in a procedure in which the amount of information to be encoded into working memory was equated across conditions, so that the difference between load conditions can only reflect aspects of memory maintenance, which could include either the storage of the information and the activities that prolong that storage, such as the direction of attention to information to be preserved. The areas identified largely comprise the frontal–parietal network that has previously been associated with working memory tasks (e.g., Chein & Fiez, 2010; Palva, Monto, Kulashekhar, & Palva, 2010; Jonides et al., 2008; Schumacher et al., 1996; Posner & Petersen, 1990).

EXPERIMENT 2

To explore the generality of the findings, in our second experiment, we used new participants and a revised procedure. One question we addressed was how sensitive the findings would be to the procedural details. Additionally, in this second experiment, we added a low-load auditory baseline with just two spoken letters (2aud) that was not included in Experiment 1. Any effect of load that is truly independent of modality should be obtained no matter whether the low-load baseline is visual or auditory in nature.

The procedure was more conventional than in Experiment 1, in that the presented memoranda were all to be remembered (Xu & Chun, 2006; Todd & Marois, 2004). The subsequent retention cue was, therefore, not needed and was eliminated (Figure 1B). Finally, to eliminate the possibility of verbal rehearsal, participants were instructed to whisper the word the continually (thrice per second) throughout task performance, a suppression task similar to that employed in previous behavioral (e.g., Baddeley, 1986) and neuroimaging (Chein & Fiez, 2010) studies.

Methods

Participants

A final sample of 15 participants included seven men and eight women, between the ages of 18 and 20 years, participated. Another participant was omitted because of excessive head movements.

Task and Stimuli

As shown in Figure 1B, the procedure for Experiment 2 was similar to that for Experiment 1, with several exceptions. Instead of presenting the same number of items on each trial followed by a postcue indicating which items to remember, participants were to remember all of the presented colored squares and spoken letters. When there was a letter probe, it was now presented acoustically, like the letters to be remembered. The visual probes for the colors were now presented for a limited time to make them comparable with the spoken probes for the letters. There were four conditions: 2vis, 2aud, 2vis2aud, and 4vis. (The condition 2aud was not present in Experiment 1). Eight practice trials outside the scanner were followed by 160 test trials during the scan, evenly divided among the four conditions. Participants in this experiment whispered the word the thrice per second to suppress rehearsal. As in previous fMRI research with suppression (Chein & Fiez, 2010), the motion correction preprocessing routines proved adequate to eliminate any potential motion artifact related to this procedure.

The scanner details were as in Experiment 1 except that, within the 10 functional runs, there were 16 task trials, each lasting 18 sec and intermixed with 36 TRs of fixation for jittering purposes. As in Experiment 1, each run began with 5 TRs and ended with 10 TRs of fixation, and the total run length was 195 TRs.

Processing and Analysis of fMRI Data

Because of the shortened trial length (6 sec shorter), condition effects were modeled for 17 time points (34 sec) and the maintenance period was defined as a 4-sec period beginning 12 sec after the onset of the ready signal (Time Points 7–8). Otherwise, preprocessing and analysis of the imaging data proceeded in a manner identical to that used for Experiment 1.

Results and Discussion

Behavioral performance levels were in the same range as in Experiment 1 (Table 1).

Two separate conjunction analyses were carried out to address the two previously described questions. First, to examine to what extent the results of Experiment 1 were procedure dependent, we carried out the conjunction analysis using the same conditions as in Experiment 1 [(4vis > 2vis) && (2vis2aud > 2vis)]. An α level of p < .05 FDR-corrected was obtained using a cluster size threshold of >31 voxels and t(14) > 2.1 (Goebel et al., 2006). As shown in Figure 2 and Table 2, primary portions of the frontal–parietal network were active in areas that overlapped considerably in the two experiments.

Table 2. 

Potential Regions of Interest Identified from Conjunction Analysis of Experiment 1 and Experiment 2 Data

Region
Location
BA
Peak Activation
Volume (cm3)
ta
Effect Size (d)
x
y
z
Experiment 1: [(4vis > 2vis) && (2vis2aud > 2vis)] 
Left IPS 7, 39, 40 −27 −49 34 10.5 2.6 0.65 
Left Inferior Frontal Gyrus, Anterior Insula 13, 44 −39 10 0.9 2.4 0.60 
Left Superior Frontal Gyrus, Precentral Gyrus 6, 32 −4 58 19.7 2.8 0.70 
Left Cerebellum NA −48 −52 −23 0.8 2.5 0.63 
Right IPS 40 42 −40 31 3.4 2.8 0.70 
Right Middle Frontal Gyrus 10 39 47 25 4.3 2.7 0.68 
Right Superior Frontal Gyrus 36 −4 52 2.5 2.7 0.68 
 
Experiment 2: [(4vis > 2vis) && (2vis2aud > 2vis)] 
Left IPS 7, 39, 40 −30 −55 34 13.4 2.8 0.72 
Left Inferior Frontal Gyrus, Anterior Insula 13 −27 17 1.1 2.4 0.62 
Left Inferior Frontal Gyrus, Middle Frontal Gyrus 10, 46 −39 41 3.4 2.4 0.62 
Left Medial Frontal Gyrus 6, 32 −3 14 43 3.0 2.5 0.65 
Left Middle Frontal Gyrus 9, 46 −36 20 31 8.6 2.5 0.65 
Right IPS 7, 39, 40 39 −52 49 7.1 2.6 0.67 
Right Middle Frontal Gyrus 9, 46 36 20 28 6.2 2.7 0.70 
Right Inferior Frontal Gyrus, Anterior Insula 13, 44 30 23 −2 1.2 2.4 0.62 
Bilateral Precuneus −73 52 3.1 2.3 0.59 
 
Experiment 2: [(4vis > 2vis) && (2vis2aud > 2vis) && (4vis > 2aud) && (2vis2aud > 2aud)] 
Left IPS 40 −27 −46 31 1.0 2.3 0.59 
Region
Location
BA
Peak Activation
Volume (cm3)
ta
Effect Size (d)
x
y
z
Experiment 1: [(4vis > 2vis) && (2vis2aud > 2vis)] 
Left IPS 7, 39, 40 −27 −49 34 10.5 2.6 0.65 
Left Inferior Frontal Gyrus, Anterior Insula 13, 44 −39 10 0.9 2.4 0.60 
Left Superior Frontal Gyrus, Precentral Gyrus 6, 32 −4 58 19.7 2.8 0.70 
Left Cerebellum NA −48 −52 −23 0.8 2.5 0.63 
Right IPS 40 42 −40 31 3.4 2.8 0.70 
Right Middle Frontal Gyrus 10 39 47 25 4.3 2.7 0.68 
Right Superior Frontal Gyrus 36 −4 52 2.5 2.7 0.68 
 
Experiment 2: [(4vis > 2vis) && (2vis2aud > 2vis)] 
Left IPS 7, 39, 40 −30 −55 34 13.4 2.8 0.72 
Left Inferior Frontal Gyrus, Anterior Insula 13 −27 17 1.1 2.4 0.62 
Left Inferior Frontal Gyrus, Middle Frontal Gyrus 10, 46 −39 41 3.4 2.4 0.62 
Left Medial Frontal Gyrus 6, 32 −3 14 43 3.0 2.5 0.65 
Left Middle Frontal Gyrus 9, 46 −36 20 31 8.6 2.5 0.65 
Right IPS 7, 39, 40 39 −52 49 7.1 2.6 0.67 
Right Middle Frontal Gyrus 9, 46 36 20 28 6.2 2.7 0.70 
Right Inferior Frontal Gyrus, Anterior Insula 13, 44 30 23 −2 1.2 2.4 0.62 
Bilateral Precuneus −73 52 3.1 2.3 0.59 
 
Experiment 2: [(4vis > 2vis) && (2vis2aud > 2vis) && (4vis > 2aud) && (2vis2aud > 2aud)] 
Left IPS 40 −27 −46 31 1.0 2.3 0.59 

aDegrees of freedom (df) for Experiment 1 and Experiment 2 analyses = 15 and 14, respectively; p < .05 FDR-corrected in all instances. Effect size d is calculated as the t value divided by the square root of (df + 1).

BA = approximate Brodmann's area(s).

Next, we carried out an additional conjunction analysis using all possible contrasts of low-load versus high-load conditions in Experiment 2 [(4vis > 2vis) && (2vis2aud > 2vis) && (4vis > 2aud) && (2vis2aud > 2aud)] to determine whether any brain areas were involved in working memory maintenance in a manner general enough to be present in all of these contrasts. [An α level of p < .05 FDR-corrected was obtained using a cluster size threshold of >32 voxels and t(14) > 2.1 (Goebel et al., 2006).] Table 2 and Figure 3 show that only one brain area emerged from this conjunction analysis as an ROI: the left IPS. Thus, what was important for activity in this region was primarily the number of items to be remembered rather than the modalities of those items. (In the right IPS, in contrast, the activity in the 2aud condition was elevated to be very similar to both the 4vis and the 2vis2aud conditions, so load comparisons with 2aud as the low-load baseline did not approach significance.)

Figure 3. 

Conjunction across all available low-load versus high-load condition contrasts in both experiments. Experiment 1, in blue, reflects brain regions demonstrating greater activation for the 4vis condition as compared with the 2vis condition in conjunction with greater activation for the 2vis2aud as compared with the 2vis condition [(4vis > 2vis) && (2vis2aud > 2vis)]. Experiment 2, in green, reflects the conjunction of four different contrasts: the same ones as in the first experiment plus contrasts in which the low-load condition comprised two spoken letters [(4vis > 2vis) && (2vis2aud > 2vis) && (4vis > 2aud) && (2vis2aud > 2aud)]. Overlap of the resulting two maps (shown in red) revealed a single region centered in the left IPS (x = −27, y = −46, z = 31). Results are viewed on the inflated surface of an exemplar brain. In all instances, p < .05, FDR-corrected.

Figure 3. 

Conjunction across all available low-load versus high-load condition contrasts in both experiments. Experiment 1, in blue, reflects brain regions demonstrating greater activation for the 4vis condition as compared with the 2vis condition in conjunction with greater activation for the 2vis2aud as compared with the 2vis condition [(4vis > 2vis) && (2vis2aud > 2vis)]. Experiment 2, in green, reflects the conjunction of four different contrasts: the same ones as in the first experiment plus contrasts in which the low-load condition comprised two spoken letters [(4vis > 2vis) && (2vis2aud > 2vis) && (4vis > 2aud) && (2vis2aud > 2aud)]. Overlap of the resulting two maps (shown in red) revealed a single region centered in the left IPS (x = −27, y = −46, z = 31). Results are viewed on the inflated surface of an exemplar brain. In all instances, p < .05, FDR-corrected.

In one possible interpretation of the findings, the left IPS represents only visual information but has to be more active to maintain that information in the presence of an additional, auditory–verbal memory load. To investigate that possibility, we compared the 2aud and 2vis2aud conditions to find out whether the additional left IPS activity in the latter condition has a behavioral correlate. In fact, the 2vis2aud-2aud BOLD contrast in the left IPS was negatively related to the loss of auditory items from working memory, as estimated from behavioral responses (specifically, the 2aud-2vis2aud difference in auditory items in memory), r(14) = −.50, p < .05, one-tailed. Given that the extent of increased BOLD activity with the addition of two visual items predicted how well memory for two auditory items was protected, preservation of auditory information may well depend on IPS activity.

Comparisons across Experiments

Additional observations pertain to the results of the two experiments in comparison with one another, revealing important similarities and differences between experiments. These occur in the behavioral results, the neuroimaging results, and the correlations between the two.

Behavioral results

Table 1 shows that in Experiment 1, the 4vis condition was easier than the 2vis2aud condition whereas in Experiment 2, the opposite was true. We speculate that this difference may result from a difference in the allocation of attention at the time of encoding for the following reasons. In Experiment 1, participants always had to encode four items in each modality and were later postcued as to what they needed to remember during the retention interval. If postcued to remember all four colors, that would have been relatively easy to access because they are all from the same physical channel. On the other hand, if postcued to remember two items from each modality, the encoding is a little more difficult because it probably involves attention-switching from one modality to the other (e.g., see Johnston & Heinz, 1978; Broadbent, 1958). In Experiment 2, on the other hand, there was no selective encoding, but in three of four trial types, the most that was presented from each modality was two items. The relatively rare (25%) case of being presented with four visual items might have left participants not fully prepared for it at encoding, having split attention to the two modalities.

Neuroimaging results

Figure 2 shows that when the same conditions were contrasted, similar brain regions were activated by a working memory load. However, Figure 3 shows that only the left IPS was associated with a BOLD signal increase whenever the memory load increased, regardless of the stimulus modality details. Given the importance of this ROI, we provide more detail regarding the temporal course of the response in that brain area in both experiments.

The BOLD signal change in the left IPS across time within a trial is illustrated in the top and bottom of Figure 4 for each condition in Experiments 1 and 2, respectively. As can be seen in both graphs, activity in the left IPS increased when information was encoded into working memory and continued as information was maintained in working memory. This left IPS region has been closely associated with capacity-limited working memory maintenance in prior research (Majerus et al., 2006, 2010; Xu & Chun, 2006; Todd & Marois, 2004).

Figure 4. 

In a key brain region, percent BOLD signal change from a pretrial baseline for each condition over time within the trial. This BOLD activity comes from the left IPS as defined by the area of overlap between conjunction analyses in the two experiments (red area in Figure 3). (Top) Experiment 1. (Bottom) Experiment 2. In each experiment, the first broken line box depicts the period used as an index of encoding, the shaded box depicts maintenance, and the second broken line box depicts responding.

Figure 4. 

In a key brain region, percent BOLD signal change from a pretrial baseline for each condition over time within the trial. This BOLD activity comes from the left IPS as defined by the area of overlap between conjunction analyses in the two experiments (red area in Figure 3). (Top) Experiment 1. (Bottom) Experiment 2. In each experiment, the first broken line box depicts the period used as an index of encoding, the shaded box depicts maintenance, and the second broken line box depicts responding.

One can see from Todd and Marois (2004) that set-size-dependent BOLD activity began during stimulus encoding in a working memory condition much more than in an immediate judgment condition. Memory-load-dependent activity during the encoding period is not surprising, inasmuch as the entry of information into the working memory system at a rapid rate is part of the encoding process (Vogel, Woodman, & Luck, 2006).

Correlations between neuroimaging and behavior

Theoretically, memory-load-related neural activity in the maintenance period should predict a load-related increase in the number of items stored in working memory. To examine this issue further, we first averaged the mean BOLD signal change across all available load contrasts (i.e., four-item minus two-item contrasts, regardless of modality) separately for each participant in each experiment. Similarly, the behavioral result was calculated as the increase in number of items stored in working memory across all available contrasts (i.e., increase in k for trials with the requirement of maintaining four as compared with two items, regardless of modality). Thus, each participant yielded mean percent signal change value for each time point and a single behavioral number to be used as a correlate.

Figure 5 shows that, in both experiments, the correlations were highest during the maintenance period and were significant (filled circles) only during that period. (In Experiment 1, it happened that the significant peak was 2 sec before the period preselected to represent maintenance in the conjunction analyses, but this still indicates a special importance of the maintenance period for working memory responses.)

Figure 5. 

Correlations between BOLD activity over time in the left IPS and behavioral responding at the end of the trial. (Top) Experiment 1. (Bottom) Experiment 2. The left IPS was defined by the area of overlap between conjunction analyses in the two experiments (red area in Figure 3). The BOLD activity used was the mean difference between low-load and high-load contrasts, averaged across all such available contrasts (in Experiment 1, 4vis vs. 2vis and 2vis2aud vs. 2vis; in Experiment 2, these same contrasts plus 4vis vs. 2aud and 2vis2aud vs. 2aud). The behavioral responding reflects the number of additional items held in working memory or increase in k in the four-item conditions compared with the two-item conditions, averaged across all such available contrasts, as described for the BOLD activity. Solid points indicate significant correlations, p < .05, with df = 15 in Experiment 1 and df = 14 in Experiment 2.

Figure 5. 

Correlations between BOLD activity over time in the left IPS and behavioral responding at the end of the trial. (Top) Experiment 1. (Bottom) Experiment 2. The left IPS was defined by the area of overlap between conjunction analyses in the two experiments (red area in Figure 3). The BOLD activity used was the mean difference between low-load and high-load contrasts, averaged across all such available contrasts (in Experiment 1, 4vis vs. 2vis and 2vis2aud vs. 2vis; in Experiment 2, these same contrasts plus 4vis vs. 2aud and 2vis2aud vs. 2aud). The behavioral responding reflects the number of additional items held in working memory or increase in k in the four-item conditions compared with the two-item conditions, averaged across all such available contrasts, as described for the BOLD activity. Solid points indicate significant correlations, p < .05, with df = 15 in Experiment 1 and df = 14 in Experiment 2.

Given the small n per experiment, we also collapsed across the two experiments for a more powerful observation of the correlations during key periods of the trial. As shown in Figure 4, the encoding period corresponded to Time Points 5 and 6 in Experiment 1 and Time Points 4 and 5 in Experiment 2 (because the stimulus presentation did not last as long in Experiment 2). Across experiments, the correlation between the average encoding period load effect on the BOLD response and the load-related increase in k was not significant, r(29) = .20, p = .14, one-tailed.

As shown in Figure 4, the predefined maintenance period corresponded to Time Points 10 and 11 in Experiment 1 and Time Points 7 and 8 in Experiment 2. Importantly, the correlation between the load effect on the BOLD response during these periods and on k did reach significance, r(29) = .43, p < .01, one-tailed.

Finally, the response period was defined as Time Points 13–14 in Experiment 1 and Time Points 10 and 11 in Experiment 2 (see Figure 4). The correlation between the average response period load effect on the BOLD response and the load-related increase in k was not significant, r(29) = .20, p = .14, one-tailed.

In summary, load-related activity in the maintenance period, but not in the encoding or response periods, was significantly correlated with a load-related increase in items stored in working memory. This point is further supported by regressions to predict the behavioral measure, the number of items stored in working memory. With the encoding and maintenance load-related BOLD activity as predictors, the maintenance activity made a unique contribution, t(28) = 2.30, p < .05, and its partial correlation with working memory was rp = .40, which was not the case for encoding, p = .68, rp = .08. Similarly, in a regression with maintenance and response period load-related activity as predictors, the maintenance activity again made a unique contribution, t(28) = 2.30, p < .05, rp = .40, which was not the case for the response period, p = .75, rp = −.06. With only encoding and response period load-related activity as predictors, their effects did not approach significance singly or jointly.

Supporting the distinction between the role of parietal activity during different parts of the trial, other researches have distinguished between maintenance and response periods in spatial working memory using repetitive TMS (rTMS) and electroencephalography. Performance was impaired when rTMS was delivered during maintenance to parietal areas, particularly in the left hemisphere (Hamidi, Tononi, & Postle, 2008). The effect of rTMS appears related to its influence on α-band activity and its synchrony with γ-band activity in a network that has the IPS as its hub (Hamidi, Slagter, Tononi, & Postle, 2009). In contrast to these findings for the maintenance period, performance was affected when rTMS was delivered during the response period not to parietal areas but to the dorsolateral pFC (Hamidi, Tononi, & Postle, 2009).

Still, it is noteworthy that neural activity stemming from encoding- and response-related processes theoretically could contribute to the BOLD response observed during the maintenance period in the left IPS and, thus, could contribute to the relation between activity in this area and successful behavioral responses. After all, encoding must include successful loading of information into working memory, and responding must include successful preparation for deployment of the maintained information.

Our results across experiments provide strong evidence of the involvement of at least one brain area, the left IPS, in a general working memory mechanism that accommodates both visual colors and spoken letters. We do not intend to make the claim that other areas, such as the right IPS, do not also contribute to this general, amodal or multimodal working memory system. Yet, it is worth noting a suggestion from one recent study that the left, as opposed to the right, IPS is primarily responsible for the maintenance of item information (Majerus et al., 2010), whereas the right IPS specializes in order information that we did not test in our procedure.

GENERAL DISCUSSION

The present finding is remarkable in that a single brain region was found to be involved in working memory maintenance regardless of the modality of the stimuli to be maintained across two experiments with different methodologies. Furthermore, the area that emerged completely on an empirical basis was the left IPS, one of just a few areas pointed out as a possible basis of visual working memory in previous studies (Xu & Chun, 2006; Todd & Marois, 2004). In short, the left IPS may be a special basis of working memory storage for items in both modalities (Majerus et al., 2010).

Mechanism of Information Maintenance in the Left IPS

The left IPS could either store abstract information derived from modality-specific information represented elsewhere or, as suggested by Ruchkin et al. (2003), it could store pointers to that modality-specific information. In any case, the role of the parietal areas in working memory maintenance appears causal in that TMS to parietal areas disrupts working memory storage whereas TMS to the dorsolateral pFC appears to disrupt only tasks that require manipulation of the information in working memory (Postle et al., 2006).

By the pointer hypothesis, the capacity limit is in how many pointers can be held at once. When multiple elements form an integrated unit or chunk (Miller, 1956), as in the combination of phonemes that make up a known word, the brain state presumably would include a single pointer linked to all of the elements making up that chunk. This conceptualization is consistent with the notion of the IPS as key area for the focus of attention (Chein & Fiez, 2010; Majerus et al., 2006, 2010; Cowan, 1995, 2001). Under this interpretation as well, it could be said that each pointer is, in a sense, abstract.

One recent study using magneto-encephalographic methods to examine the level of neural synchrony between brain areas (Palva et al., 2010) indicates that the IPS, more than the rest of the frontal–parietal network, is the hub of a system accounting for individual differences in working memory capacity. This is in accord with the present finding (Figure 5) that individuals who showed larger working memory storage increases as a function of load also showed more left IPS activation increases during the maintenance period as a function of load.

It is further possible that the pointers used for working memory are also used in perceptual tasks as part of a more general focus-of-attention mechanism (Cowan, 1995; cf. Magen, Emmanouil, McMains, Kastner, & Treisman, 2009). An ERP indicator of working memory maintenance, contralateral delay activity, also appears to be present when attention is distributed to items in a perceptual field (Drew & Vogel, 2008) and magneto-encephalographic synchrony data suggest that it may reflect disengagement of attention on the side of the brain ipsilateral to the memory load (van Dijk, van der Werf, Mazaheri, Medendorp, & Jensen, 2010; Mazaheri & Jensen, 2008). It is notable that the notion of a brain mechanism limited to just few working memory pointers is similar to what has been proposed not only for working memory (e.g., Ruchkin et al., 2003) but also for moving objects in the perceptual field (Pylyshyn & Storm, 1988; see also Cowan, 2001).

How Does the Left IPS Differ from the Right IPS?

The past literature suggests the possibility that the right IPS also would be especially involved in abstract or multimodal working memory storage (Majerus et al., 2006, 2010; Xu & Chun, 2006; Todd & Marois, 2004), although there are differences between processing in the left versus right IPS (Majerus et al., 2010; Jonides et al., 2008; Yantis et al., 2002), as in adjacent parietal regions (Ravizza, Delgado, Chein, Becker, & Fiez, 2004). In the present study, we found that the right IPS showed increased activity in the 4vis and 2vis2aud conditions relative to the 2vis condition (Table 1). A similar activity increase, however, was not observed in Experiment 2 when comparisons with a 2aud baseline condition also were included. Although the theoretical importance of this difference between the left and right IPS cannot be determined from the present study, it is worth noting that Majerus et al. (2010) proposed on the basis of their data that the left IPS preserves item information as in the present study whereas the right IPS preserves order information.

Alternative Interpretations

One theoretically possible alternative to the present interpretation is that the left IPS represents only one type of information, either visual or auditory–verbal. According to this interpretation, either the visual items were recoded (i.e., mentally translated) into color names or, perhaps less plausibly, the verbal items were visually recoded. Verbal recoding of easily labeled visual information is a common strategy, but articulatory suppression, which was included in our second experiment, is a fairly effective way to counteract it (Baddeley, 1986). Morey and Cowan (2004, 2005) found that working memory for arrays of colors was unaffected by articulatory suppression (repetition of a known seven-digit number), whereas it was greatly affected by an auditory memory load (repetition of a new random seven-digit number). This suggests that verbal recoding is not typically used for color arrays. Also, Xu and Chun (2006) found left (and right) IPS activity for visual objects with no ready-made labels, again suggesting that the left IPS can use visually encoded information. The similarity of neuroimaging results without suppression (Experiment 1) and with suppression (Experiment 2), as shown in Figure 2, again suggests that verbal recoding of the visual information did not play an important role. Of course, we cannot be certain that it played absolutely no role, but we have made a strong effort to prevent it.

We did not take steps specifically to prevent the converse, that is, recoding of verbal information into a visual form, inasmuch as there is no easy rehearsal mechanism that would encourage that recoding. Whereas verbal information can be rehearsed rather effortlessly provided that the items are phonologically distinct (Camos, Lagner, & Barrouillet, 2009), the refreshment of visual information requires attention (Raye, Johnson, Mitchell, Greene, & Johnson, 2007). It seems especially implausible that individuals would recode verbal stimuli into a visual form when they also have to remember other visual information. The finding of left IPS activity for a variety of verbal materials (e.g., Chein & Fiez, 2010; Majerus et al., 2006, 2010) also suggests that this brain area does not work exclusively with visual codes. Future research could further strengthen the case for the presence of dual codes by establishing that there is functional connectivity between IPS activity and both auditory–verbal and visual–spatial activity elsewhere in the posterior cortex during working memory tasks.

Concluding Remarks

In other fMRI researches, multisensory areas in perception have been identified (Beauchamp, 2005). The present findings are complementary, revealing multisensory brain areas in working memory. Of these areas, the left IPS was the only area clearly active regardless of the modalities of the low-load condition (2vis or 2aud) and of the higher-load condition (4vis or 2vis2aud). This brain area would have been predicted on the basis of prior research (Majerus et al., 2010; Xu & Chun, 2006; Todd & Marois, 2004). The results build on prior evidence for an amodal working memory system for verbal information (Schumacher et al., 1996).

It will take additional research to determine whether the left IPS is unique for amodal or multimodal storage and whether it holds item information or pointers to that item information held elsewhere (Ruchkin et al., 2003). Recent research at least suggests that the IPS displays a similar item capacity limit no matter whether the items are to be perceptually processed or held in working memory (Silk, Bellgrove, Wrafter, Mattingley, & Cunnington, 2010; Mitchell & Cusack, 2008), in keeping with the notion that it may reflect items in the focus of attention. It also is a matter of current heated debate whether the working memory capacity limit comes from the distribution of attention to at most a few items (Fukuda, Awh, & Vogel, 2010; Cowan & Rouder, 2009; Rouder et al., 2008; Zhang & Luck, 2008; Xu & Chun, 2006) or the distribution of a limited resource pool of attention to an unlimited number of items, thinly if necessary (Bays & Husain, 2008, 2009; Wilken & Ma, 2004). Under the resource pool theory, however, it would be unclear why IPS BOLD activity levels off at about four items (e.g., Mitchell & Cusack, 2008; Todd & Marois, 2004).

The perceptual and working memory areas of the brain for amodal or multimodal cognition both are likely to contribute to the neural basis for the formation of new conceptual knowledge, which should be independent of the input modality. (There is, e.g., a concept of a green square that can be elicited pictorially with spoken words or with printed words.) Highlighting the role of abstraction in memory and learning, the present results appear to require a departure from the traditional view of working memory, which included only phonological and visual storage mechanisms. There is some kind of maintenance within the working memory system that is more general or abstract (Baddeley, 2001; Cowan, 2001), and here, we have begun to identify its neural basis.

Acknowledgments

This research was supported by the Brain Imaging Center and NIH grant R01-HD-21338 to Cowan.

Reprint requests should be sent to Nelson Cowan, Department of Psychological Sciences, University of Missouri, 18 McAlester Hall, Columbia, MO 65211, or via e-mail: CowanN@missouri.edu.

REFERENCES

Baddeley
,
A.
(
1986
).
Working memory.
Oxford, UK
:
Clarendon Press
.
Baddeley
,
A.
(
2001
).
The magic number and the episodic buffer.
Behavioral and Brain Sciences
,
24
,
117
118
.
Bandettini
,
P.
,
Jesmanowicz
,
A.
,
Wong
,
E.
, &
Hyde
,
J.
(
1993
).
Processing strategies for time-course data sets in functional MRI of the human brain.
Magnetic Resonance in Medicine
,
30
,
161
173
.
Bays
,
P. M.
, &
Husain
,
M.
(
2008
).
Dynamic shifts of limited working memory resources in human vision.
Science
,
321
,
851
854
.
Bays
,
P. M.
, &
Husain
,
M.
(
2009
).
Response to comment on “Dynamic shifts of limited working memory resources in human vision.”
Science
,
323
,
877d
.
Beauchamp
,
M.
(
2005
).
See me, hear me, touch me: Multisensory integration in lateral occipital-temporal cortex.
Current Opinion in Neurobiology
,
15
,
145
153
.
Broadbent
,
D. E.
(
1958
).
Perception and communication.
New York
:
Pergamon Press
.
Camos
,
V.
,
Lagner
,
P.
, &
Barrouillet
,
P.
(
2009
).
Two maintenance mechanisms of verbal information in working memory.
Journal of Memory and Language
,
61
,
457
469
.
Chein
,
J.
, &
Fiez
,
J.
(
2010
).
Evaluating models of working memory through the effects of concurrent irrelevant information.
Journal of Experimental Psychology: General
,
139
,
117
137
.
Christ
,
S.
,
Van Essen
,
D.
,
Watson
,
J.
,
Brubaker
,
L.
, &
McDermott
,
K.
(
2009
).
The contributions of prefrontal cortex and executive control to deception: Evidence from activation likelihood estimate meta-analyses.
Cerebral Cortex
,
19
,
1557
1566
.
Cohen
,
J. D.
,
Perlstein
,
W. M.
,
Braver
,
T. S.
,
Nystrom
,
L. E.
,
Noll
,
D. C.
,
Jonides
,
J.
,
et al
(
1997
).
Temporal dynamics of brain activation during a working memory task.
Nature
,
386
,
604
608
.
Cottingham
,
J.
,
Stoothoff
,
R.
,
Murdoch
,
D.
, &
Kenny
,
A.
(Eds.) (
1991
).
The philosophical writings of Descartes (Volume 3: The correspondence).
Cambridge University Press
.
Cowan
,
N.
(
1995
).
Attention and memory: An integrated framework.
New York
:
Oxford University Press
.
Cowan
,
N.
(
2001
).
The magical number 4 in short-term memory: A reconsideration of mental storage capacity.
The Behavioral and Brain Sciences
,
24
,
87
185
.
Cowan
,
N.
, &
Rouder
,
J. N.
(
2009
).
Comment on “Dynamic shifts of limited working memory resources in human vision.”
Science
,
323
,
877c
.
Darwin
,
C.
,
Turvey
,
M.
, &
Crowder
,
R.
(
1972
).
An auditory analogue of the sperling partial report procedure: Evidence for brief auditory storage.
Cognitive Psychology
,
3
,
255
267
.
Drew
,
T.
, &
Vogel
,
E. K.
(
2008
).
Neural measures of individual differences in selecting and tracking multiple moving objects.
Journal of Neuroscience
,
28
,
4183
4191
.
Forman
,
S. D.
,
Cohen
,
J. D.
,
Fitzgerald
,
M.
,
Eddy
,
W. F.
,
Mintun
,
M. A.
, &
Noll
,
D. C.
(
1995
).
Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): Use of a cluster-size threshold.
Magnetic Resonance in Medicine
,
33
,
636
647
.
Fukuda
,
K.
,
Awh
,
E.
, &
Vogel
,
E. K.
(
2010
).
Discrete capacity limits in visual working memory.
Current Opinion in Neurobiology
,
20
,
177
182
.
Goebel
,
R.
,
Esposito
,
F.
, &
Formisano
,
E.
(
2006
).
Analysis of functional image analysis contest (FIAC) data with brainvoyager QX: From single-subject to cortically aligned group general linear model analysis and self-organizing group independent component analysis.
Human Brain Mapping
,
27
,
392
401
.
Hamidi
,
M.
,
Slagter
,
H. A.
,
Tononi
,
G.
, &
Postle
,
B. R.
(
2009
).
Repetitive transcranial magnetic stimulation affects behavior by biasing endogenous cortical oscillations.
Frontiers in Integrative Neuroscience
,
3
,
1
12. doi:10.3389/neuro.07.014.2009
.
Hamidi
,
M.
,
Tononi
,
G.
, &
Postle
,
B. R.
(
2008
).
Evaluating frontal and parietal contributions to spatial working memory with repetitive transcranial magnetic stimulation.
Brain Research
,
1230
,
202
210
.
Hamidi
,
M.
,
Tononi
,
G.
, &
Postle
,
B. R.
(
2009
).
Evaluating the role of prefrontal and parietal cortices in memory-guided response with repetitive transcranial magnetic stimulation.
Neuropsychologia
,
47
,
295
302
.
James
,
W.
(
1890
).
The principles of psychology.
New York
:
Henry Holt
.
Johnston
,
W. A.
, &
Heinz
,
S. P.
(
1978
).
Flexibility and capacity demands of attention.
Journal of Experimental Psychology: General
,
107
,
420
435
.
Jonides
,
J.
,
Lewis
,
R.
,
Nee
,
D.
,
Lustig
,
C.
,
Berman
,
M.
, &
Moore
,
K.
(
2008
).
The mind and brain of short-term memory.
Annual Review of Psychology
,
59
,
193
224
.
Klemen
,
J.
,
Büchel
,
C.
,
Bühler
,
M.
,
Menz
,
M.
, &
Rose
,
M.
(
2010
).
Auditory working memory load impairs visual ventral stream processing: Toward a unified model of attentional load.
Journal of Cognitive Neuroscience
,
22
,
437
446
.
Luck
,
S.
, &
Vogel
,
E.
(
1997
).
The capacity of visual working memory for features and conjunctions.
Nature
,
390
,
279
284
.
Magen
,
H.
,
Emmanouil
,
T.
,
McMains
,
S.
,
Kastner
,
S.
, &
Treisman
,
A.
(
2009
).
Attentional demands predict short-term memory load response in posterior parietal cortex.
Neuropsychologia
,
47
,
1790
1798
.
Majerus
,
S.
,
D'Argembeau
,
A.
,
Martinez Perez
,
T.
,
Belayachi
,
S.
,
Van der Linden
,
M.
,
Collette
,
F.
,
et al
(
2010
).
The commonality of neural networks for verbal and visual short-term memory.
Journal of Cognitive Neuroscience
,
22
,
2570
2593
.
Majerus
,
S.
,
Poncelet
,
M.
,
Van der Linden
,
M.
,
Albouy
,
G.
,
Salmon
,
E.
,
Sterpenich
,
V.
,
et al
(
2006
).
The left intraparietal sulcus and verbal short-term memory: Focus of attention or serial order?
Neuroimage
,
32
,
880
891
.
Mazaheri
,
A.
, &
Jensen
,
O.
(
2008
).
Asymmetric amplitude modulations of brain oscillations generate slow evoked responses.
Journal of Neuroscience
,
28
,
7781
7787
.
Miller
,
G. A.
(
1956
).
The magical number seven, plus or minus two: Some limits on our capacity for processing information.
Psychological Review
,
63
,
81
97
.
Mitchell
,
D. J.
, &
Cusack
,
R.
(
2008
).
Flexible, capacity-limited activity of posterior parietal cortex in perceptual as well as visual short-term memory tasks.
Cerebral Cortex
,
18
,
1788
1798
.
Morey
,
C. C.
, &
Cowan
,
N.
(
2004
).
When visual and verbal memories compete: Evidence of cross-domain limits in working memory.
Psychonomic Bulletin & Review
,
11
,
296
301
.
Morey
,
C. C.
, &
Cowan
,
N.
(
2005
).
When do visual and verbal memories conflict? The importance of working-memory load and retrieval.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
31
,
703
713
.
Nichols
,
T.
,
Brett
,
M.
,
Andersson
,
J.
,
Wager
,
T.
, &
Poline
,
J.-B.
(
2005
).
Valid conjunction inference with the minimum statistic.
Neuroimage
,
25
,
653
660
.
Palva
,
J. M.
,
Monto
,
S.
,
Kulashekhar
,
S.
, &
Palva
,
S.
(
2010
).
Neuronal synchrony reveals working memory networks and predicts individual memory capacity.
Proceedings of the National Academy of Sciences, U.S.A.
,
107
,
7580
7585
.
Posner
,
M.
, &
Petersen
,
S.
(
1990
).
The attention system of the human brain.
Annual Review of Neuroscience
,
13
,
25
42
.
Postle
,
B.
,
Ferrarelli
,
F.
,
Hamidi
,
M.
,
Feredoes
,
E.
,
Massimini
,
M.
,
Peterson
,
M.
,
et al
(
2006
).
Repetitive transcranial magnetic stimulation dissociates working memory manipulation from retention functions in the prefrontal, but not posterior parietal, cortex.
Journal of Cognitive Neuroscience
,
18
,
1712
1722
.
Pylyshyn
,
Z. W.
, &
Storm
,
R. W.
(
1988
).
Tracking multiple independent targets: Evidence for a parallel tracking mechanism.
Spatial Vision
,
3
,
179
197
.
Ravizza
,
S.
,
Delgado
,
M.
,
Chein
,
J.
,
Becker
,
J.
, &
Fiez
,
J.
(
2004
).
Functional dissociations within the inferior parietal cortex in verbal working memory.
Neuroimage
,
22
,
562
573
.
Raye
,
C. L.
,
Johnson
,
M. K.
,
Mitchell
,
K. J.
,
Greene
,
E. J.
, &
Johnson
,
M. R.
(
2007
).
Refreshing: A minimal executive function.
Cortex
,
43
,
135
145
.
Rissman
,
J.
,
Gazzaley
,
A.
, &
D'Esposito
,
M.
(
2009
).
The effect of non-visual working memory load on top–down modulation of visual processing.
Neuropsychologia
,
47
,
1637
1646
.
Rouder
,
J. N.
,
Morey
,
R. D.
,
Cowan
,
N.
,
Zwilling
,
C. E.
,
Morey
,
C. C.
, &
Pratte
,
M. S.
(
2008
).
An assessment of fixed-capacity models of visual working memory.
Proceedings of the National Academy of Sciences, U.S.A.
,
105
,
5975
5979
.
Ruchkin
,
D. S.
,
Grafman
,
J.
,
Cameron
,
K.
, &
Berndt
,
R. S.
(
2003
).
Working memory retention systems: A state of activated long-term memory.
Behavioral and Brain Sciences
,
26
,
709
777
.
Saults
,
J.
, &
Cowan
,
N.
(
2007
).
A central capacity limit to the simultaneous storage of visual and auditory arrays in working memory.
Journal of Experimental Psychology: General
,
136
,
663
684
.
Schumacher
,
E. H.
,
Lauber
,
E.
,
Awh
,
E.
,
Jonides
,
J.
,
Smith
,
E. E.
, &
Koeppe
,
R. A.
(
1996
).
Evidence for an amodal verbal working memory system.
Neuroimage
,
3
,
79
88
.
Silk
,
T. J.
,
Bellgrove
,
M. A.
,
Wrafter
,
P.
,
Mattingley
,
J. B.
, &
Cunnington
,
R.
(
2010
).
Spatial working memory and spatial attention rely on common neural processes in the intraparietal sulcus.
Neuroimage
,
53
,
718
724
.
Sperling
,
G.
(
1960
).
The information available in brief visual presentations.
Psychological Monographs: General and Applied
,
74
,
1
29
.
Talairach
,
J.
, &
Tournoux
,
P.
(
1988
).
Co-planar stereotaxic atlas of the human brain.
New York
:
Thieme
.
Todd
,
J.
, &
Marois
,
R.
(
2004
).
Capacity limit of visual short-term memory in human posterior parietal cortex.
Nature
,
428
,
751
754
.
van Dijk
,
H.
,
van der Werf
,
J.
,
Mazaheri
,
A.
,
Medendorp
,
W. P.
, &
Jensen
,
O.
(
2010
).
Modulations of oscillatory activity with amplitude asymmetry can produce cognitively relevant event-related responses.
Proceedings of the National Academy of Sciences, U.S.A.
,
107
,
900
905
.
Vogel
,
E. K.
,
Woodman
,
G. F.
, &
Luck
,
S. J.
(
2006
).
The time course of consolidation in visual working memory.
Journal of Experimental Psychology: Human Perception and Performance
,
32
,
1436
1451
.
Wilken
,
P.
, &
Ma
,
W. J.
(
2004
).
A detection theory account of change detection.
Journal of Vision
,
4
,
1120
1135
.
Xu
,
Y.
, &
Chun
,
M.
(
2006
).
Dissociable neural mechanisms supporting visual short-term memory for objects.
Nature
,
440
,
91
95
.
Yantis
,
S.
,
Schwarzbach
,
J.
,
Serences
,
J.
,
Carlson
,
R.
,
Steinmetz
,
M.
,
Pekar
,
J.
,
et al
(
2002
).
Transient neural activity in human parietal cortex during spatial attention shifts.
Nature Neuroscience
,
5
,
995
1002
.
Zhang
,
W.
, &
Luck
,
S. J.
(
2008
).
Discrete fixed-resolution representations in visual working memory.
Nature
,
453
,
233
235
.