Memory improves when encoding and retrieval processes overlap. Here, we investigated how the neural bases of long-term memory encoding vary as a function of the degree to which functional processes engaged at study are engaged again at test. In an incidental learning paradigm, electrical brain activity was recorded from the scalps of healthy adults while they made size judgments on intermixed series of pictures and words. After a 1-hr delay, memory for the items was tested with a recognition task incorporating remember/know judgments. In different groups of participants, studied items were either probed in the same mode of presentation (word–word; picture–picture) or in the alternative mode of presentation (word–picture; picture–word). Activity over anterior scalp sites predicted later memory of words, irrespective of type of test probe. Encoding-related activity for pictures, by contrast, differed qualitatively depending on how an item was cued at test. When a picture was probed with a picture, activity over anterior scalp sites predicted encoding success. When a picture was probed with a word, encoding-related activity was instead maximal over posterior sites. Activity differed according to study–test congruency from around 100 msec after picture onset. These findings indicate that electrophysiological correlates of encoding are sensitive to the similarity between processes engaged at study and test. The time course supports a direct and not merely consequential role of encoding–retrieval overlap in encoding. However, because congruency only affected one type of stimulus material, encoding–retrieval overlap may not be a universal organizing principle of neural correlates of memory.

It is unclear how neural correlates of episodic memory retrieval differ depending on the type of material that is retrieved. Here, we used a source memory task to compare electrical brain activity for the recollection of three types of stimulus material. At study, healthy adults judged how well visually presented objects, words, and faces fitted with paired auditorily presented names of locations. At test, only visual stimuli were presented. The task was to decide whether an item had been presented earlier and, if so, what location had been paired with the item. Stimulus types were intermixed across trials in Experiment 1 and presented in separate study–test lists in Experiment 2. A graded pattern of memory performance was observed across objects, words, and faces in both experiments. Between 300 and 500 msec, event-related potentials for recollected objects and faces showed a more frontal scalp distribution compared to words in both experiments. Later in the recording epoch, all three stimulus materials elicited recollection effects over left posterior scalp sites. However, these effects extended more anteriorly for objects and faces when stimulus categories were blocked. These findings demonstrate that the neural correlates of recollection are material specific, the crucial difference being between pictorial and verbal material. Faces do not appear to have a special status. The sensitivity of recollection effects to the kind of experimental design suggests that, in addition to type of stimulus material, higher-level control processes affect the cognitive and neural mechanisms underlying episodic retrieval.

It has been assumed that the effective encoding of information into memory primarily depends on neural activity elicited when an event is initially encountered. Recently, it has been shown that memory formation also relies on neural activity just before an event. The precise role of such activity in memory is currently unknown. Here, we address whether prestimulus activity affects the encoding of auditory and visual events, is set up on a trial-by-trial basis, and varies as a function of the type of recognition judgment an item later receives. Electrical brain activity was recorded from the scalps of 24 healthy young adults while they made semantic judgments on randomly intermixed series of visual and auditory words. Each word was preceded by a cue signaling the modality of the upcoming word. Auditory words were preceded by auditory cues and visual words by visual cues. A recognition memory test with remember/know judgments followed after a delay of about 45 min. As observed previously, a negative-going, frontally distributed modulation just before visual word onset predicted later recollection of the word. Crucially, the same effect was found for auditory words and observed on stay as well as switch trials. These findings emphasize the flexibility and general role of prestimulus activity in memory formation, and support a functional interpretation of the activity in terms of semantic preparation. At least with an unpredictable trial sequence, the activity is set up anew on each trial.

In auditory–visual synesthesia, sounds automatically elicit conscious and reliable visual experiences. It is presently unknown whether this reflects early or late processes in the brain. It is also unknown whether adult audiovisual synesthesia resembles auditory-induced visual illusions that can sometimes occur in the general population or whether it resembles the electrophysiological deflection over occipital sites that has been noted in infancy and has been likened to synesthesia. Electrical brain activity was recorded from adult synesthetes and control participants who were played brief tones and required to monitor for an infrequent auditory target. The synesthetes were instructed to attend either to the auditory or to the visual (i.e., synesthetic) dimension of the tone, whereas the controls attended to the auditory dimension alone. There were clear differences between synesthetes and controls that emerged early (100 msec after tone onset). These differences tended to lie in deflections of the auditory-evoked potential (e.g., the auditory N1, P2, and N2) rather than the presence of an additional posterior deflection. The differences occurred irrespective of what the synesthetes attended to (although attention had a late effect). The results suggest that differences between synesthetes and others occur early in time, and that synesthesia is qualitatively different from similar effects found in infants and certain auditory-induced visual illusions in adults. In addition, we report two novel cases of synesthesia in which colors elicit sounds, and vice versa.

Research into the neural underpinnings of memory formation has focused on the encoding of familiar verbal information. Here, we address how the brain supports the encoding of novel information that does not have meaning. Electrical brain activity was recorded from the scalps of healthy young adults while they performed an incidental encoding task (syllable judgments) on separate series of words and “nonwords” (nonsense letter strings that are orthographically legal and pronounceable). Memory for the items was then probed with a recognition memory test. For words as well as nonwords, event-related potentials differed depending on whether an item would subsequently be remembered or forgotten. However, the polarity and timing of the effect varied across item type. For words, subsequently remembered items showed the usually observed positive-going, frontally distributed modulation from around 600 msec after word onset. For nonwords, by contrast, a negative-going, spatially widespread modulation predicted encoding success from 1000 msec onward. Nonwords also showed a modulation shortly after item onset. These findings imply that the brain supports the encoding of familiar and unfamiliar letter strings in qualitatively different ways, including the engagement of distinct neural activity at different points in time. The processing of semantic attributes plays an important role in the encoding of words and the associated positive frontal modulation.