Audiovisual speech perception provides an opportunity to investigate the mechanisms underlying multimodal processing. By using nonspeech stimuli, it is possible to investigate the degree to which audiovisual processing is specific to the speech domain. It has been shown in a match-to-sample design that matching across modalities is more difficult in the nonspeech domain as compared to the speech domain. We constructed a biophysically realistic neural network model simulating this experimental evidence. We propose that a stronger connection between modalities in speech underlies the behavioral difference between the speech and the nonspeech domain. This could be the result of more extensive experience with speech stimuli. Because the match-to-sample paradigm does not allow us to draw conclusions concerning the integration of auditory and visual information, we also simulated two further conditions based on the same paradigm, which tested the integration of auditory and visual information within a single stimulus. New experimental data for these two conditions support the simulation results and suggest that audiovisual integration of discordant stimuli is stronger in speech than in nonspeech stimuli. According to the simulations, the connection strength between auditory and visual information, on the one hand, determines how well auditory information can be assigned to visual information, and on the other hand, it influences the magnitude of multimodal integration.

When listening to modified speech, either naturally or artificially altered, the human perceptual system rapidly adapts to it. There is some debate about the nature of the mechanisms underlying this adaptation. Although some authors propose that listeners modify their prelexical representations, others assume changes at the lexical level. Recently, Larsson, Vera, Sebastian-Galles, and Deco [Lexical plasticity in early bilinguals does not alter phoneme categories: I. Neurodynamical modelling. Journal of Cognitive Neuroscience, 20, 76–94, 2008] proposed a biologically plausible computational model to account for some existing data, one which successfully modeled how long-term exposure to a dialect triggers the creation of new lexical entries. One specific prediction of the model was that prelexical (phoneme) representations should not be affected by dialectal exposure (as long as the listener is exposed to both standard and dialectal pronunciations). Here we present a series of experiments testing the predictions of the model. Native listeners of Catalan, with extended exposure to Spanish-accented Catalan, were tested on different auditory lexical decision tasks and phoneme discrimination tasks. Behavioral and electrophysiological recordings were obtained. The results supported the predictions of our model. On the one hand, both error rates and N400 measurements indicated the existence of alternative lexical entries for dialectal varieties. On the other hand, no evidence of alterations at the phoneme level, either in the behavioral discrimination task or in the electrophysiological measurement (MMN), could be detected. The results of the present study are compared with those obtained in short-term laboratory exposures in an attempt to provide an integrative account.

The prefrontal cortex is believed to be important for cognitive control, working memory, and learning. It is known to play an important role in the learning and execution of conditional visuomotor associations, a cognitive task in which stimuli have to be associated with actions by trial-and-error learning. In our modeling study, we sought to integrate several hypotheses on the function of the prefrontal cortex using a computational model, and compare the results to experimental data. We constructed a module of prefrontal cortex neurons exposed to two different inputs, which we envision to originate from the inferotemporal cortex and the basal ganglia. We found that working memory properties do not describe the dominant dynamics in the prefrontal cortex, but the activation seems to be transient, probably progressing along a pathway from sensory to motor areas. During the presentation of the cue, the dynamics of the prefrontal cortex is bistable, yielding a distinct activation for correct and error trails. We find that a linear change in network parameters relates to the changes in neural activity in consecutive correct trials during learning, which is important evidence for the underlying learning mechanisms.

Sebastián-Gallés et al. [The influence of initial exposure on lexical representation: Comparing early and simultaneous bilinguals. Journal of Memory and Language, 52 , 240–255, 2005] contrasted highly proficient early Spanish-Catalan and Catalan-Spanish bilinguals, using Catalan materials in a lexical decision task (LDT). They constructed two types of experimental pseudowords, substituting Catalan phoneme /e/ for Catalan /ɛ/, or vice versa. Catalan-dominant bilinguals showed a performance asymmetry across experimental conditions, making more mistakes for /ɛ/→/e/ changes, than for /e/→/ɛ/ ones. This was considered evidence of a developed acceptance of mispronounced Catalan /ɛ/-words, caused by exposure to a bilingual environment where mispronunciations by Spanish-dominant bilinguals using their /e/-category abound. Although this indicated modified or added lexical representations, an open issue is whether such lexical information also modifies phoneme categories. We address this using a biophysically realistic neurodynamic model, describing neural activity at the synaptic and spiking levels. We construct a network of pools of neurons, representing phonemic and lexical processing. Carefully analyzing the dependency of network dynamics on connection strengths, by first exploring parameter space under steady-state assumptions (mean-field scans), then running spiking simulations, we investigate the neural substrate role in a representative LDT. We also simulate a phoneme discrimination task to address whether lexical changes affect the phonemic level. We find that the same network configuration which displays asymmetry in the LDT shows equal performance discriminating the two modeled phonemes. Thus, we predicted that the Catalan-dominant bilinguals do not alter their phoneme categories, although showing signs of having stored a new word variation in the lexicon. To explore this prediction, a syllable discrimination task involving the /e/-/ɛ/ contrast was set up, using Catalan-dominants displaying performance asymmetry in a repetition of the original LDT. Discrimination task results support the prediction, showing that these subjects discriminate both categories equally well. We conclude that subjects often exposed to dialectal word variations can store these in their lexicons, without altering their phoneme representations.

A key issue in the neurophysiology of cognition is the problem of sequential learning. Sequential learning refers to the ability to encode and represent the temporal order of discrete elements occurring in a sequence. We show that the short-term memory for a sequence of items can be implemented in an autoassociation neural network. Each item is one of the attractor states of the network. The autoassociation network is implemented at the level of integrate-and-fire neurons so that the contributions of different biophysical mechanisms to sequence learning can be investigated. It is shown that if it is a property of the synapses or neurons that support each attractor state that they adapt, then everytime the network is made quiescent (e.g., by inhibition), then the attractor state that emerges next is the next item in the sequence. We show with numerical simulations implementations of the mechanisms using (1) a sodium inactivation-based spike-frequency-adaptation mechanism, (2) a Ca 2+ -activated K + current, and (3) short-term synaptic depression, with sequences of up to three items. The network does not need repeated training on a particular sequence and will repeat the items in the order that they were last presented. The time between the items in a sequence is not fixed, allowing the items to be read out as required over a period of up to many seconds. The network thus uses adaptation rather than associative synaptic modification to recall the order of the items in a recently presented sequence.

Single-neuron recordings, functional magnetic resonance imaging (fMRI) data, and the effects of lesions indicate that the prefrontal cortex (PFC) is involved in some types of working memory and related cognitive processes. Based on these data, two different models of the topographical and functional organization of the PFC have been proposed: organization-by-stimulus-domain, and organization-by-process. In this article, we utilize an integrate-and-fire network to model both single-neuron and fMRI data on short-term memory in order to understand data obtained in topologically different parts of the PFC during working memory tasks. We explicitly model the mechanisms that underlie workingmemory-related activity during the execution of delay tasks that have a “what”-then-“where” design (with both object and spatial delayed responses within the same trial). The model contains different populations of neurons (as found experimentally) in attractor networks that respond in the delay period to the stimulus object, the stimulus position, and to combinations of both object and position information. The pools are arranged hierarchically and have global inhibition through inhibitory interneurons to implement competition. It is shown that a biasing attentional input to define the current relevant information (object or location) enables the system to select the correct neuronal populations during the delay period in what is a biased competition model of attention. The processes occurring at the AMPA and NMDA synapses are dynamically modeled in the integrate-and-fire implementation to produce realistic spiking dynamics. It is shown that the fMRI data characteristic of the dorsal PFC and linked to spatial processing and manipulation of items can be reproduced in the model by a high level of inhibition, whereas the fMRI data characteristic of the ventral PFC and linked to object processing can be produced by a lower level of inhibition, even though the network is itself topographically homogeneous with no spatial topology of the neurons. This article, thus, not only presents a model for how spatial versus object short-term memory could be implemented in the PFC, but also shows that the fMRI BOLD signal measured during such tasks from different parts of the PFC could reflect a higher level of inhibition dorsally, without this dorsal region necessarily being primarily spatial and the ventral region object-related.