Facial expressions provide information about an individual's intentions and emotions and are thus an important medium for nonverbal communication. Theories of embodied cognition assume that facial mimicry and resulting facial feedback plays an important role in the perception of facial emotional expressions. Although behavioral and electrophysiological studies have confirmed the influence of facial feedback on the perception of facial emotional expressions, the influence of facial feedback on the automatic processing of such stimuli is largely unexplored. The automatic processing of unattended facial expressions can be investigated by visual expression-related MMN. The expression-related MMN reflects a differential ERP of automatic detection of emotional changes elicited by rarely presented facial expressions (deviants) among frequently presented facial expressions (standards). In this study, we investigated the impact of facial feedback on the automatic processing of facial expressions. For this purpose, participants ( n = 19) performed a centrally presented visual detection task while neutral (standard), happy, and sad faces (deviants) were presented peripherally. During the task, facial feedback was manipulated by different pen holding conditions (holding the pen with teeth, lips, or nondominant hand). Our results indicate that automatic processing of facial expressions is influenced and thus dependent on the own facial feedback.

Behavioral control is influenced not only by learning from the choices made and the rewards obtained but also by “what might have happened,” that is, inference about unchosen options and their fictive outcomes. Substantial progress has been made in understanding the neural signatures of direct learning from choices that are actually made and their associated rewards via reward prediction errors (RPEs). However, electrophysiological correlates of abstract inference in decision-making are less clear. One seminal theory suggests that the so-called feedback-related negativity (FRN), an ERP peaking 200–300 msec after a feedback stimulus at frontocentral sites of the scalp, codes RPEs. Hitherto, the FRN has been predominantly related to a so-called “model-free” RPE: The difference between the observed outcome and what had been expected. Here, by means of computational modeling of choice behavior, we show that individuals employ abstract, “double-update” inference on the task structure by concurrently tracking values of chosen stimuli (associated with observed outcomes) and unchosen stimuli (linked to fictive outcomes). In a parametric analysis, model-free RPEs as well as their modification because of abstract inference were regressed against single-trial FRN amplitudes. We demonstrate that components related to abstract inference uniquely explain variance in the FRN beyond model-free RPEs. These findings advance our understanding of the FRN and its role in behavioral adaptation. This might further the investigation of disturbed abstract inference, as proposed, for example, for psychiatric disorders, and its underlying neural correlates.

When a stimulus is associated with a reward, it becomes prioritized, and the allocation of attention to that stimulus increases. For low-level features, such as color, this reward-based allocation of attention can manifest early in time and as a faster and stronger shift of attention to targets with that color, as reflected by the N2pc (a parieto-occipital electrophysiological component peaking at ∼250 msec). It is unknown, however, if reward associations can similarly modulate attentional shifts to complex objects or object categories, or if reward-related modulation of attentional allocation to such stimuli would occur later in time or through a different mechanism. Here, we used magnetoencephalographic recordings in 24 participants to investigate how object categories with a reward association would modulate the shift of attention. On each trial, two colored squares were presented, one in a target color and the other in a distractor color, each with an embedded object. Participants searched for the target-colored square and performed a corner discrimination task. The embedded objects were from either a rewarded or non-rewarded category, and if a rewarded-category object were present within the target-colored square, participants could earn extra money for correct performance. We observed that when the target color contained an object from a rewarded versus a non-rewarded category, the neural shift of attention to the target was faster and of greater magnitude, although the rewarded objects were not relevant for correct task performance. These results suggest that reward associations of complex objects can rapidly modulate attentional allocation to a target.

The perceived intensity of sensory stimuli is reduced when these stimuli are caused by the observer's actions. This phenomenon is traditionally explained by forward models of sensory action–outcome, which arise from motor processing. Although these forward models critically predict anticipatory modulation of sensory neural processing, neurophysiological evidence for anticipatory modulation is sparse and has not been linked to perceptual data showing sensory attenuation. By combining a psychophysical task involving contrast discrimination with source-level time–frequency analysis of MEG data, we demonstrate that the amplitude of alpha-oscillations in visual cortex is enhanced before the onset of a visual stimulus when the identity and onset of the stimulus are controlled by participants' motor actions. Critically, this prestimulus enhancement of alpha-amplitude is paralleled by psychophysical judgments of a reduced contrast for this stimulus. We suggest that alpha-oscillations in visual cortex preceding self-generated visual stimulation are a likely neurophysiological signature of motor-induced sensory anticipation and mediate sensory attenuation. We discuss our results in relation to proposals that attribute generic inhibitory functions to alpha-oscillations in prioritizing and gating sensory information via top–down control.

Attention to task-relevant features leads to a biasing of sensory selection in extrastriate cortex. Features signaling reward seem to produce a similar bias, but how modulatory effects due to reward and attention relate to each other is largely unexplored. To address this issue, it is critical to separate top–down settings defining reward relevance from those defining attention. To this end, we used a visual search paradigm in which the target's definition (attention to color) was dissociated from reward relevance by delivering monetary reward on search frames where a certain task-irrelevant color was combined with the target-defining color to form the target object. We assessed the state of neural biasing for the attended and reward-relevant color by analyzing the neuromagnetic brain response to asynchronously presented irrelevant distractor probes drawn in the target-defining color, the reward-relevant color, and a completely irrelevant color as a reference. We observed that for the prospect of moderate rewards, the target-defining color but not the reward-relevant color produced a selective enhancement of the neuromagnetic response between 180 and 280 msec in ventral extrastriate visual cortex. Increasing reward prospect caused a delayed attenuation (220–250 msec) of the response to reward probes, which followed a prior (160–180 msec) response enhancement in dorsal ACC. Notably, shorter latency responses in dorsal ACC were associated with stronger attenuation in extrastriate visual cortex. Finally, an analysis of the brain response to the search frames revealed that the presence of the reward-relevant color in search distractors elicited an enhanced response that was abolished after increasing reward size. The present data together indicate that when top–down definitions of reward relevance and attention are separated, the behavioral significance of reward-associated features is still rapidly coded in higher-level cortex areas, thereby commanding effective top–down inhibitory control to counter a selection bias for those features in extrastriate visual cortex.

Human observers can readily track up to four independently moving items simultaneously, even in the presence of moving distractors. Here we combined EEG and magnetoencephalography recordings to investigate the neural processes underlying this remarkable capability. Participants were instructed to track four of eight independently moving items for 3 sec. When the movement ceased a probe stimulus consisting of four items with a higher luminance was presented. The location of the probe items could correspond fully, partly, or not at all with the tracked items. Participants reported whether the probe items fully matched the tracked items or not. About half of the participants showed slower RTs and higher error rates with increasing correspondence between tracked items and the probe. The other half, however, showed faster RTs and lower error rates when the probe fully matched the tracked items. This latter behavioral pattern was associated with enhanced probe-evoked neural activity that was localized to the lateral occipital cortex in the time range 170–210 msec. This enhanced response in the object-selective lateral occipital cortex suggested that these participants performed the tracking task by visualizing the overall shape configuration defined by the vertices of the tracked items, thereby producing a behavioral advantage on full-match trials. In a later time range (270–310 msec) probe-evoked neural activity increased monotonically as a function of decreasing target–probe correspondence in all participants. This later modulation, localized to superior parietal cortex, was proposed to reflect the degree of mismatch between the probe and the automatically formed visual STM representation of the tracked items.

Perceptual decision-making performance depends on several cognitive and neural processes. Here, we fit Ratcliff's diffusion model to accuracy data and reaction-time distributions from one numerical and one verbal two-choice perceptual-decision task to deconstruct these performance measures into the rate of evidence accumulation (i.e., drift rate), response criterion setting (i.e., boundary separation), and peripheral aspects of performance (i.e., nondecision time). These theoretical processes are then related to individual differences in brain activation by means of multiple regression. The sample consisted of 24 younger and 15 older adults performing the task in fMRI before and after 100 daily 1-hr behavioral training sessions in a multitude of cognitive tasks. Results showed that individual differences in boundary separation were related to striatal activity, whereas differences in drift rate were related to activity in the inferior parietal lobe. These associations were not significantly modified by adult age or perceptual expertise. We conclude that the striatum is involved in regulating response thresholds, whereas the inferior parietal lobe might represent decision-making evidence related to letters and numbers.

Attention to one feature of an object can bias the processing of unattended features of that object. Here we demonstrate with ERPs in visual search that this object-based bias for an irrelevant feature also appears in an unattended object when it shares that feature with the target object. Specifically, we show that the ERP response elicited by a distractor object in one visual field is modulated as a function of whether a task-irrelevant color of that distractor is also present in the target object that is presented in the opposite visual field. Importantly, we find this modulation to arise with a delay of approximately 80 msec relative to the N2pc—a component of the ERP response that reflects the focusing of attention onto the target. In a second experiment, we demonstrate that this modulation reflects enhanced neural processing in the unattended object. These observations together facilitate the surprising conclusion that the object-based selection of irrelevant features is spatially global even after attention has selected the target object.

Perceptual decision-making performance depends on several cognitive and neural processes. Here, we fit Ratcliff's diffusion model to accuracy data and reaction-time distributions from one numerical and one verbal two-choice perceptual-decision task to deconstruct these performance measures into the rate of evidence accumulation (i.e., drift rate), response criterion setting (i.e., boundary separation), and peripheral aspects of performance (i.e., nondecision time). These theoretical processes are then related to individual differences in brain activation by means of multiple regression. The sample consisted of 24 younger and 15 older adults performing the task in fMRI before and after 100 daily 1-hr behavioral training sessions in a multitude of cognitive tasks. Results showed that individual differences in boundary separation were related to striatal activity, whereas differences in drift rate were related to activity in the inferior parietal lobe. These associations were not significantly modified by adult age or perceptual expertise. We conclude that the striatum is involved in regulating response thresholds, whereas the inferior parietal lobe might represent decision-making evidence related to letters and numbers.

Attention to one feature of an object can bias the processing of unattended features of that object. Here we demonstrate with ERPs in visual search that this object-based bias for an irrelevant feature also appears in an unattended object when it shares that feature with the target object. Specifically, we show that the ERP response elicited by a distractor object in one visual field is modulated as a function of whether a task-irrelevant color of that distractor is also present in the target object that is presented in the opposite visual field. Importantly, we find this modulation to arise with a delay of approximately 80 msec relative to the N2pc—a component of the ERP response that reflects the focusing of attention onto the target. In a second experiment, we demonstrate that this modulation reflects enhanced neural processing in the unattended object. These observations together facilitate the surprising conclusion that the object-based selection of irrelevant features is spatially global even after attention has selected the target object.

Effective adaptation to the demands of a changing environment requires flexible cognitive control. The medial and the lateral frontal cortices are involved in such control processes, putatively in close interplay with the BG. In particular, dopaminergic projections from the midbrain (i.e., from the substantia nigra [SN] and the ventral tegmental area) have been proposed to play a pivotal role in modulating the activity in these areas for cognitive control purposes. In that dopaminergic involvement has been strongly implicated in reinforcement learning, these ideas suggest functional links between reinforcement learning, where the outcome of actions shapes behavior over time, and cognitive control in a more general context, where no direct reward is involved. Here, we provide evidence from functional MRI in humans that activity in the SN predicts systematic subsequent trial-to-trial RT prolongations that are thought to reflect cognitive control in a stop-signal paradigm. In particular, variations in the activity level of the SN in one trial predicted the degree of RT prolongation on the subsequent trial, consistent with a modulating output signal from the SN being involved in enhancing cognitive control. This link between SN activity and subsequent behavioral adjustments lends support to theoretical accounts that propose dopaminergic control signals that shape behavior both in the presence and in the absence of direct reward. This SN-based modulatory mechanism is presumably mediated via a wider network that determines response speed in this task, including frontal and parietal control regions, along with the BG and the associated subthalamic nucleus.

Faces expressing fear may attract attention in an automatic bottom–up fashion. Here we address this issue with magneto-encephalographic (MEG) recordings in subjects performing a demanding visual search combined with the presentation of irrelevant neutral or fearful faces. The impact of the faces' emotional expression on attentional selection was assessed by analyzing the N2pc component—a modulation of the event-related magnetic field response known to reflect attentional focusing in visual search. We observed that lateralized fearful faces elicited an N2pc approximately between 240 and 400 msec in ventral extrastriate cortex that was independent of the N2pc reflecting target selection in visual search. Despite their clear influence on neural processing, fearful faces did not significantly influence behavioral performance. To clarify this discrepancy, we further performed an MEG experiment in which the demands of the search task were reduced. Under those conditions, lateralized fearful faces elicited an N2pc response that was again independent of the N2pc response to the search target. Behavioral performance was, however, influenced in a significant manner, suggesting that for behavioral effects to appear, sufficient attentional resources need to be left unoccupied by the search task—a notion put forward by the perceptual load theory. Our observations are taken to indicate that irrelevant fearful faces influence attentional processing in extrastriate visual cortex in an automatic fashion and independent of other task-relevant attentional operations. However, this may not necessarily be echoed at the behavioral level as long as task-relevant selection operations exhaust attentional resources.

Knowledge about cause and effect relationships (e.g., virus–epidemic) is essential for predicting changes in the environment and for anticipating the consequences of events and one's own actions. Although there is evidence that predictions and learning from prediction errors are instrumental in acquiring causal knowledge, it is unclear whether prediction error circuitry remains involved in the mental representation and evaluation of causal knowledge already stored in semantic memory. In an fMRI study, participants assessed whether pairs of words were causally related (e.g., virus–epidemic) or noncausally associated (e.g., emerald–ring). In a second fMRI study, a task cue prompted the participants to evaluate either the causal or the noncausal associative relationship between pairs of words. Causally related pairs elicited higher activity in OFC, amygdala, striatum, and substantia nigra/ventral tegmental area than noncausally associated pairs. These regions were also more activated by the causal than by the associative task cue. This network overlaps with the mesolimbic and mesocortical dopaminergic network known to code prediction errors, suggesting that prediction error processing might participate in assessments of causality even under conditions when it is not explicitly required to make predictions.

Exploring a novel environment can facilitate subsequent hippocampal long-term potentiation in animals. We report a related behavioral enhancement in humans. In two separate experiments, recollection and free recall, both measures of hippocampus-dependent memory formation, were enhanced for words studied after a 5-min exposure to unrelated novel as opposed to familiar images depicting indoor and outdoor scenes. With functional magnetic resonance imaging, the enhancement was predicted by specific activity patterns observed during novelty exposure in parahippocampal and dorsal prefrontal cortices, regions which are known to be linked to attentional orienting to novel stimuli and perceptual processing of scenes. Novelty was also associated with activation of the substantia nigra/ventral tegmental area of the midbrain and the hippocampus, but these activations did not correlate with contextual memory enhancement. These findings indicate remarkable parallels between contextual memory enhancement in humans and existing evidence regarding contextually enhanced hippocampal plasticity in animals. They provide specific behavioral clues to enhancing hippocampus-dependent memory in humans.

How do visual luminance, shape, motion, and depth bind together in the brain to represent the coherent percept of a 3-D object within hundreds of milliseconds (msec)? We provide evidence from simultaneous magnetoencephalographic (MEG) and electroencephalographic (EEG) data that perception of 3-D objects defined by luminance or motion elicits sequential activity in human visual cortices within 500 msec. Following activation of the primary visual cortex around 100 msec, 3-D objects elicited sequential activity with only little overlap (dynamic 3-D shapes: MT-LO-Temp; stationary 3-D shapes: LO-Temp). A delay of 80 msec, both in MEG/EEG responses and in reaction times (RTs), was found when additional motion information was processed. We also found significant positive correlations between RT, and MEG and EEG responses in the right temporal location. After about 400 msec, long-lasting activity was observed in the parietal cortex and concurrently in previously activated regions. Novel time-frequency analyses indicate that the activity in the lateral occipital (LO) complex is associated with an increase of induced power in the gamma band, a hallmark of binding. The close correspondence of an induced gamma response with concurrent sources located in the LO in both experimental conditions at different points in time (∼200 msec for luminance and ∼300 msec for dynamic cues) strongly suggests that the LO is the key region for the assembly of object features. The assembly is fed forward to achieve coherent perception of a 3-D object within 500 msec.

Bilingual individuals need effective mechanisms to prevent interference between their languages. Using event-related brain potentials (ERPs) and functional magnetic resonance imaging (fMRI), we present evidence for interference of phonological information from the nontarget language in German—Spanish bilinguals. A tacit picture-naming task was used in which bilinguals and monolinguals had to make speeded responses based on the first letter of the picture's name in the target language. In one condition, subjects were required to respond when the name began with a vowel and to withhold a response if it started with a consonant. Stimuli had been selected such that in half of the trials, the names in both languages necessitated the same response, whereas in the other half, responses were different for the two languages. For the bilinguals, the language in which the stimuli had to be named was changed after each block. Bilinguals showed phonological interference compared with monolingual performance, which was evident in their performance, ERPs, and fMRI patterns. Nonlanguage-specific brain areas such as the left middle prefrontal cortex were found to be crucial for the control of interference.

We addressed the hypothesis that perceptual priming and explicit memory have distinct neural correlates at encoding. Event-related potentials (ERPs) were recorded while participants studied visually presented words at deep versus shallow levels of processing (LOPs). The ERPs were sorted by whether or not participants later used studied words as completions to three-letter word stems in an intentional memory test, and by whether or not they indicated that these completions were remembered from the study list. Study trials from which words were later used and not remembered (primed trials) and study trials from which words were later used and remembered (remembered trials) were compared to study trials from which words were later not used (forgotten trials), in order to measure the ERP difference associated with later memory (DM effect). Primed trials involved an early (200–450 msec) centroparietal negative-going DM effect. Remembered trials involved a late (900–1200 msec) right frontal, positive-going DM effect regardless of LOP, as well as an earlier (600–800 msec) central, positive-going DM effect during shallow study processing only. All three DM effects differed topographically, and, in terms of their onset or duration, from the extended (600–1200 msec) fronto-central, positive-going shift for deep compared with shallow study processing. The results provide the first clear evidence that perceptual priming and explicit memory have distinct neural correlates at encoding, consistent with Tulving and Schacter's (1990) distinction between brain systems concerned with perceptual representation versus semantic and episodic memory. They also shed additional light on encoding processes associated with later explicit memory, by suggesting that brain processes influenced by LOP set the stage for other, at least partially separable, brain processes that are more directly related to encoding success.

Various prefrontal cortical regions have been shown to be activated during emotional stimulation, whereas neurochemical mechanisms underlying emotional processing in the prefrontal cortex remain unclear. We therefore investigated the influence of the GABA-A potentiator lorazepam on prefrontal cortical emotional—motor spatio-temporal activation pattern in a combined functional magnetic resonance imaging/magnetoencephalography study. Lorazepam led to the reversal in orbito-frontal activation pattern, a shift of the early magnetic field dipole from the orbito-frontal to medial prefrontal cortex, and alterations in premotor/motor cortical function during negative and positive emotional stimulation. It is concluded that negative emotional processing in the orbito-frontal cortex may be modulated either directly or indirectly by GABA-A receptors. Such a modulation of orbito-frontal cortical emotional function by lorazepam has to be distinguished from its effects on cortical motor function as being independent from the kind of processing either emotional or nonemotional.

Recent developments towards event-related functional magnetic resonance imaging has greatly extended the range of experimental designs. If the events occur in rapid succession, the corresponding time-locked responses overlap significantly and need to be deconvolved in order to separate the contributions of different events. Here we present a deconvolution approach, which is especially aimed at the analysis of fMRI data where sequence- or context-related responses are expected. For this purpose, we make the assumption of a hemodynamic response function (HDR) with constant yet not predefined shape but with possibly variable amplitudes. This approach reduces the number of variables to be estimated but still keeps the solutions flexible with respect to the shape. Consequently, statistical efficiency is improved. Temporal variations of the HDR strength are directly indicated by the amplitudes derived by the algorithm. Both the estimation efficiency and statistical inference are further supported by an improved estimation of the noise covariance. Using synthesized data sets, both differently shaped HDRs and varying amplitude factors were correctly identified. The gain in statistical sensitivity led to improved ratios of false- and true-positive detection rates for synthetic activations in these data. In an event-related fMRI experiment with a human subject, different HDR amplitudes could be derived corresponding to stimulation at different visual stimulus contrasts. Finally, in a visual spatial attention experiment we obtained different fMRI response amplitudes depending on the sequences of attention conditions.

In psycholinguistic research, there has been considerable interest in understanding the interactions of difFerent types of linguistic information during language processing. For example, does syntactic information interact with semantic or pragmatic information at an early stage of language processing, or only at later stages in order to resolve ambiguities of language? Developing reliable measures of language processes such as syntax and semantics is important to address many of these theoretical issues in psycholinguistics. In the present study, event-related brain potentials (ERPs) were recorded from healthy young subjects while they read pairs of words presented one word at a time. The ERPs for the second word of each pair were compared as a function of whether the preceding word was or was not (1) semantically related (i.e., synonyms; “semantic condition”) or (2) grammatically correct (“syntactic condition”). In the semantic condition the ERPs obtained to words preceded by nonsemantically related words elicited an N400 component that was maximal over centroparietal scalp regions. In contrast, in the syntactic condition the ERPs obtained to words preceded by grammatically incorrect articles or pronouns yielded a negativity with a later onset, and a frontopolar, left hemisphere scalp maximum. This replicates our previous findings of a syntactic negativity in a word pair design that was performed in the German language. Further, the present data provide scalp distributional information, which suggests that the syntactic negativity represents brain processes that are dissociable from the centroparietal N400 component. Thus, these findings provide strong evidence for a separate negative polarity ERP component that indexes syntactic aspects of language processing.