How do comprehenders build up overall meaning representations of visual real-world events? This question was examined by recording event-related potentials (ERPs) while participants viewed short, silent movie clips depicting everyday events. In two experiments, it was demonstrated that presentation of the contextually inappropriate information in the movie endings evoked an anterior negativity. This effect was similar to the N400 component whose amplitude has been previously reported to inversely correlate with the strength of semantic relationship between the context and the eliciting stimulus in word and static picture paradigms. However, a second, somewhat later, ERP component—a posterior late positivity—was evoked specifically when target objects presented in the movie endings violated goal-related requirements of the action constrained by the scenario context (e.g., an electric iron that does not have a sharp-enough edge was used in place of a knife in a cutting bread scenario context). These findings suggest that comprehension of the visual real world might be mediated by two neurophysiologically distinct semantic integration mechanisms. The first mechanism, reflected by the anterior N400-like negativity, maps the incoming information onto the connections of various strengths between concepts in semantic memory. The second mechanism, reflected by the posterior late positivity, evaluates the incoming information against the discrete requirements of real-world actions. We suggest that there may be a tradeoff between these mechanisms in their utility for integrating across people, objects, and actions during event comprehension, in which the first mechanism is better suited for familiar situations, and the second mechanism is better suited for novel situations.

The aim of this study was to gain further insights into how the brain distinguishes between meaning and syntax during language comprehension. Participants read and made plausibility judgments on sentences that were plausible, morpho-syntactically anomalous, or pragmatically anomalous. In an event-related potential (ERP) experiment, morphosyntactic and pragmatic violations elicited significant P600 and N400 effects, respectively, replicating previous ERP studies that have established qualitative differences in processing conceptually and syntactic anomalies. Our main focus was a functional magnetic resonance imaging (fMRI) study in which the same subjects read the same sentences presented in the same pseudorandomized sequence while performing the same task as in the ERP experiment. Rapid-presentation event-related fMRI methods allowed us to estimate the hemodynamic response at successive temporal windows as the sentences unfolded word by word, without assumptions about the shape of the underlying response function. Relative to nonviolated sentences, the pragmatic anomalies were associated with an increased hemodynamic response in left temporal and inferior frontal regions and a decreased response in the right medial parietal cortex. Relative to nonviolated sentences, the morphosyntactic anomalies were associated with an increased response in bilateral medial and lateral parietal regions and a decreased response in left temporal and inferior frontal regions. Thus, overlapping neural networks were modulated in opposite directions to the two types of anomaly. These fMRI findings document both qualitative and quantitative differences in how the brain distinguishes between these two types of anomalies. This suggests that morphosyntactic and pragmatic information can be processed in different ways but by the same neural systems.