Traditional views link semantic context integration to neurophysiological activity at 300–500 msec. To study possible early dynamics related to semantic context integration, we recorded, in passive oddball paradigm, magnetic evoked responses to spoken word pairs, the second word being either congruent or incongruent with the first one. The same experimental words were placed in orthogonally varied context, thus providing a strict control for any effects of acoustic, phonological, and psycholinguistic stimulus features. Responses to the same critical words were obtained also outside of semantic context. We found that regardless of their acoustic features, semantically incongruent stimuli elicited a brain response already at ∼115 msec after the critical word onset. The same words did not produce such deflection in semantically legal context. The responses were maximal at left temporal and inferior frontal cortical sites, which was also confirmed by distributed current source analysis. The left temporal activation preceded the frontal one by ∼16 msec. No late response dynamics (>350 msec) were found that would reflect the semantic modulation in this nonattend passive design, indicating the possible role of attention in generating the later responses. Our results suggest that the earliest brain processes of semantic context integration can occur at ∼100 msec after the onset of spoken words in the left inferior frontal and superior temporal cortex.